AI assistants for analysts

1) Definition and value

AI assistant for analysts is an interface (chat, panel in BI, IDE/SQL extension, voice) that translates natural language into correct analytical actions: writing SQL/DBT, explaining metrics, plotting, looking for anomalies, generating notes, experimental plans, etc.
Value: reducing time from question to insight, leveling expertise between teams, reducing the burden on senior analysts, improving the quality of documentation and reusing knowledge.

2) Key use cases

SQL copilot: generating/optimizing queries, explaining the execution plan, index hints.
BI-copilot: creating widgets/dashboards, auto-comments to graphs ("what has changed and why").
Data discovery: search for tables/metrics by glossary, link and activity.
Quality & observability: formation of data tests, triage of anomalies, proposal of fixes.
Experiments: A/B design, power calculation, results analysis, text reports.
ML acceleration: draft features/pipelines, model comparison, monitoring generation.
Documentation: summary of PR/diffuses in diagrams, auto-README for shop windows, Q&A by catalog.
Communications: designer of analytical notes, briefs and presentations.

3) Architectural patterns

1. RAG (Retrieval-Augmented Generation): LLM responds by relying on enterprise content (directories, schemas, glossary, SQL examples) extracted through vector/character search.
2. Tool agents: LLM calls tools (SQL execution, table profiling, plotting, dbt docs, Jira/GitHub, Slack) using the function protocol.
3. Guarded execution: sandbox, resource limits, dangerous request policy (DML prohibited, SELECT only), escalation to a person.
4. Semantic layer: uniform business metrics and dimensions as a source of truth; SQL generation by semantics, not by raw tables.
5. Cache and determinism: cache of prompts (prompt + context), fixing versions of models and data, reproducibility control.

4) Integrations and embedding points

DWH/OLAP: BigQuery, Snowflake, Redshift, ClickHouse; read-only roles, RLS/CLS.
BI/laptops: Looker/Power BI/Tableau/Metabase, Jupyter/VS Code; extensions/bots.
Catalogs/lineages: DataHub/Amundsen/Collibra; indexing definitions and owners.
Pipelines: dbt/Airflow/Argo/Prefect; generation of tests, descriptions, release notes.
Communications: Slack/Teams/Jira/Confluence; auto-posts of insights and tasks.

5) Security, access and compliance

Authentication/SSO: OIDC/SAML, SCIM for groups and roles.
RLS/CLS: filters by tenant/role/region; PII/PCI masking.
Query policy: whitelisting schemas, time/line limit, DDL/DML prohibition.
Audit and logging: who asked what, what data was viewed/exported.
Confidentiality in RAG: storage of corporate documents only; encryption; prohibition of external training on private data.
Regulatory: log retention, DSAR, storage localization in the right regions.

6) UX patterns and interaction

Chat + Tools: dialog with action buttons ("start SQL," "build a graph," "create a quality test").
Explainability: highlighting sources from which definitions/SQL fragments are taken; links to glossary and lineage.
Confirm & Run: Double confirmation before heavy requests, value/time estimate.
Few-shot examples: "show similar queries/guidelines" button.
Mentor mode: detailed explanations of why such a plan/method was chosen.
Accessibility: keyboard navigation, snippet copying with one click, export to Markdown/PDF.

7) Prompt-engineering (basic templates)

7. 1 Metric explanation

Task: Explain the <KPI> metric.
Output: definition, formula, table sources, owner, update window, caveats.
Format: short summary + markdown list.
Limitations: Rely only on the semantic layer and glossary.

7. 2 SQL Generation by Semantics

Context: semantic objects {metric: "conversion_rate", dims: ["country, "" channel"], time: "day"}.
Task: generate SELECT for BigQuery, considering RLS by region.
Check: limit of 2000 lines, sorted by date, filter for the last 90 days.

7. 3 A/B Test Plan

Business question <description>.
Deduce: hypothesis, metrics (primary/guardrail), MDE, power calculation, duration,
stratification, analysis plan (CUPED/permutation), stopping criteria.

8) Evaluation of quality (evals) and control of hallucinations

SQL-evals: comparing results with reference queries; equivalence check (delta threshold).
Doc-grounding: The assistant is required to quote the ID of the documents/metrics used in the response.
Linter rules: SQL style, 'SELECT' prohibition, mandatory time/tenant filters.
Negative tests: provocative requests ("give personal data" → refusal).
Red team: regular security/privacy scenarios.

9) Performance and cost

Caching: results of frequently repeated queries, embeddings, retrieved-chunks.
Token reduction: brief system prompts, aggressive relevant sampling.
Join and precompute pools: materialized storefronts for popular questions.
Budget-guards: quotas per user/team, cost-to-insight report.

10) MLOps and operation

Versioning: models, prompts, tools, RAG indexes - with version numbers and changelog.
Monitoring: latency, errors, share of responses with sources, frequency of manual SQL edits.
Incidents: folback mode (secure responses with links), quick rollback of prompts/models.
Releases: canary calculations; comparison "old assistant vs new" by business metrics.
Personnel training: guide for safe requests, anti-patterns, ethics.

11) Assistant Success Metrics

Adoption: MAU/WAU, share of active analysts, reuse.
Speed: median time to correct SQL/graph/response.
Quality: share of responses without edits, accuracy on eval sets, coverage with links to sources.
Economy: cost per insight/request, saving man-hours.
Business impact: uplift of the speed of report releases, reduction of SLA violations in analytics.

12) Antipatterns

Chat instead of data: lack of semantic layer and glossary → chaos in metrics.
Unrestricted rights: assistant access to sales without RLS/CLS and audit.
Hallucinations without grounding: responses without references and verifiable sources.
Absence of evals: releases "by eye," an increase in incidents.
Single-tenant prompts: hard-wired paths to schemes → pain when moving.
Only iframe-embedding: inability to call tools and do actions.

13) Implementation Roadmap

1. Discovery: list of analyst tasks, sources of truth (semantics/glossary), risks.
2. MVP: chat + SQL generation in 3-5 showcases, read-only access, RAG by glossary, basic evals.
3. Scale: tool agents (BI, dbt, Jira), example catalog, explainability, audit.
4. Hardening: negative tests, red-team, budget-guards, retentions of logs and DSAR.
5. Growth: personalization by role, auto-alerts/recommendations, voice interface, external partners.

14) Pre-release checklist

• Connected SSO, roles/groups, RLS/CLS and PII masking.
• Semantic layer and glossary cover MVP KPI, there are owners.
• Queries are schema/quota restricted, DML/DDL is not allowed.
• Evals: SQL reference/response set, quality thresholds, and alerts.
• Logs and auditing enabled; incident plan and folback mode is ready.
• UX: heavy operation confirmation, sources in responses, export to Markdown/PDF.
• User documentation: Prompt guide, anti-patterns, examples.

15) Examples of "live" prompts for the assistant

"Find the 90-day conversion charts for the TR region, explain the formulas."

"Generate SQL: p95 latency by service X, by day, filter by prod traffic, up to 2k rows."

"Plot the ARPPU by channel, explain the anomalies, make a conclusion in 5 theses."

"Make an A/B plan for the new bonus mechanics: metrics, MDE, power, guardrails."

"Create quality tests for the payments showcase: freshness ≤ 30 minutes, uniqueness txn_id."

Bottom line: AI assistants for analysts are not smart chat, but a managed knowledge and tool platform. Their value is manifested when there is a semantic layer, strict accesses, eval-process and integration into working tools. Then the assistant really reduces the time to insight and improves the quality of solutions.