Saturday, September 19, 2026

RAG — Interview Field Notes
AI SYSTEMS · INTERVIEW FIELD NOTES

RAG — the little library that teaches an LLM what it didn't know.

Don't memorize RAG buzzwords. Understand the journey: knowledge → retrieval → evidence → generation → evaluation → production learning.

Retrieval-Augmented Generation Production RAG GraphRAG Agentic RAG Evaluation
01

Why RAG?

๐Ÿง 

LLM knowledge

Knowledge stored inside model parameters is useful, but private or frequently changing enterprise information isn't automatically there.

๐Ÿ“š

External knowledge

PDFs, policies, tickets, Git, databases and internal wikis live outside the model and change continuously.

Don't make the model memorize the library. Give it a library card.
RETRIEVE ONLY THE EVIDENCE NEEDED FOR THIS QUESTION
02

The entire RAG story — one flow

flowchart LR %% ================================ %% KNOWLEDGE PLANE %% ================================ subgraph ING["๐Ÿ“š KNOWLEDGE PLANE"] A["Documents
PDF • Wiki • Git • DB"] B["Parse / OCR
Layout"] C["Chunk"] D["Embed"] E["Search Index
Vector + BM25"] A --> B B --> C C --> D D --> E end %% ================================ %% QUERY %% ================================ Q["๐Ÿ‘ค USER
QUESTION"] R["Query
Understanding"] Q --> R %% ================================ %% RETRIEVAL %% ================================ subgraph RET["๐Ÿ”Ž RETRIEVAL PLANE"] BM["BM25
Exact terms"] VS["Vector Search
Meaning"] HY["Hybrid
Fusion"] RR["Reranker
Top evidence"] BM --> HY VS --> HY HY --> RR end R --> BM R --> VS E --> BM E --> VS %% ================================ %% SECURITY %% ================================ SEC["๐Ÿ” Auth / RBAC
Tenant Filter"] R --> SEC SEC --> BM SEC --> VS %% ================================ %% GENERATION %% ================================ RR --> CTX["๐Ÿงฉ Context Builder
chunks + metadata"] CTX --> LLM["๐Ÿค– LLM
generate from evidence"] LLM --> OUT["๐Ÿ’ฌ Grounded Answer
+ citations"] %% ================================ %% ADDITIONAL KNOWLEDGE %% ================================ LLM -. "need another source?" .-> G["GraphRAG"] LLM -.-> SQL["SQL"] LLM -.-> API["Live API"] G --> CTX SQL --> CTX API --> CTX %% ================================ %% PRODUCTION LEARNING %% ================================ OUT --> TRACE["๐Ÿ”ฌ Production Trace"] TRACE --> SAMPLE["Real Query Sample"] SAMPLE --> FIX["๐Ÿ”ง Error Analysis"] FIX --> EVAL["๐Ÿ“Š Regression / Golden Eval"] EVAL -. "release gate" .-> R %% ================================ %% STYLES %% ================================ classDef source fill:#eaf2ff,stroke:#155dcc,color:#152033,stroke-width:2px classDef retrieve fill:#e8f8fa,stroke:#007f91,color:#152033,stroke-width:2px classDef generate fill:#f1ecff,stroke:#6842b8,color:#152033,stroke-width:2px classDef security fill:#fdebf4,stroke:#ad2864,color:#152033,stroke-width:2px classDef ops fill:#e8f8f1,stroke:#087b55,color:#152033,stroke-width:2px class A,B,C,D,E source class BM,VS,HY,RR retrieve class CTX,LLM,OUT,G,SQL,API generate class SEC security class TRACE,SAMPLE,FIX,EVAL ops
↔ Scroll horizontally to explore the full RAG architecture ↔
03

First understand the building blocks

01 · Tokenization

Text → tokens

LLMs don't directly read words. Text is split into tokens using methods such as BPE, WordPiece or SentencePiece.

02 · Embedding

Meaning → vector

Text becomes numbers in a vector space. Similar meanings tend to occupy nearby regions.

03 · Chunking

Document → evidence

Large documents are split into retrievable units while preserving enough local context.

04 · Metadata

Evidence → control

Store document ID, version, page, tenant, permissions, timestamp and embedding version.

05 · Index

Evidence → searchable

Store chunks so the system can quickly find candidates using lexical and semantic search.

06 · Context

Evidence → prompt

Retrieved chunks are assembled into a controlled context window before generation.

Bad parsing → bad chunks → bad embeddings → bad retrieval → bad answer.
RAG QUALITY STARTS BEFORE THE LLM
04

Retrieval is the real game

BM25

Finds important words and exact terms.

CVE-2026-12345
INC-48291
SAP-PRD-001

Best for: identifiers, names and exact terminology.

Vector Search

Finds semantic similarity.

"work from home"
≈
"remote work"

Best for: meaning, paraphrases and concepts.

BM25
+
Vector
→
Hybrid
→
Rerank
→
Top-K

Two-stage retrieval: cheap retrieval creates a broad candidate set; the reranker spends more computation choosing the strongest evidence.

05

When basic RAG isn't enough

Query

Query rewriting

Transform vague questions into queries that are easier for the retrieval system to understand.

Recall

Multi-query

Search the same question from multiple perspectives, merge results and rerank them.

Retrieval

HyDE

Create a hypothetical answer, embed it, then search for real supporting documents.

Context

Parent-child retrieval

Find a precise child chunk but return its larger parent context to the model.

Security

Metadata filtering

Filter tenant, RBAC, version, department and language before generation.

Knowledge

Graph / SQL / API

Graph for relationships, SQL for structured facts and APIs for live information.

06

RAG → Agentic RAG

Basic RAG

Fixed retrieval path.

Question
↓
Search
↓
Context
↓
Answer

Agentic RAG

The system decides which tool or retrieval path is needed.

Question
↓
Decide
↓
Vector / Graph / SQL / API
↓
Enough evidence?
↓
Answer
07

Golden data = the answer key

๐Ÿ† Golden evaluation record
{ "question": "What is the refund period?", "ground_truth": "30 days", "relevant_chunks": ["refund_policy_18"], "expected_citation": "refund_policy.pdf#page=4" }

Golden data is a trusted benchmark . It tells you whether a new RAG version still works.

Recall@K Did the right evidence appear?
MRR / NDCG Was relevant evidence ranked highly?
Faithfulness Is the answer grounded in evidence?
Answer relevance Did the system answer the question?
Citation correctness Does the citation support the claim?
Latency / Cost Can the system operate efficiently?
08

The production learning loop

Golden Set → Offline Eval → Deploy → Production Traces → Real Queries → Evaluate → Error Analysis → Fix → Regression Case → Golden Set

PRODUCTION FAILURES BECOME FUTURE TEST CASES

09

Golden data ≠ production tracing

๐Ÿ† Golden data

Controlled benchmark.

Known question
+
Known truth

Answers:
“How good is this version?”

๐Ÿ”ฌ Production tracing

Runtime microscope.

User
→ Rewrite
→ Retrieve
→ Rerank
→ LLM
→ Answer

Answers:
“What actually happened?”

10

Three production rules people forget

Security

Authorize before retrieval

Apply RBAC / ABAC / tenant filters before evidence reaches the model.

Safety

Retrieved text is untrusted

Documents can contain prompt injection. Retrieved text is evidence, not automatically instructions.

Freshness

Version everything

Track document, index, chunking and embedding-model versions.

11

Production checklist

Parsing / OCR
Chunking
Embeddings
Vector index
BM25
Hybrid retrieval
Reranking
Metadata filters
RBAC
Versioning
Incremental indexing
Caching
Tracing
Golden evaluation
Regression tests
Latency
Token cost
Prompt injection defense
12

The answer to remember

๐ŸŽ™️ 30-second interview answer

“RAG separates knowledge from the model. Instead of expecting an LLM to memorize private or frequently changing information, we ingest external sources, parse and chunk them, create embeddings, and index them for retrieval. At query time we can combine semantic and lexical retrieval, apply metadata and security filters, rerank candidates, and give the most relevant evidence to the LLM for grounded generation. In production I’d also handle incremental ingestion, document and embedding versioning, authorization, prompt-injection risks, observability, golden-data evaluation, regression testing, latency and cost.”

Knowledge → retrieve → hybrid → rerank → evidence → generate → evaluate → trace → learn → regression test.
REMEMBER THE STORY · NOT THE BUZZWORDS
RAG INTERVIEW FIELD NOTES · KNOWLEDGE → RETRIEVAL → GENERATION → EVALUATION → PRODUCTION

No comments:

Post a Comment