Rag vector search

Declining to answer when the retrieved evidence is insufficient, for example by replying that the information was not found. It is a desired behaviour in RAG, since a wrong confident answer is usually worse than no answer.

Example. When no chunk scores above the threshold, the assistant says 'I could not find this in the policy documents'.

Related: Similarity threshold, Grounding, Hallucination

Enforcing who may see which documents at search time, so restricted content is never retrieved for unauthorised users. Permissions are typically stored as metadata and applied as filters before results reach the model.

Example. An intern's query cannot retrieve chunks tagged 'confidential: finance', because the filter uses the intern's role.

Related: Metadata filtering, Multi-tenancy, Pre-filtering

A RAG design where a language model acts as an agent that decides whether to search, which source or tool to use, how to phrase queries and whether to search again. It handles harder questions than a fixed retrieve-then-generate pipeline, with more latency and cost.

Example. An agent chooses to query the HR wiki first, finds it lacking, and then queries the payroll database.

Related: Iterative retrieval, Multi-hop retrieval, Query decomposition

A single parameter in hybrid search, usually between 0 and 1, that sets the balance between vector and keyword scores. An alpha of 1 uses only vector scores, 0 only keyword scores, and tuning it on real queries improves results.

Example. With alpha = 0.7, the score is 0.7 times the semantic score plus 0.3 times the keyword score.

Related: Score fusion, Hybrid search, BM25

The fraction of the true nearest neighbours that an approximate index actually returns, measured against exact search results. It quantifies the accuracy lost for speed, and must be reported with the latency at which it was achieved.

Example. An index returning 9 of the true 10 nearest vectors has recall@10 of 0.9.

Related: Approximate nearest neighbour search, Recall-latency trade-off, Flat index

A measure of how close a generated answer is to a trusted reference answer, combining factual overlap and sometimes semantic similarity. It needs labelled ground truth, unlike faithfulness, which needs only the context.

Example. The reference says 'Rs 500' and the answer says 'Rs 500', so correctness is high.

Related: Faithfulness, Answer relevance, RAG evaluation

A RAG evaluation measure of how directly the generated answer addresses the user's question, regardless of whether it is factually right. Off-topic, evasive or padded answers score low, even when everything they say is supported by the context.

Example. For 'What is the notice period?', an answer describing the company history scores low.

Related: Faithfulness, Context relevance, RAG evaluation

Search that returns vectors that are very likely, but not guaranteed, to be the closest ones, using an index to skip most comparisons. It makes searching hundreds of millions of vectors feasible in milliseconds, at the cost of occasionally missing a true neighbour.

Example. An ANN index returns 9 of the true top 10 neighbours in 3 ms, while brute force needs 900 ms.

Related: HNSW, IVF, ANN recall

A retrieval setting where queries are short, such as a question, and documents are long, such as a passage, so they differ in form. Some embedding models expect query and document prefixes to handle this properly.

Example. A model card says to prefix queries with 'query:' and passages with 'passage:' before embedding.

Related: Embedding model, Bi-encoder, Dense retrieval

Linking each claim in a generated answer to the specific evidence it draws on, and checking that the evidence really supports it. Evaluating attribution catches answers that cite a source which does not say what is claimed.

Example. A checker tests whether the cited paragraph on 'notice period' actually states 60 days.

Related: Citation, Faithfulness, Grounding

Reducing every dimension of an embedding to a single bit, usually its sign, so vectors can be compared with very fast Hamming distance. It gives roughly 32-fold memory savings, and is often followed by rescoring with full vectors.

Example. A 1,024-dimension float vector becomes 128 bytes, and 10 million vectors fit in about 1.3 GB.

Related: Scalar quantisation, Hamming distance, Rescoring

A contiguous piece of a document, often a few sentences to a few paragraphs, that is embedded and stored as one retrievable unit. Chunks must be small enough to match a specific question and large enough to carry meaning by themselves.

Example. A 30-page policy is cut into 60 chunks of about 400 tokens, each stored with its page number and section title.

Related: Chunk size, Chunk overlap, Embedding

Repeating a small number of tokens at the boundary of neighbouring chunks so that a sentence or idea cut by a split still appears whole in at least one chunk. It reduces lost context at boundaries but increases index size and duplicate results.

Example. With 500-token chunks and 50-token overlap, the last 50 tokens of chunk 3 are also the first 50 tokens of chunk 4.

Related: Chunk size, Chunking, Sliding window chunking

The target length of each chunk, measured in tokens, characters or sentences. Small chunks give precise matches but lose surrounding context; large chunks keep context but dilute the embedding and use more of the model's context window. It is one of the most tuned RAG settings.

Example. Moving from 1,000-token chunks to 300-token chunks raised the share of questions whose answer appeared in the top 5 results from 68% to 74% on one test set.

Related: Chunk overlap, Chunking, Context window

A pointer from a generated statement to the source passage that supports it, such as a document title and page. Citations let users verify answers and let developers check that claims are grounded.

Example. The answer ends with '[HR Policy, p. 12]' pointing to the chunk used.

Related: Grounding, Attribution, Faithfulness

A retrieval model using late interaction: it encodes queries and documents into per-token vectors and scores with a maximum-similarity operation. It offers accuracy near cross-encoders with far faster search, at the cost of larger index storage.

Example. A ColBERT index stores about 100 small vectors for a 100-token passage, instead of one.

Related: Late interaction, Multi-vector retrieval, Bi-encoder

Choosing the sequence in which retrieved passages appear in the prompt. Because position can affect how well a model uses a passage, systems often place the highest-ranked ones first or last.

Example. The reranked top passage goes first, the second-best goes last, and weaker ones sit in between.

Related: Lost in the middle, Reranking, Prompt template

A RAG evaluation measure of whether relevant chunks are ranked above irrelevant ones in the retrieved list. It rewards putting useful passages at the top, which matters because position affects model use.

Example. Retrieved order relevant, irrelevant, relevant scores better than irrelevant, irrelevant, relevant.

Related: Context recall, Precision at k, Context relevance

A RAG evaluation measure of how much of the information needed for the reference answer is present in the retrieved context. Low recall means the retriever missed key facts, so the generator cannot answer completely.

Example. If a reference answer has 4 facts and the context contains 3, context recall is 0.75.

Related: Context precision, Recall at k, Faithfulness

A RAG evaluation measure of how much of the retrieved context is actually useful for answering the question. Low values mean the retriever is fetching noise that wastes tokens and may distract the model.

Example. If 2 of 5 retrieved chunks bear on the question, context relevance is about 0.4.

Related: Context precision, Faithfulness, Retrieval grading

Adding context to each chunk before embedding, such as the document title, section heading or a short generated summary of where the chunk sits. It helps chunks that lose meaning when isolated, like a paragraph that says only 'the fee is 2%'.

Example. The chunk 'The fee is 2%' is prefixed with 'HDFC credit card agreement, Foreign transactions:' before embedding.

Related: Metadata, Chunking, Embedding

An approach where each chunk is prepended with a short, model-written explanation of its place in the document before indexing, applied to both embeddings and keyword indexes. It reduces failures caused by chunks that lack context.

Example. The chunk 'Revenue rose 3%' is prefixed with 'From ACME's Q2 2024 report, Asia segment'.

Related: Contextual chunk enrichment, Hybrid search, Chunking

Retrieval-based question answering applied to multi-turn chat, where each new message may depend on earlier turns. Systems rewrite follow-ups into standalone queries and manage history so that retrieval sees the user's real intent.

Example. The user asks 'What about for seniors?' and the system rewrites it as 'Fixed deposit interest rates for senior citizens'.

Related: Query rewriting, Retrieval-augmented generation, Context window

A RAG variant that evaluates the quality of retrieved passages and, if they look irrelevant or weak, triggers a fallback such as a web search or query rewrite before generating. It aims to stop poor retrieval from producing poor answers.

Example. If a grader marks all 5 retrieved chunks as off-topic, the system reruns the search with a rewritten query.

Related: Self-RAG, Retrieval grading, Agentic RAG

A distance measure defined as 1 minus cosine similarity, so vectors pointing the same way have distance 0. Many vector databases report it instead of similarity, and smaller values mean the items are closer in meaning.

Example. Two vectors with cosine similarity 0.8 have cosine distance 0.2.

Related: Cosine similarity, Euclidean distance, Vector index

Finding documents by comparing the dense embedding of a query with dense embeddings of documents, so matches are based on meaning. It handles paraphrases well but can miss exact identifiers or rare terms.

Example. The query 'how to reset my password' retrieves an article titled 'Recover account access' with no shared keywords.

Related: Sparse retrieval, Bi-encoder, Hybrid search

A graph-based approximate nearest-neighbour method designed to keep the bulk of the index on SSD storage while holding only compressed vectors in memory. It lets a single machine serve very large collections at low cost.

Example. A billion-vector collection is served from one machine with a small RAM footprint by keeping the graph on NVMe disk.

Related: HNSW, Vector quantisation, Approximate nearest neighbour search

A component that reads raw sources such as PDFs, web pages, spreadsheets or database exports and converts them into plain text plus metadata for a RAG pipeline. Good loaders preserve structure like headings and tables, since lost structure makes later chunks harder to retrieve.

Example. A PDF loader extracts text from a 40-page annual report and records page numbers as metadata for each section.

Related: Ingestion pipeline, Text extraction, Metadata

Improving an embedding model for a specialised field, such as law or medicine, by further training on in-domain text or query-passage pairs. It helps when general models do not separate specialist terms well.

Example. A model fine-tuned on 20,000 Indian legal query-passage pairs retrieves case-law paragraphs more accurately.

Related: Contrastive learning, Embedding model, Hard negative mining

In HNSW, the size of the candidate list examined while inserting each vector during index building. Higher values produce a better-connected graph and higher recall, but make the build slower.

Example. Setting efConstruction to 200 instead of 100 doubles build time but improves the recall of the finished index.

Related: HNSW, M parameter, efSearch

In HNSW, the size of the candidate list kept during a query. Larger values explore more of the graph, raising recall and latency together, so it is the main knob for tuning speed versus accuracy at query time.

Example. Increasing efSearch from 50 to 200 lifts recall@10 from 92% to 98% while latency roughly doubles.

Related: HNSW, ANN recall, Recall-latency trade-off

A list of numbers that represents a piece of content such as text or an image so that similar content has nearby lists. Models learn these vectors from data, and search systems compare them to find related items by meaning rather than exact words.

Example. The sentences 'cheap flights to Goa' and 'low-cost air tickets to Goa' receive vectors with cosine similarity around 0.9.

Related: Embedding model, Vector, Cosine similarity

Storing computed embeddings keyed by the input text, so repeated inputs need not be embedded again. It cuts cost and latency during ingestion, when most chunks are unchanged, and for frequently repeated user queries.

Example. A hash of each chunk's text is used to skip embedding the 90% of chunks that did not change.

Related: Semantic cache, Ingestion pipeline, Embedding

A mismatch that appears when embeddings in an index were made with one model or version and queries with another, or when data changes so much that old vectors no longer represent it well. It degrades retrieval, and the fix is re-embedding.

Example. After switching embedding models for queries only, recall drops sharply since the index still holds old-model vectors.

Related: Re-embedding, Embedding model, Index refresh

Brute-force search that computes the distance from the query to every stored vector and returns the true closest ones. It is perfectly accurate and fine for small collections, but its cost grows linearly with the collection size.

Example. Comparing a query against 50,000 vectors takes a few milliseconds, so exact search is fine.

Related: Flat index, Approximate nearest neighbour search, Nearest neighbour search

An open-source library from Meta for similarity search and clustering of dense vectors. It offers flat, IVF, HNSW and quantised indexes, runs on CPU and GPU, and is widely used as a building block for retrieval.

Example. A prototype builds a FAISS IVF index over 500,000 chunk embeddings and queries it from a Python script.

Related: Vector index, IVF, HNSW

A RAG evaluation measure of whether every claim in the answer is supported by the retrieved context. It is usually computed as supported claims divided by total claims, and detects made-up content even when the answer sounds plausible.

Example. If 8 of 10 claims in an answer are supported by the context, faithfulness = 8 / 10 = 0.8.

Related: Answer relevance, Context relevance, Hallucination

Splitting text into pieces of a set length regardless of content. It is simple and predictable, but it can cut sentences or sections in half, so it is often paired with overlap or replaced by methods that respect structure.

Example. A script slices every document into consecutive 512-token windows with no regard for headings.

Related: Chunking, Recursive character splitting, Chunk overlap

An index that stores vectors as-is and searches by comparing the query to every one. It gives exact results with no build step, and is a baseline for measuring how accurate approximate indexes are.

Example. A flat index over 100,000 vectors is used to compute ground-truth neighbours for testing an HNSW index.

Related: Exact nearest neighbour search, Vector index, ANN recall

Searching the entire text of documents for words or phrases, usually with an inverted index, stemming and ranking. It is the classic keyword engine that hybrid RAG systems combine with vector search.

Example. Searching 'refund policy' in a database's full-text engine returns pages ranked by term frequency.

Related: Inverted index, BM25, Hybrid search

A curated set of questions paired with verified correct answers and, ideally, the passages that support them. It serves as a stable benchmark for comparing RAG configurations and catching regressions.

Example. A team writes 150 real customer questions with reviewed answers and source pages, and reruns the set after each change.

Related: RAG evaluation, Synthetic test set, Retrieval evaluation

A family of RAG methods that build a knowledge graph of entities and relationships from documents, often with summaries of graph communities, and retrieve from it. It suits questions about connections or themes across many documents.

Example. From 1,000 news articles, a graph links companies to their founders and investors, letting the system answer 'which investors back both startups?'.

Related: Knowledge graph, Multi-hop retrieval, RAPTOR

When a language model states something fluent but unsupported or false, such as an invented figure or citation. RAG reduces it by supplying evidence, but the model can still add claims that the retrieved text does not contain.

Example. The bot says the refund window is 45 days, but the retrieved policy says 30.

Related: Faithfulness, Grounding, Citation

The number of positions at which two equal-length binary strings differ. It is very fast to compute on bit vectors, making it useful for comparing binary-quantised embeddings or hash signatures.

Example. Comparing 10110 and 10011 gives Hamming distance 2.

Related: Binary quantisation, Locality-sensitive hashing, Euclidean distance

Storing text at several granularities, for example document, section and paragraph, linked in a tree. Retrieval can match at one level and move up or down to fetch more or less context as needed.

Example. A report is indexed as 1 document node, 12 section nodes and 150 paragraph nodes, each pointing to its parent.

Related: Parent-document retrieval, Chunking, RAPTOR

Hierarchical Navigable Small World, a graph-based index where each vector links to nearby vectors across several layers. Search starts at a sparse top layer and greedily hops toward the query, descending to denser layers. It gives fast, accurate search but uses substantial memory.

Example. With HNSW, a query over 10 million vectors visits roughly a few hundred nodes rather than all 10 million.

Related: Approximate nearest neighbour search, efSearch, M parameter

Combining keyword-based and vector-based retrieval and merging their results into a single ranked list. Keyword search catches exact terms and identifiers, while vector search catches paraphrases, so the mix usually beats either alone.

Example. A query 'iPhone 15 battery drain' gets exact model matches from BM25 and related troubleshooting articles from embeddings.

Related: BM25, Dense retrieval, Reciprocal rank fusion

Hypothetical Document Embeddings: a model first writes a plausible, possibly imperfect, answer to the question, and that text is embedded and used as the search query. Answer-like text tends to sit closer to real answer passages than a short question does.

Example. For 'Why is my Wi-Fi slow at night?', the system generates a short paragraph about congestion and searches with its embedding.

Related: Query rewriting, Embedding, Dense retrieval

Updating a search index so it reflects changes in the source data: new, edited or deleted documents. Stale indexes cause outdated answers, so systems schedule refreshes or stream updates, and track document versions.

Example. A nightly job compares file hashes and re-embeds only the 40 documents that changed that day.

Related: Upsert, Ingestion pipeline, Knowledge base

The offline process that prepares source material for retrieval: loading files, extracting text, cleaning it, splitting into chunks, attaching metadata, computing embeddings and writing them to an index. Its design decides what the retriever can ever find, so errors here silently hurt every later answer.

Example. A nightly job loads new PDFs from cloud storage, parses them, splits them into 500-token chunks, embeds them and upserts them into a vector database.

Related: Chunking, Document loader, Upsert, Knowledge base

A data structure that maps each term to the list of documents containing it, like the index at the back of a book. It makes keyword lookups fast and is the basis of search engines and BM25 scoring.

Example. The term 'GST' points to documents 4, 19 and 203, so a query for 'GST' reads only those three.

Related: BM25, Sparse retrieval, Full-text search

Inverted file index: vectors are grouped into clusters using k-means, and each query is compared only with vectors in the few clusters whose centres are nearest. It cuts the work per query but can miss neighbours that sit in unsearched clusters.

Example. With 1,000 clusters and nprobe = 10, a query scans about 1% of the vectors.

Related: nprobe, Product quantisation, k-means clustering

The collection of documents or records a RAG system searches, such as manuals, wiki pages, tickets or database rows. It is usually cleaned, split into chunks, embedded and indexed ahead of time so lookups are fast at question time.

Example. A telecom's knowledge base holds 12,000 support articles in English and Hindi, re-indexed every night.

Related: Ingestion pipeline, Chunking, Vector index

A structured store of entities as nodes and their relationships as labelled edges, such as 'Infosys - headquartered in - Bengaluru'. It supports precise, connected queries that plain text similarity struggles with.

Example. A graph links the drug 'metformin' to the condition 'type 2 diabetes' through a 'treats' edge.

Related: GraphRAG, Entity linking, Multi-hop retrieval

A retrieval design that keeps one vector per token for queries and documents, and scores a pair by matching each query token to its best document token and summing. It is more expressive than a single vector while allowing precomputed document representations.

Example. A query of 8 tokens is matched against each document's stored token vectors and the 8 best matches are added up.

Related: ColBERT, Bi-encoder, Multi-vector retrieval

Document parsing that detects headings, paragraphs, tables, lists and figures on a page instead of reading raw text streams. Keeping this structure lets chunkers split at natural boundaries and lets tables be stored as tables, which improves retrieval on reports and forms.

Example. A parser recognises a balance-sheet table in an annual report and stores it as a Markdown table instead of scrambled numbers in a line.

Related: Text extraction, Table extraction, Structure-aware chunking

Retrieval where a neural model assigns weights to vocabulary terms, including expansion terms not in the text, producing a sparse vector usable with an inverted index. It blends keyword-style exactness with some semantic matching.

Example. SPLADE-style models add related words such as 'automobile' to a passage about 'car' with learned weights.

Related: Sparse retrieval, BM25, Inverted index

Using a language model to grade another model's outputs against criteria such as faithfulness or relevance. It scales evaluation beyond manual review, though judges have biases and should be checked against human labels.

Example. A judge model reads the context and answer and marks each claim 'supported' or 'unsupported'.

Related: Faithfulness, RAG evaluation, Golden dataset

An approximate search technique using hash functions designed so that similar vectors collide in the same bucket with high probability. A query checks only items in its matching buckets. It has theoretical guarantees but is often outperformed in practice by graph indexes.

Example. Random-hyperplane hashing turns each vector into a 64-bit signature, and vectors with equal signature prefixes are compared.

Related: Approximate nearest neighbour search, Hamming distance, Vector index

In HNSW, the maximum number of links each node keeps to its neighbours. Larger values build a denser graph that improves recall and robustness but increases memory use and index build time.

Example. Raising M from 16 to 32 increases graph memory roughly proportionally and improves recall at a given search effort.

Related: HNSW, efConstruction, efSearch

A selection method that picks results that are both relevant to the query and different from those already chosen, using a weight between the two. It reduces near-duplicate passages and increases the variety of information sent to the model.

Example. With lambda = 0.7, the second pick is a passage on a new sub-topic rather than a copy of the first.

Related: Top-k retrieval, Diversity, Reranking

Restricting a vector search to records whose metadata matches conditions, such as a date range, language or department. It narrows the candidate set to relevant, permitted items and can be applied before or after the similarity search.

Example. A query for 'leave policy' is limited to chunks where department = 'HR' and year >= 2024.

Related: Pre-filtering, Post-filtering, Metadata

Answering a question that needs several pieces of evidence chained together, where each retrieval step depends on what earlier steps found. It is harder than single-step lookup because the second query is unknown until the first result is read.

Example. To find 'the CEO of the company that acquired Flipkart', first retrieve the acquirer, then search for its CEO.

Related: Query decomposition, Agentic RAG, Iterative retrieval

Generating several rephrasings of one question, running retrieval for each and merging the results into a single deduplicated list. It reduces sensitivity to one particular wording and improves coverage of the relevant documents in the collection.

Example. A model writes 3 variants of 'ways to reduce cloud spend', and the union of their top 5 results is deduplicated.

Related: Query rewriting, Reciprocal rank fusion, Query expansion

Serving many separate customers from one shared system while keeping each one's data isolated. In RAG, this means every retrieval is restricted to the requesting tenant's documents, via namespaces or filters, to prevent data leaks between customers.

Example. A SaaS help-desk tool serves 200 companies and tags every vector with its company id, enforcing that filter on each search.

Related: Namespace, Metadata filtering, Access control in retrieval

Representing each document by several vectors, such as one per token, sentence or aspect, rather than a single one. It captures more detail and can improve matching, but needs more storage and a scoring rule that combines vectors.

Example. A long article is stored as 6 section vectors, and its score is the best match among them.

Related: Late interaction, ColBERT, Embedding

Searching across different content types, such as text and images, by embedding them into a shared space so a text query can find images or the reverse. It powers image search and RAG over slides or scanned pages.

Example. A query 'red silk saree' returns product photos whose image embeddings are near the text embedding.

Related: Image embedding, Embedding, Vector database

The task of finding the stored items whose vectors are closest to a query vector under a chosen distance measure. Exact search compares against everything; approximate search trades a little accuracy for large speed gains.

Example. Given a query vector, return the 5 stored vectors with the smallest Euclidean distance.

Related: Approximate nearest neighbour search, Exact nearest neighbour search, Top-k retrieval

In IVF indexes, the number of clusters searched for each query. Raising it examines more vectors, which improves the chance of finding true neighbours but slows the search, so it is tuned to balance recall against latency for the product.

Example. Moving nprobe from 5 to 40 raised recall@10 from 80% to 96% but made queries about 6 times slower.

Related: IVF, ANN recall, Recall-latency trade-off

A retrieval design where small chunks are used for matching but the larger parent section or document they came from is returned to the model. It combines the precise matching of small chunks with the fuller context of big ones.

Example. A 150-token child chunk matches the query, and the system returns its whole 1,200-token parent section to the generator.

Related: Small-to-big retrieval, Chunking, Sentence window retrieval

An open-source extension that adds a vector column type and similarity operators to PostgreSQL, with exact search and approximate indexes such as HNSW and IVF. It lets teams keep embeddings next to relational data in one database.

Example. A startup stores product rows and their 768-dimension embeddings in one Postgres table and queries by cosine distance with SQL.

Related: Vector database, HNSW, Metadata filtering

Running the nearest-neighbour search first and then discarding results that fail the metadata condition. It is simple, but a strict filter can leave fewer than the requested number of results, since many top hits get thrown away.

Example. The search returns 10 hits, but only 2 pass the language = 'hi' filter, so the user sees 2 results.

Related: Pre-filtering, Metadata filtering, Top-k retrieval

Applying metadata conditions before or during the nearest-neighbour search so only allowed items are considered. Results always honour the filter, but very selective filters can make graph or partitioned indexes slow or less accurate.

Example. A search for tenant_id = 42 only traverses vectors belonging to tenant 42 and returns 10 full results.

Related: Post-filtering, Metadata filtering, HNSW

A compression method that splits each vector into equal sub-vectors and replaces each with the id of its nearest entry in a small learned codebook. It shrinks vectors many times over and allows fast approximate distance calculations, with some loss of accuracy.

Example. A 768-dimension float vector of 3,072 bytes is stored as 96 one-byte codes, a 32-fold reduction.

Related: Vector quantisation, IVF, Scalar quantisation

Breaking text into short, self-contained factual statements, each rewritten so it makes sense on its own, and indexing those instead of raw passages. It can improve precision because each unit expresses a single fact.

Example. The sentence 'It was founded in 1998 and is based in Pune' becomes 'Company X was founded in 1998' and 'Company X is based in Pune'.

Related: Chunking, Contextual chunk enrichment, Embedding

Splitting a complex, multi-part question into simpler sub-questions, retrieving evidence for each one separately and then combining the findings into one answer. It helps when no single passage contains everything the question needs.

Example. 'Compare the leave policy in India and the US' becomes two sub-queries, one per country.

Related: Multi-hop retrieval, Query rewriting, Agentic RAG

Adding related words, synonyms or generated variants to a query so that retrieval covers several phrasings of the same need. It raises the chance of matching a relevant document that uses different terms from the user's own question.

Example. 'car loan' is expanded to 'car loan OR vehicle finance OR auto loan'.

Related: Query rewriting, Multi-query retrieval, Sparse retrieval

Transforming a user's question into a form that retrieves better, such as fixing typos, expanding abbreviations or making a follow-up self-contained using chat history. A language model often performs the rewriting before search.

Example. After 'Tell me about the Pune branch', the follow-up 'and its timings?' is rewritten to 'What are the timings of the Pune branch?'.

Related: Query expansion, Multi-query retrieval, Conversational RAG

Measuring a RAG system's quality by testing retrieval and generation separately and together, using a set of questions with known answers or relevant documents. Splitting the analysis shows whether failures come from missing passages or poor answering.

Example. On 200 test questions, the retriever finds the right chunk in 88% of cases, and the generator answers correctly in 79%.

Related: Retrieval evaluation, Faithfulness, Golden dataset

An open-source framework that offers metrics for evaluating RAG pipelines, such as faithfulness, answer relevance, context precision and context recall. Many of its metrics are computed with a language model acting as judge over questions, answers and contexts.

Example. A developer runs Ragas over 100 question-answer-context triples and gets per-metric scores.

Related: RAG evaluation, Faithfulness, Context recall

A technique that recursively clusters chunks, summarises each cluster and repeats to form a tree of summaries, so retrieval can draw on both fine details and high-level themes. It helps with questions about whole long documents.

Example. Leaf chunks of a book are summarised per chapter, then chapters per part, and the query can match any level.

Related: Hierarchical chunking, Chunking, GraphRAG

Recomputing all stored vectors with a new or updated embedding model and rebuilding the index. It is required when changing models, since vectors from different models are not comparable, and can be costly for large collections.

Example. Moving to a new model means re-embedding all 8 million chunks and building a fresh index.

Related: Embedding drift, Ingestion pipeline, Index refresh

The tension in approximate search where raising accuracy, by searching more of the index, increases query time. Tuning parameters such as efSearch or nprobe moves along this curve, and choices depend on the product's latency budget.

Example. Recall of 95% at 4 ms may become 99% at 12 ms after raising the search effort.

Related: ANN recall, efSearch, nprobe

A method for merging several ranked lists by scoring each item as the sum of 1 / (k + rank) across lists, where k is often 60. It needs no score calibration and rewards items ranked highly by multiple retrievers.

Example. A document ranked 1st by BM25 and 3rd by vector search scores 1/61 + 1/63, about 0.0323.

Related: Hybrid search, Score fusion, Reranking

A chunking method that tries to split on the largest natural separator first, such as blank lines, then single newlines, then sentences, then spaces, moving to smaller separators only while a piece is still too long. It keeps paragraphs and sentences together whenever possible.

Example. Separators are tried in order: paragraph break, newline, full stop, space; a 900-token section is split at paragraph breaks into two 450-token chunks.

Related: Fixed-size chunking, Chunking, Structure-aware chunking

A model that scores how well each candidate passage answers a query, by reading query and passage together. Rerankers are more accurate than embedding similarity but too slow to run over an entire collection, so they only rescore a shortlist.

Example. A reranker gives the correct paragraph a score of 0.94, lifting it from rank 12 to rank 1.

Related: Reranking, Cross-encoder, Two-stage retrieval

A second stage that takes the candidates from the first retrieval and reorders them using a more accurate but slower model. It improves the precision of the few passages finally sent to the language model.

Example. Retrieve 50 candidates with vectors, rerank them with a cross-encoder, and pass the best 5 to the generator.

Related: Cross-encoder, Two-stage retrieval, Reranker

Re-computing exact or higher-precision similarity for a shortlist found with compressed vectors, then reordering it. It recovers much of the accuracy lost to quantisation while keeping most of the speed benefit.

Example. Binary vectors find 200 candidates, and full-precision vectors rescore them to give the final top 10.

Related: Binary quantisation, Product quantisation, Two-stage retrieval

Assessing only the retriever, by checking whether known relevant passages appear in its top results. It uses ranking metrics such as recall at k, hit rate and NDCG, and runs without any language model.

Example. For 100 questions, the correct chunk is in the top 5 for 91, giving hit rate 0.91.

Related: Hit rate, Normalised discounted cumulative gain, Recall at k

A pattern in which a system first looks up relevant passages from an external knowledge source and then passes them to a language model along with the question, so the answer is grounded in that material. It lets a model use private or fresh information without retraining and makes answers easier to check against sources.

Example. A Bengaluru bank chatbot searches its 3,000 policy PDFs for the top 5 passages about home-loan prepayment and asks the model to answer using only those.

Related: Retriever, Grounding, Vector database, Context window

The component of a RAG system that takes a query and returns the most relevant pieces of a knowledge base. It may use vector similarity, keyword matching or both. Its quality sets a ceiling on the final answer, because the generator cannot use a passage that was never fetched.

Example. A retriever receives 'refund rules for cancelled flights' and returns the 5 highest-scoring paragraphs from an airline's help pages.

Related: Retrieval-augmented generation, Top-k retrieval, Hybrid search

Compressing vector values by storing each number with fewer bits, for example converting 32-bit floats to 8-bit integers. It cuts memory by about four times with small accuracy loss and is simpler than product quantisation.

Example. Storing each of 1,536 dimensions as 1 byte instead of 4 reduces 1 million vectors from about 6 GB to 1.5 GB.

Related: Product quantisation, Vector quantisation, Binary quantisation

Combining relevance scores from different retrievers into one score, often after normalising them to a common range and using a weighted sum. The weight controls how much keyword or semantic evidence counts.

Example. final = 0.6 * normalised_vector_score + 0.4 * normalised_bm25_score.

Related: Reciprocal rank fusion, Hybrid search, Alpha weighting

An approach in which a language model is trained to decide when to retrieve and to critique its own retrieved passages and outputs using special reflection tokens. It aims for retrieval only when useful and answers better supported by evidence.

Example. The model skips retrieval for a greeting but retrieves for a factual question and marks a passage as supportive.

Related: Corrective RAG, Retrieval-augmented generation, Agentic RAG

A cache that returns a stored answer when a new question is close enough in meaning, by embedding similarity, to an earlier one. It saves model calls, but a loose threshold can return a wrong answer to a subtly different question.

Example. 'How do I reset my password?' reuses the cached answer to 'Steps to change my password' at similarity 0.95.

Related: Embedding cache, Similarity threshold, Retrieval-augmented generation

A chunking approach that embeds neighbouring sentences and starts a new chunk where the similarity between them drops sharply, signalling a topic change. Chunks then follow meaning rather than length, though it costs extra embedding computation and results vary with the threshold.

Example. Sentences about loan eligibility are grouped together; when the text moves to repayment, similarity falls below 0.6 and a new chunk starts.

Related: Chunking, Embedding, Cosine similarity

Embedding individual sentences but, at retrieval time, returning each matched sentence together with a fixed number of sentences before and after it. This gives precise matching with local context and avoids storing large chunks.

Example. A matched sentence is returned with 3 sentences on each side, giving the model 7 sentences of context.

Related: Small-to-big retrieval, Parent-document retrieval, Chunking

A minimum similarity score a result must reach to be returned. Using it lets a RAG system return nothing, and say it does not know, when no stored passage is close enough, instead of passing weak matches to the model.

Example. Chunks with cosine similarity below 0.35 are dropped, so an off-topic question gets no context and the bot declines.

Related: Top-k retrieval, Cosine similarity, Grounding

A strategy that indexes small units, such as single sentences, for accurate matching and then expands to a larger surrounding unit before generation. It counters the trade-off where small chunks match well but give too little context.

Example. A single sentence about 'interest on late payment' is matched, then the surrounding three paragraphs are sent to the model.

Related: Parent-document retrieval, Sentence window retrieval, Chunk size

Retrieval based on sparse term-weight vectors, where most entries are zero and each nonzero entry stands for a word or subword. Classic examples use inverted indexes with BM25; learned sparse models also exist. It excels at exact terms and rare names.

Example. A search for the error code 'ERR_4031' finds the one page containing that exact string.

Related: Dense retrieval, BM25, Inverted index

Chunking that follows a document's own structure, such as headings, list items, code blocks or table boundaries, so each chunk is a coherent section. It suits Markdown, HTML and manuals, where a heading gives strong context for the text beneath it.

Example. A Markdown manual is split at each second-level heading so 'Installation' and 'Troubleshooting' become separate chunks.

Related: Recursive character splitting, Layout-aware parsing, Metadata

An evaluation set whose questions and answers are generated by a language model from the documents themselves, saving manual labelling. It scales cheaply, but needs a human spot-check because generated questions can be too easy or unrealistic.

Example. A model writes 2 questions per chunk from 500 chunks, giving 1,000 test questions that are then sampled and reviewed.

Related: Golden dataset, RAG evaluation, LLM-as-judge

Recovering rows and columns from tables in PDFs or images so their cells stay linked to their headers. Plain text extraction often flattens tables into meaningless number strings, so RAG over financial or scientific documents depends on doing this well.

Example. A fee schedule table is converted to rows such as 'Savings account | Minimum balance | Rs 10,000' so a query about minimum balance can match it.

Related: Layout-aware parsing, Text extraction, Chunking

Turning files such as PDFs, scans or HTML into clean text. Digital PDFs can be parsed directly, while scanned documents need optical character recognition. Bad extraction, such as jumbled columns or dropped tables, produces poor chunks and poor retrieval.

Example. A two-column research paper is extracted so that the left column is read fully before the right, preventing sentences from being interleaved.

Related: Document loader, Layout-aware parsing, Ingestion pipeline

Returning the k highest-scoring items for a query, where k is a chosen number such as 5 or 10. A small k risks missing the answer, while a large k fills the model's context with noise and cost.

Example. Setting k = 5 sends the five most similar chunks to the generator.

Related: Retriever, Similarity threshold, Context window

A design with a fast first stage that fetches a broad candidate set, followed by a slower, more precise stage that reranks it. It balances speed over large collections with accuracy on the final results.

Example. Stage one returns 100 chunks in 20 ms; stage two reranks them in 150 ms and keeps 5.

Related: Reranking, Retriever, Cross-encoder

A database built to store embeddings together with their metadata and to quickly return the vectors closest to a query vector. It typically uses approximate indexes to stay fast at large scale, and adds filtering, updates and persistence.

Example. A team stores 10 million chunk embeddings in a vector database and retrieves the top 5 matches for each question in under 50 ms.

Related: Vector index, Approximate nearest neighbour search, Metadata filtering

The data structure that organises stored vectors so nearest-neighbour queries need not compare against every vector. Common types include graph-based, cluster-based and hashing-based indexes, each trading speed, memory and accuracy differently.

Example. Building an HNSW index over 2 million vectors lets a query touch only a few thousand of them.

Related: HNSW, IVF, Flat index

Scaling a vector to have length 1 by dividing it by its norm. With normalised vectors, dot product equals cosine similarity, so systems can use the faster dot product for angle-based search.

Example. The vector [3, 4] has norm 5 and becomes [0.6, 0.8].

Related: Cosine similarity, Dot product similarity, Embedding

Lossy compression of vectors by mapping each to an approximation from a limited set of representatives. In vector search it reduces memory and speeds up comparisons, at some cost in accuracy.

Example. Compressing 100 million vectors with quantisation lets the index fit in RAM on one server.

Related: Product quantisation, Scalar quantisation, Vector index