Prompt context engineering

A method where the model is told to recall or invent similar solved problems, write them out with solutions, and then use them to solve the new one. It removes the need for hand-written examples.

Example. For a geometry question, the model first writes two related area problems with solutions, then solves the target problem.

Related: Few-shot chain-of-thought, Step-back prompting, Generated knowledge prompting

A technique of supplying the first words of the model's reply yourself, so the model continues from them instead of starting fresh. It steers format and tone, for example forcing the reply to begin with an opening brace so that JSON follows. Support depends on the provider.

Example. End the message list with an assistant turn containing '{"sentiment":' so the model completes only the value, such as "negative".

Related: Structured output, Output format specification, Message roles

A method in which a language model proposes many candidate instructions for a task, each candidate is scored on example data, and the best-scoring instruction is kept. It replaces manual trial and error with a search. Often abbreviated APE.

Example. Given 20 labelled examples, the model proposes 30 instructions; the one with the highest accuracy on a validation set is selected.

Related: Prompt optimisation, Meta-prompting, DSPy

A provider service where you submit many requests together to be processed later, often within hours, at a lower price than live calls. It suits large jobs that need no instant reply, such as labelling a whole dataset.

Example. Upload a file of 50,000 product descriptions to classify overnight at reduced cost, then download the results file.

Related: Token pricing, Prompt caching, Token budget

How well the reasoning a model writes out reflects the real factors that led to its answer. A model can produce a tidy explanation that does not match what actually drove the output, so written reasoning should not be treated as proof of how it decided.

Example. A model is nudged by a hint in the prompt to choose option B, but its written reasoning never mentions the hint.

Related: Chain-of-thought prompting, Reasoning model, Scratchpad

A technique that asks a model to write out its intermediate reasoning steps before giving a final answer. Showing the work helps on maths, logic and multi-step questions, because each step conditions the next. It costs extra output tokens.

Example. 'A shop sells 3 pens for Rs 45. What do 7 pens cost? Think step by step.' The model writes 45/3 = 15 each, then 7 x 15 = 105.

Related: Zero-shot chain-of-thought, Few-shot chain-of-thought, Self-consistency

A technique in which a model drafts an answer, plans verification questions about its own claims, answers them independently, and then produces a corrected final answer. It aims to reduce made-up facts in longer text.

Example. After writing a bio of a cricketer, the model asks 'In which year did he debut?' separately and fixes any mismatch.

Related: Self-refine, Reflexion, Grounding

A common style of language model API where you send a list of role-tagged messages and receive the model's next message. Many providers and open-source servers copy this shape, which makes code easier to switch between them.

Example. A POST request with model name, a messages list and max_tokens returns a JSON object whose choices hold the reply text.

Related: Message roles, Stateless API, Streaming response

A technique that chooses, as examples for a step-by-step prompt, those with the longest reasoning chains, and prefers the most common answer among longer sampled chains. The idea is that more complex demonstrations lead to better multi-step reasoning.

Example. From 10 candidate examples, pick the 4 that need the most reasoning steps to put into the prompt.

Related: Few-shot chain-of-thought, Self-consistency, Dynamic few-shot selection

The share of a model's available input space you allow each kind of content, such as instructions, documents, memory and tool output. Because space and cost are limited, deciding what deserves room is a core design choice.

Example. Cap retrieved documents at 6,000 tokens so that history and instructions never get squeezed out.

Related: Token budget, Context engineering, Context truncation

Shrinking the material placed in a prompt, through summarising, removing low-value passages or dropping filler words, while keeping the facts needed for the task. It saves cost and delay and can improve focus on long inputs.

Example. A 12,000-token set of meeting notes is reduced to a 2,500-token digest of decisions and action items.

Related: Prompt compression, Rolling summary, Context budget

The craft of deciding what information goes into a model's input at each step: instructions, history, retrieved documents, tool results and examples, and in what order and amount. It goes beyond wording a single prompt and matters most for long-running agents.

Example. An assistant keeps the latest 5 messages, a summary of older ones, and the 3 most relevant policy passages for each turn.

Related: Prompt engineering, Context budget, Context compression

Placing as much source material as possible into the prompt in one go, relying on the model's large input capacity rather than selecting passages. It is easy but raises cost and can bury the relevant part among noise.

Example. Pasting an entire 150-page policy PDF into the prompt to answer a single question about leave rules.

Related: Long-context prompting, Lost in the middle, Context budget

Cutting text so that it fits within the allowed input size, by dropping the oldest messages, the middle, or the end. It is simple but can remove important facts, so what to keep must be chosen with care.

Example. When a chat exceeds the limit, the app deletes the oldest turns but always keeps the system prompt.

Related: Sliding window history, Context budget, Rolling summary

A prompting method that shows the model both a correct reasoning example and an incorrect one, labelled as such, so it learns what mistakes to avoid. It aims to improve step-by-step accuracy over showing only good examples.

Example. One example correctly computes 15% of 240 as 36, another wrongly gives 15, marked 'wrong: forgot to multiply by 0.15'.

Related: Chain-of-thought prompting, Few-shot chain-of-thought, Few-shot prompting

A simple chat-memory approach that stores the raw messages of a conversation and replays them in the prompt each turn, as offered by frameworks such as LangChain. It is easy and lossless but grows without limit, so it is often paired with trimming or summaries.

Example. A chatbot keeps all 12 messages in a list and sends them with the 13th user question.

Related: Conversation history, Sliding window history, Rolling summary

The list of earlier user and assistant messages sent back to the model so it can follow the thread of a chat. Since models do not remember past calls, managing this list well decides whether replies stay coherent and affordable.

Example. To answer 'And in Hindi?', the request includes the previous English answer so the model knows what to translate.

Related: Stateless API, Multi-turn conversation, Sliding window history

Marker characters or strings such as triple quotes, dashes or tags that separate parts of a prompt, for example the instructions from the text to be processed. They make boundaries clear to the model and lower the risk of mixing instructions with data.

Example. Summarise the text between ### markers: ### {email_body} ### so the model does not follow commands hidden in the email.

Related: XML tags in prompts, Prompt template, Markdown formatting in prompts

A message role offered by some model APIs for instructions written by the application developer, ranked above the end user's messages but working like a system prompt. It lets the builder set rules that a user cannot easily override, and it supports a clear order of trust among instructions.

Example. A travel app sends a developer message: 'Always answer in the customer's language and never reveal internal pricing rules', then passes the user's chat below it.

Related: System prompt, Instruction hierarchy, Message roles

A method that adds a short hint, such as a few keywords, into the prompt to guide the model toward the desired content of the output. A small helper model can generate the hint. It steers summaries and dialogue without changing the main model.

Example. Adding 'Hint: monsoon, delay, refund' before asking for a summary of a train-delay complaint.

Related: Prompt optimisation, Zero-shot prompting, Automatic prompt engineer

An open-source framework that treats language model programs as modules with typed inputs and outputs, and automatically tunes their prompts and examples against a metric you define. It lets developers write code rather than hand-written prompt strings.

Example. You declare a module 'question -> answer', provide 50 labelled questions and a metric, and an optimiser searches for better prompts and demonstrations.

Related: Prompt optimisation, Automatic prompt engineer, Prompt template

Choosing the examples for a prompt at request time from a larger bank, usually by finding the ones most similar to the current input using embeddings. It gives the model relevant demonstrations instead of a fixed set, which often improves accuracy on varied inputs at the cost of extra retrieval.

Example. A bank of 500 labelled complaints is searched for the 4 closest to the new complaint, and those four are placed in the prompt.

Related: Few-shot prompting, In-context learning, Prompt template

A prompt containing several examples in which each answer includes its reasoning steps, so the model imitates that reasoning style on a new question. It usually beats bare examples on multi-step problems because the reasoning pattern is demonstrated.

Example. Two sample questions each show 'Step 1... Step 2... Answer: 42' before the third real question is posed.

Related: Chain-of-thought prompting, Few-shot prompting, Self-consistency

Including a small number of input-output examples (usually 2 to 8) in the prompt so the model copies the pattern for a new input. It teaches format, tone and edge cases without retraining. The quality, variety and order of the examples strongly affect the result.

Example. To tag support tickets, the prompt lists five tickets with labels such as 'refund' or 'delivery delay', then adds a sixth ticket without a label.

Related: In-context learning, Dynamic few-shot selection, Zero-shot prompting

A sampling setting that lowers the chance of a token in proportion to how many times it has already appeared in the output. It reduces word-for-word repetition. Too high a value can push the text toward odd vocabulary.

Example. Set a frequency penalty of 0.5 to stop a product description from repeating 'premium quality' in every line.

Related: Presence penalty, Repetition penalty, Logit bias

A feature where the model, instead of plain text, returns a request to run a named function with arguments in JSON that your code then executes. You describe available functions in advance. It connects language models to databases, APIs and calculators.

Example. Given get_weather(city), the model answers a Bengaluru weather question with {"name":"get_weather","arguments":{"city":"Bengaluru"}}.

Related: Tool schema, Tool choice, ReAct prompting

A grammar format used by the llama.cpp inference project to describe allowed output patterns, based on Backus-Naur form. A model server reads it and only allows tokens that keep the text valid under the grammar, for example forcing JSON or a fixed list of answers.

Example. A GBNF rule 'root ::= "yes" | "no"' limits a local model's reply to one of two words.

Related: Grammar-constrained decoding, Schema-constrained generation, Structured output

A two-step technique where the model first writes out relevant facts about the topic, and those facts are then added to the prompt when answering the actual question. It can help commonsense questions, though generated facts may be wrong.

Example. For 'Can a penguin fly?', the model first writes 'Penguins are flightless birds', then answers using that statement.

Related: Step-back prompting, Grounding, Prompt chaining

A technique that limits, at each step of generation, which tokens are allowed so the whole output follows a formal grammar or pattern, such as a regular expression or JSON syntax. It guarantees a valid shape rather than hoping the model complies.

Example. A regex like '[0-9]{6}' forces the model to output only a six-digit Indian pincode.

Related: Schema-constrained generation, GBNF grammar, Outlines library

Tying a model's answer to supplied source material or verified data, and telling it to answer only from that material. It reduces invented facts and lets users check claims against the source.

Example. 'Answer using only the policy text below; if the answer is not there, say you do not know.'

Related: Quote-first prompting, Chain-of-Verification, Context engineering

The ability of a large language model to pick up a task from instructions and examples placed in its prompt, with no change to its weights. The learning lasts only for that request. It underlies few-shot prompting and explains why prompts can act like programs.

Example. Show a model three examples of converting Indian dates like '05/03/2026' into 'DD Month YYYY' and it converts a fourth date correctly with no training.

Related: Few-shot prompting, Zero-shot prompting, Prompt engineering

The ability of a language model to carry out what a prompt asks, including constraints on length, format, language and content. Models trained on instructions do this far better than raw pretrained ones. Weak compliance shows up as ignored constraints or missing steps.

Example. Asked for 'exactly three bullets, each under 12 words', a model with strong instruction following returns three short bullets and nothing else.

Related: Prompt engineering, Output format specification, System prompt

A defined order of trust among the instruction sources a model sees, typically system or developer rules first, then the user, then content from tools or documents. Training models to follow it helps them ignore commands hidden in lower-trust text.

Example. If a web page fetched by a tool says 'ignore your rules', the model treats it as data, not an order.

Related: System prompt, Developer message, Message roles

An open-source Python library that wraps model API clients so you can ask for a Pydantic model as the response type. It handles the schema, validation and automatic retries, making structured extraction a few lines of code.

Example. client.chat.completions.create(response_model=Invoice, ...) returns an Invoice object rather than raw text.

Related: Pydantic validation of LLM output, Structured output, Output repair

A provider option that guarantees the model's reply is syntactically valid JSON. It does not by itself guarantee that the fields, names or types match what your code expects, so validation is still needed.

Example. With JSON mode on, the reply always parses, but a field might be called 'total' instead of the 'amount' you wanted.

Related: Structured output, Schema-constrained generation, Output repair

A technique that first asks the model to break a hard problem into simpler sub-questions, then solves them in order, feeding earlier answers into later ones. It helps when the test problem is harder than the examples shown.

Example. A long word problem is split into 'How many boxes per truck?', then 'How many trucks?', then the final total.

Related: Task decomposition, Chain-of-thought prompting, Prompt chaining

A parameter that adds a fixed number to the score of chosen tokens before one is picked, making them more or less likely, or effectively banned. It can force a choice among labels or suppress unwanted words.

Example. Give the tokens for 'Yes' and 'No' a large positive bias so the model answers with only one of them.

Related: Logprobs, Frequency penalty, Structured output

The logarithms of the probabilities a model assigns to the tokens it generates, and sometimes to the top alternatives at each position. They let you measure confidence, rank choices and build classifiers from a single-token answer.

Example. A yes/no prompt returns logprob -0.05 for 'Yes' (about 95% probability) and -3.0 for 'No', suggesting high confidence.

Related: Logit bias, Structured extraction, Best-of-N sampling

Working with prompts that carry very large inputs, such as whole books or codebases, and applying habits that help: putting documents before questions, labelling sections and asking for quotes. Capacity alone does not guarantee good use of the text.

Example. Place three 40-page reports at the top in tagged blocks, then ask the comparison question at the end.

Related: Lost in the middle, Context stuffing, Quote-first prompting

A documented pattern where models use information at the start and end of a long input better than information placed in the middle. It means where you place key facts in a long prompt can change answer accuracy.

Example. In a 20-document prompt, a model answers correctly when the relevant passage is first or last, but misses it in position 10.

Related: Position bias, Needle in a haystack test, Long-context prompting

A technique that asks a model to explain its answer, then to explain each explanation, building a tree of statements and dropping the ones that contradict each other. It aims to keep only mutually consistent reasoning.

Example. The model says 'A' is true because X, then is asked 'if X were false...'; inconsistent branches are removed.

Related: Self-consistency, Tree of Thoughts, Chain-of-thought prompting

A tendency of models in few-shot prompts to favour the answer label that appears most often among the examples. It can skew results when the examples are unbalanced. Balancing the labels and shuffling their order reduces it.

Example. If 6 of 8 examples in a sentiment prompt are labelled 'positive', the model may call neutral reviews positive too often.

Related: Recency bias in prompts, Few-shot prompting, Prompt sensitivity

A method for texts too long for one prompt: split it into chunks, summarise each chunk separately (map), then combine those summaries into a final summary (reduce). Chunks can be handled in parallel.

Example. A 300-page report is split into 30 chunks, each summarised in 100 words, and the 30 summaries are merged into one page.

Related: Refine summarisation, Prompt chaining, Context budget

Using headings, bullet lists and code fences inside a prompt to organise long instructions, and asking for Markdown in the reply. Clean structure helps both the model and human reviewers. Some applications need plain text instead, so the format should be stated clearly.

Example. A prompt has sections '## Task', '## Rules' and '## Input', and asks the reply to use a numbered list.

Related: Delimiters, XML tags in prompts, Output format specification

A request parameter that limits the number of tokens the model may generate in its reply. It controls cost and delay, but a low value can cut an answer off mid-sentence. It does not limit the size of the input.

Example. max_tokens=300 for a short summary; if the model needs more it stops at 300 with a 'length' finish reason.

Related: Token budget, Stop sequence, Thinking budget

The labels attached to each message in a chat-style API call, typically system, user, assistant and tool. The role tells the model who is speaking and how much weight to give the text. Getting the roles right keeps instructions, questions, model replies and tool outputs separate.

Example. A request holds three messages: {role: system, ...}, {role: user, 'What is my EMI?'} and {role: assistant, 'Please share the loan amount.'}.

Related: System prompt, User prompt, Chat template

Using a language model to write, critique or improve prompts, so that the prompt itself is a model output. It shifts effort from hand-tuning wording to describing the task and letting the model draft candidate instructions.

Example. Ask a model: 'Write a system prompt for a bot that answers questions about Aadhaar update rules; list its weaknesses.'

Related: Automatic prompt engineer, Prompt optimisation, Prompt engineering

An exchange where the user and model send several messages back and forth, and each new reply depends on the earlier ones. Since the model keeps no memory of its own, the application must resend the relevant history with each call.

Example. A user asks 'What is the tenure of the plan?' and then 'Can I extend it?', where 'it' refers to the plan mentioned earlier.

Related: Conversation history, Stateless API, Sliding window history

An evaluation that hides one specific fact (the needle) at varying depths inside a long block of unrelated text (the haystack) and checks whether the model can retrieve it. It probes how well a model uses long inputs.

Example. Insert 'The secret code is 4172' at 60% depth in a 100,000-token text and ask the model for the code.

Related: Lost in the middle, Long-context prompting, Position bias

Giving a model exactly one worked example of the task inside the prompt before the real input. The single example shows the desired format and style. It is a light way to fix output shape when an instruction alone is unclear.

Example. The prompt shows one line, 'Input: Rs 5,000 paid to Ravi -> Output: {"amount":5000,"payee":"Ravi"}', then asks for the next SMS.

Related: Zero-shot prompting, Few-shot prompting, In-context learning

An open-source Python library for guiding text generation with regular expressions, JSON Schemas and grammars, by restricting the tokens the model can pick at each step. It works with local open-weights models.

Example. Use Outlines with a schema so a local model must output {"city": string, "pincode": 6-digit string}.

Related: Grammar-constrained decoding, Schema-constrained generation, Instructor library

The part of a prompt that states exactly what shape the answer must take, such as length, fields, language, or a schema. Being explicit here saves parsing effort later and makes outputs comparable across runs.

Example. 'Reply with a JSON object with keys city, pincode and confidence (0 to 1). Output nothing else.'

Related: Structured output, Assistant prefill, Output parser

Code that reads the raw text of a model reply and turns it into a usable structure, for instance pulling out JSON, a list or text between tags. Good parsers handle stray text around the answer and report clear errors.

Example. A parser finds the text between <answer> and </answer> and converts '42' into the integer 42.

Related: Structured output, Output repair, XML tags in prompts

Fixing a malformed or invalid model reply, either with simple code such as closing a missing bracket, or by sending the error back to the model and asking it to correct itself. It raises success rates when formats are not strictly enforced.

Example. A reply misses the closing brace; the code adds it, or re-prompts with 'Invalid JSON: unterminated object. Return corrected JSON.'

Related: Pydantic validation of LLM output, JSON mode, Output parser

A feature in which the model asks for several independent tool calls in a single turn, so the application can run them at the same time and return all results together. It cuts waiting time when the calls do not depend on each other.

Example. Asked for weather in Mumbai, Delhi and Chennai, the model returns three get_weather calls at once.

Related: Function calling, Tool choice, Tool result message

A zero-shot variant of step-by-step prompting that first tells the model to devise a plan for the problem and then to carry out that plan step by step. It targets the missing-step errors of plain step-by-step prompts.

Example. 'First understand the problem and make a plan, then carry out the plan.' is added before a multi-part budget question.

Related: Zero-shot chain-of-thought, Least-to-most prompting, Task decomposition

A pattern where a model treats information differently depending on where it sits in the prompt, often paying more attention to the start and end than the middle. It affects long documents, lists of options and answers judged side by side. Reordering inputs is the usual test.

Example. When two candidate answers are shown to a judge model, it picks the first one more often regardless of quality.

Related: Lost in the middle, Recency bias in prompts, Long-context prompting

A sampling setting that lowers the chance of any token that has already appeared at least once, regardless of how often. It nudges the model to bring in new words and topics rather than staying on what it has said.

Example. A presence penalty of 0.6 encourages a brainstorming answer to list varied ideas rather than circling one theme.

Related: Frequency penalty, Repetition penalty, Logit bias

An approach where the model writes short programs as its reasoning steps and an interpreter runs them to get the answer, instead of doing arithmetic in text. It cuts calculation errors because the computer does the maths. Often shortened to PAL.

Example. For a compound-interest question, the model outputs Python 'principal * (1 + 0.07) ** 5', and the interpreter returns 1.4026 times the principal.

Related: Chain-of-thought prompting, Function calling, ReAct prompting

The full input text given to a language model to make it produce an output. It can hold instructions, background facts, examples and the user's question. Because the model only sees what is in this input, the wording and content of it largely decide how good the answer is.

Example. A prompt for a Zomato support bot might be: 'You are a polite agent. Customer message: My biryani arrived cold. Draft a reply in two sentences.'

Related: System prompt, Prompt engineering, User prompt

A provider feature that stores the processed form of a long, repeated prompt prefix so later requests starting with the same text are cheaper and faster. It works best when stable parts such as the system prompt and documents come first and changing parts last.

Example. A 20,000-token policy document placed at the start of every request is cached, so only the short question is processed fresh each time.

Related: Token pricing, Context budget, System prompt

Linking several prompts in sequence, where the output of one becomes part of the input to the next. Each step has a narrow job, which makes results easier to test and debug than one giant prompt. Frameworks such as LangChain popularised the pattern.

Example. Step 1 extracts claims from a news article, step 2 fact-checks each claim, step 3 writes a two-line verdict.

Related: Task decomposition, Least-to-most prompting, Map-reduce summarisation

A family of methods that shorten a prompt automatically, for example by scoring tokens or sentences with a small model and deleting the least informative ones, as done by the LLMLingua research approach. It reduces cost while trying to keep accuracy.

Example. A 3,000-token retrieved context is compressed to 800 tokens before being sent to the main model.

Related: Context compression, Token pricing, Context budget

The practice of designing and refining the text sent to a language model so it gives reliable, useful answers without changing the model's weights. It covers wording, examples, structure, and testing variations. It is the cheapest way to improve behaviour before trying retrieval or fine-tuning.

Example. A team tries three versions of a resume-screening prompt on 50 sample resumes and keeps the one that matches recruiter decisions most often.

Related: Prompt template, Prompt optimisation, Context engineering

The systematic improvement of a prompt using a test set and a score, by trying variations of wording, examples and structure, by hand or by algorithm. It replaces guesswork with measured gains, and needs a good evaluation set to avoid overfitting to it.

Example. Score five prompt variants on 100 labelled support emails and adopt the one with 92% correct routing.

Related: Prompt regression testing, DSPy, Automatic prompt engineer

A central store where an organisation keeps its approved prompts with names, versions, owners and metadata, so applications fetch them at run time instead of hard-coding text. It supports review, reuse and quick updates without redeploying code.

Example. An app calls registry.get('invoice-extractor', version='prod') and gets the current approved prompt text.

Related: Prompt versioning, Prompt template, Prompt regression testing

Running a fixed set of test inputs with expected outcomes every time a prompt or model changes, to catch cases that used to work and now fail. It is the prompt equivalent of unit tests and is often run in continuous integration.

Example. After editing a prompt, 150 saved test tickets are rerun and the build fails if accuracy drops below 90%.

Related: Prompt versioning, Prompt sensitivity, Prompt optimisation

How much a model's output changes when the prompt is reworded in ways that should not matter, such as changing punctuation, order of examples or synonyms. High sensitivity makes results fragile, so serious teams test many phrasings before trusting a prompt.

Example. Changing 'Answer:' to 'Response:' in a prompt moves accuracy on a 200-question test from 71% to 64%.

Related: Prompt regression testing, Recency bias in prompts, Prompt optimisation

A reusable prompt text with named blanks that are filled in with real values at run time. It keeps the fixed instructions consistent and lets code insert the changing parts. Templates are the base unit for versioning and testing prompts.

Example. 'Translate the following {source_lang} text to {target_lang}: {text}' is filled with Hindi, English and a customer message.

Related: Template variable, Prompt versioning, Prompt engineering

Keeping numbered, tracked versions of prompts, usually in source control or a prompt tool, with notes on what changed. It lets teams roll back a bad change and link a quality shift to a specific edit.

Example. Prompt 'support-reply' goes from v12 to v13 after adding a refund rule, and a drop in scores is traced to v13.

Related: Prompt registry, Prompt regression testing, Prompt template

Checking a model's structured reply against a Pydantic model, a Python class that declares fields and types, and turning it into a typed object. Validation errors show exactly what is wrong, which can be fed back to the model for a retry.

Example. A class Invoice(amount: float, gst: float) rejects a reply where amount is the text 'about 500'.

Related: Instructor library, Output repair, Structured output

A technique for long-document tasks where the model is asked to first extract the exact quotes relevant to the question and only then answer using those quotes. It anchors the answer to the source and makes checking easier.

Example. Asked about a termination clause, the model lists two verbatim sentences inside <quotes> tags, then answers based on them.

Related: Grounding, XML tags in prompts, Long-context prompting

A pattern in which the model alternates between written reasoning ('Thought'), an action such as a search or tool call ('Action'), and the result it gets back ('Observation'), repeating until it can answer. It is the basis of many agent loops.

Example. Thought: I need today's USD-INR rate. Action: search('USD to INR'). Observation: 83.2. Thought: now compute.

Related: Function calling, Chain-of-thought prompting, Reflexion

A setting offered by some model APIs that controls how much thinking a reasoning model does before answering, typically as levels such as low, medium and high. Higher effort usually improves hard-task accuracy but raises latency and cost.

Example. Use low effort for extracting an invoice number and high effort for debugging a tricky concurrency bug.

Related: Reasoning model, Thinking budget, Token budget

A tendency of models to lean toward the last example or the last instruction in a prompt, giving it extra weight. The order of few-shot examples can therefore change the output. Testing several orderings or placing the key rule last helps manage it.

Example. With the final example labelled 'spam', a model labels an uncertain email as spam; swapping the example order flips the answer.

Related: Majority label bias, Position bias, Few-shot prompting

A long-document method that summarises the first chunk, then passes that summary and the next chunk to the model to update it, and repeats through the text. It keeps order and detail but runs sequentially and is slower than map-reduce.

Example. Summarise chapter 1, then update the summary with chapter 2, and so on to the end of a book.

Related: Map-reduce summarisation, Rolling summary, Prompt chaining

A technique in which a model reviews the outcome of its earlier attempt, writes a short verbal lesson about what went wrong, and keeps it in context for the next attempt. It improves results over repeated tries without any weight updates.

Example. After failing a coding test, the model notes 'I forgot the empty-list case' and includes that note when it retries.

Related: Self-refine, ReAct prompting, Chain-of-Verification

A setting, common in open-source inference tools, that divides or reduces the score of tokens already present in the text so far, discouraging loops. It acts similarly to frequency and presence penalties but is defined by its own formula.

Example. In a local model server, repetition_penalty=1.1 stops a story from repeating the same sentence.

Related: Frequency penalty, Presence penalty, Logit bias

A technique where the model is asked to first restate the user's question in clearer words and then answer its own rewritten version. It can fix ambiguity in how a question was worded, and the rewrite can be done by a separate model call.

Example. 'Rephrase and expand the question, then respond.' turns 'Is it ok to break a lease early?' into a fuller question about notice and penalties.

Related: Step-back prompting, Prompt chaining, Meta-prompting

Telling the model to act as a specific kind of expert or character, such as an auditor or a teacher, to shape tone, vocabulary and focus. It is a quick way to set style, though it does not add knowledge the model lacks.

Example. Start with 'You are a chartered accountant explaining TDS to a first-time freelancer in simple language.'

Related: System prompt, Prompt engineering, Output format specification

A running short summary of earlier conversation that the model updates as new turns arrive and sends along with recent messages. It keeps key facts alive while using far fewer tokens than the full history.

Example. After 30 turns, the prompt holds a 150-word summary plus the last 4 messages.

Related: Sliding window history, Context compression, Conversation history

Producing model output that is forced to obey a supplied schema, such as a JSON Schema listing required fields, types and allowed values. Enforcement usually happens during decoding, so invalid tokens cannot be chosen, giving reliable parsing.

Example. Provide a schema where 'status' must be one of 'paid', 'pending' or 'failed'; the model cannot output any other value.

Related: Structured output, Grammar-constrained decoding, JSON mode

A designated area of the model's output where it writes working notes and intermediate steps before the final answer. Keeping the notes separate lets the application hide or ignore the messy reasoning while keeping its accuracy benefit.

Example. The prompt says 'Think inside <scratchpad> tags, then give the final answer inside <answer> tags.' Only the answer is shown to the user.

Related: Chain-of-thought prompting, XML tags in prompts, Chain-of-thought faithfulness

An option in some APIs to fix the random number seed used in sampling, so repeated identical requests give the same or very similar output. It helps debugging and testing but does not guarantee identical results across model updates.

Example. Run the same prompt twice with seed=42 and compare outputs when investigating a regression.

Related: Prompt regression testing, Best-of-N sampling, Max tokens

A prompting pattern where the model states follow-up questions it needs to answer first ('Are follow-up questions needed here?'), answers them one by one, often with a search tool, and then composes the final answer to a multi-hop question.

Example. For 'Who is older, the CEO of Infosys or of TCS?', it asks for each person's birth year before comparing.

Related: ReAct prompting, Least-to-most prompting, Task decomposition

A method that samples several different reasoning paths for the same question at a non-zero randomness setting, then takes the most common final answer by majority vote. It improves reliability on reasoning tasks but multiplies the cost by the number of samples.

Example. Ask a maths question 5 times; answers are 105, 105, 95, 105, 110, so 105 is chosen.

Related: Chain-of-thought prompting, Best-of-N sampling, Tree of Thoughts

A loop in which the same model writes a draft, critiques it, and rewrites it using its own feedback, repeating a few times. It can raise quality on writing and code without extra training, though it can also reinforce mistakes the model cannot see.

Example. A model drafts a cover letter, lists three weaknesses such as vague achievements, then rewrites the letter to fix them.

Related: Reflexion, Chain-of-Verification, Prompt chaining

A technique where the model first writes a short outline of its answer, then expands each outline point, often in parallel calls. The main aim is faster responses by expanding points at the same time, and sometimes clearer structure.

Example. An outline of five bullet points on 'benefits of UPI' is produced, then five calls expand each bullet at once.

Related: Task decomposition, Prompt chaining, Chain-of-thought prompting

A conversation strategy that sends only the most recent N messages or tokens with each request, letting older ones drop off. It bounds cost and size, though early details are forgotten unless summarised.

Example. Keep the last 10 messages of a support chat; the customer's order id given in message 2 is lost unless stored elsewhere.

Related: Context truncation, Rolling summary, Conversation history

A design where the server keeps no record of earlier requests, so every call must carry all the information needed. Most language model APIs work this way: the client sends the whole conversation each time. It simplifies scaling but makes token use grow with conversation length.

Example. On turn 10 of a chat, your code sends all 10 user messages and 9 replies to the API again.

Related: Conversation history, Multi-turn conversation, Token budget

A technique where the model is first asked a more general question about the underlying principle, and then uses that answer to solve the specific problem. Stepping back to the concept reduces errors from getting lost in details.

Example. Before answering a specific gas-law question, the model first states 'What is the ideal gas law and what do its variables mean?'

Related: Chain-of-thought prompting, Least-to-most prompting, Generated knowledge prompting

A string that, when the model generates it, makes generation halt immediately. It prevents rambling, and helps cut output at a known marker such as the end of a section. The stop string itself is usually not returned.

Example. Setting stop=['\n\nUser:'] makes the model stop before it invents the next user turn.

Related: Max tokens, Output format specification, Streaming response

Using a model to pull specific fields out of unstructured text, like names, dates and amounts, and return them in a fixed structure. It replaces brittle rules for documents that vary widely in layout.

Example. From a WhatsApp order message, extract {"item": "kurta", "size": "M", "pincode": "560001"}.

Related: Structured output, Schema-constrained generation, Instructor library

Getting a model to reply in a fixed, machine-readable shape such as JSON with named fields, instead of free text. It lets programs consume the answer directly. Methods range from careful prompting to decoding that enforces a schema.

Example. A resume parser returns {"name": "Asha", "skills": ["SQL","Python"], "years": 3} every time.

Related: JSON mode, Schema-constrained generation, Output parser

A message placed at the start of a conversation that sets the model's role, rules, tone and limits for every later turn. Developers write it once, and end users usually never see it. It is the main place to say what the assistant should and should not do.

Example. A bank app's system prompt says: 'You are a support assistant for a savings bank in India. Answer only about accounts and cards. Never give investment advice.'

Related: User prompt, Message roles, Instruction hierarchy

Splitting a complicated request into smaller steps, each handled by its own prompt or its own stage in one prompt. Smaller steps are easier to check and to fix, and they often give more accurate results than one large instruction.

Example. Instead of one prompt to 'analyse this annual report', run three: extract key figures, compare with last year, then write a summary.

Related: Prompt chaining, Least-to-most prompting, Chain-of-thought prompting

A named placeholder in a prompt template, written like {customer_name}, that the program replaces with a real value each time. Good handling includes escaping unsafe content and checking length, because the inserted text becomes part of the instructions the model reads.

Example. In 'Reply to {customer_name} about order {order_id}', the code supplies 'Meera' and 'OD4471'.

Related: Prompt template, Delimiters, Prompt versioning

A cap on the number of tokens a model may spend on its internal reasoning before it must answer. Setting it balances answer quality against cost and delay. Simple queries need a small budget, hard ones a larger one.

Example. Set a thinking budget of 2,000 tokens for a quick classification and 16,000 for a multi-step maths proof.

Related: Reasoning effort, Reasoning model, Max tokens

The plan for how many tokens each part of a request may use, covering instructions, history, retrieved documents and the reply, within the model's limit and your cost target. Budgeting stops overflow errors and surprise bills.

Example. With a 16,000-token limit you allot 1,000 to the system prompt, 9,000 to documents, 4,000 to history and 2,000 to the answer.

Related: Context budget, Max tokens, Token pricing

Measuring how many tokens a text will use, with the model's own tokenizer or a provider's counting tool, before sending it. Counts differ between models and languages, and Hindi or Tamil text often uses more tokens than English for the same meaning.

Example. Run the tokenizer on a 3-page contract to confirm it fits in the 8,000-token input allowance.

Related: Token budget, Context budget, Token pricing

The way providers charge for model use, usually a price per million input tokens and a different, higher price per million output tokens. Knowing it lets you estimate cost per request and see why long prompts and long replies matter.

Example. If input costs Rs 100 per million tokens and a request uses 2,000 input tokens, that part costs Rs 0.20.

Related: Token counting, Prompt caching, Batch API

A request setting that controls whether the model may, must, or must not call a tool, or must call one specific tool. It gives developers control, for example forcing a structured extraction call every time.

Example. Set tool_choice to a fixed function 'extract_invoice' so the reply is always that function's arguments.

Related: Function calling, Tool schema, Parallel tool calls

The plain-language text in a tool definition explaining what the tool does, when to use it and what each parameter means. Since the model relies on it to choose tools, clear and specific descriptions strongly improve correct tool use.

Example. 'Look up an order's delivery status by order id. Use only when the customer gives an id starting with OD.'

Related: Tool schema, Function calling, Tool choice

The message sent back to the model after a tool has run, containing the tool's output and linked to the model's call by an identifier. The model reads it to write its final reply or make another call.

Example. Role 'tool', call id 'abc1', content '{"status":"Out for delivery"}', which the model turns into a friendly update.

Related: Function calling, Message roles, Parallel tool calls

The machine-readable definition of a tool that is sent to the model: its name, a description, and a JSON Schema for its parameters. The model reads it to decide when to call the tool and how to fill in the arguments correctly.

Example. A schema names 'convert_currency' with required string fields 'from', 'to' and a number field 'amount'.

Related: Function calling, Tool description, Tool choice

A prompting framework that lets a model explore several candidate reasoning steps at each stage, rate them, and keep or discard branches, much like a search over a tree. It suits puzzles and planning where a single straight line of reasoning can fail. It needs many model calls.

Example. For a 24-game puzzle, the model proposes three next operations, scores each as promising or not, and continues only from the best ones.

Related: Chain-of-thought prompting, Self-consistency, Prompt chaining

The message in a conversation that carries the actual request or question from the person (or program) using the model. It changes on every turn, unlike the fixed rules in the system prompt. In an API call it is sent with the role 'user'.

Example. In a chat, the user prompt is 'Summarise this GST notice in three bullets', followed by the pasted notice text.

Related: System prompt, Message roles, Multi-turn conversation

A prompt style that wraps each section in named tags like <document> or <instructions>. The names tell the model what each block is, help it refer to blocks by name, and make outputs easy to extract. Several model providers recommend it for long, multi-part prompts.

Example. <contract>...text...</contract><question>What is the notice period?</question> and ask the answer inside <answer></answer> tags.

Related: Delimiters, Prompt template, Output parser

Triggering step-by-step reasoning with a plain phrase such as 'Let's think step by step', without supplying any worked examples. It was shown in research to improve accuracy on reasoning benchmarks for large models with almost no prompt effort.

Example. Adding 'Let's think step by step.' to a word problem makes the model list its calculation before the final number.

Related: Chain-of-thought prompting, Zero-shot prompting, Few-shot chain-of-thought

Asking a model to do a task using only an instruction, with no worked examples in the prompt. It relies on what the model already learned in training. It is the simplest starting point and often good enough for common tasks.

Example. Prompt: 'Classify this review as positive or negative: The delivery to Pune was two days late.' No sample reviews are shown.

Related: One-shot prompting, Few-shot prompting, Zero-shot chain-of-thought