Context Engineering: Managing Instructions, Data, Memory and State for Better AI Systems

As context windows have expanded, AI applications have gained the capacity to process entire documents, lengthy conversations and large collections of retrieved material in a single call. That capacity creates an appealing assumption: If a model could benefit from more information, we can simply give it more.

Longer context can raise inference cost and latency, however — especially when it’s sent again with each model call. It can also make the model’s job harder: Relevant information may be buried in factually accurate but irrelevant context, while stale or conflicting information introduces new opportunities for error. And for each call, the application may need to resolve identity and permissions, rank or compress the results, expose the appropriate tools, carry forward decisions and then validate the output and preserve the state required for the next call.

Context engineering is how applications make those decisions. It determines how instructions, data, memory, tools and prior state are selected, organized and maintained for a model or agent. Done well, it gives the model a focused, trustworthy view of the data task at hand. Done poorly, it can undermine even a capable model. The failure appears in the output, though its source lies in the system that assembled the input.

Context engineering, a core practice in AI engineering, is the discipline of designing dynamic systems that provide a large language model (LLM) with the right information and tools, in the right format, at the right time. The goal is to help the model reliably complete a task by assembling the context it needs for each call or step.

That context usually contains several layers:

• System instructions that define the model’s role, rules and operating constraints
• User input describing the current request
• Retrieved knowledge from documents, databases or other sources
• Tool definitions that describe the functions the model may call
• Short-term memory from the current conversation or workflow
• Long-term memory stored across sessions
• Structured-output schemas for downstream systems
• Prior state, including identifiers, completed actions and unresolved decisions

Together, these components form the context stack. The exact composition will change from one request to the next.

Leo Rodriguez, Principal Product Marketing Manager, AI/ML at Snowflake, points out that enterprise context has to encode the business meaning behind the data: “In the pre-AI world, a data scientist often had the context in their head: which tables to use, which definitions mattered and which data source of truth to trust. In the AI world, you have to make that context explicit through semantic models, business definitions and governed data relationships.”

Consider an internal support agent being asked why a customer’s invoice increased. The user’s question alone doesn’t contain the information needed for the agent to answer. Before invoking the model, the application may need to resolve the customer and account, retrieve the invoice, identify the applicable contract, locate the current pricing policy and preserve any findings from earlier turns. If the agent is permitted to issue a credit, the model may also need a tool definition that describes the adjustment API and the fields it requires.

The context builder assembles those inputs into a temporary working set. It may query several systems, filter records by user permissions, rank retrieved passages, remove duplicate material and allocate a token budget among instructions, source data, tool schemas and history. Once the model responds, the application validates the result and records any state required for the next call.

This work shapes model behavior without changing model weights. Fine-tuning modifies a model through additional training, helping it learn patterns that should apply across many requests. Context engineering, however, operates at inference time — the model remains unchanged while the surrounding application supplies information specific to the current user, task and point in the workflow.

In agent systems, context engineering is a component of harness engineering. Context engineering focuses on what the model receives at each step, including instructions, retrieved data, memory, tool definitions and prior state. Harness engineering covers the broader runtime that assembles that context and also manages execution, tool calls, permissions, feedback, error handling and progression across steps. Context engineering also applies outside agent harnesses, however, including single-call and RAG applications that still need to construct an effective model input.

The scope of context engineering will depend on the application. In a simple retrieval-augmented generation application, context engineering may center on selecting and formatting relevant documents. In a multistep agent, the system must also track state, coordinate tools, isolate work across sub-agents and decide what information to carry forward after each action.

Model failures receive much of the attention when an AI application produces the wrong result. In practice, the model may have performed reasonably given what it received. The failure, in these cases, was in the system that prepared the model call rather than in the model’s underlying capability.

Common examples include:

• Incorrect retrieval: The application finds text that is semantically similar to the query but belongs to the wrong customer, product version or jurisdiction.
• Missing state: An agent completes one step, then loses the record ID, approval status or constraint needed for the next.
• Stale information: Cached results, stored memories or indexed documents no longer reflect the source system.
• Permission leakage: Retrieval happens across the full corpus, with access filtering applied only after sensitive content has already entered the model context.
• Context-window pressure: Instructions, tool schemas, documents and conversation history compete for limited space, leading the application to truncate or compress useful information.
• Instruction conflicts: System rules, user requests, retrieved text and tool descriptions give the model inconsistent directions.
• Schema mismatch: The model receives an outdated tool definition or generates fields that the downstream API no longer accepts.

Switching to a larger model may hide some of these defects during testing, but it doesn’t repair the underlying pipeline. If the application supplies the wrong account record, a more capable model may simply produce a more convincing incorrect answer.

Prompt engineering focuses on designing the instructions presented to the model at inference time: how the task is phrased, which constraints are included and what kind of response is requested.

Context engineering covers the larger system that builds the model’s working environment around that instruction.

A prompt might say: “Review this support case, determine the likely cause and recommend the next action.” The request is clear, but it leaves several questions unresolved. Which customer records are relevant? What product version is the customer running? Which support policy applies? What actions have already been attempted? Which tools may the agent call, and under whose permissions?

A production application can’t answer those questions through static wording alone. It has to assemble dynamic context at runtime.

The distinction is especially visible in agent workflows. A prompt might describe the overall objective, while the context changes after every step. Once an agent retrieves a record, calls a tool or receives an approval, the system must update its state and construct a new model request. The next call may contain a different subset of documents, a new tool result and revised instructions based on what has already happened.

Prompt engineering remains part of that work. Clear instructions still influence output quality, tool selection and adherence to constraints. Within a context pipeline, however, instruction design sits alongside retrieval, memory, state management, authorization and output validation.

For each model call, the application constructs a temporary working set from information distributed across several systems. That process typically begins before the final prompt is rendered.

Resolve identity, permissions and state

The context builder first establishes who’s making the request and where the request sits within the larger workflow.

Identity determines which data sources, records and tools the application is allowed to expose. Workflow state supplies the information needed to continue from an earlier step: session identifiers, selected entities, completed actions, pending approvals or intermediate results.

In a multitenant application, these values should shape retrieval at query time. Fetching unrestricted records and removing unauthorized results later creates an unnecessary exposure path. Filters based on tenant, role, region or document-level permissions belong as close to the underlying data access as possible.

The same principle applies to tools. A model doesn’t need the definition of every function available to the broader application. The system should expose only the tools permitted for the current user and task.

Select candidate context

Once identity and state are resolved, the application determines which sources are likely to contain useful information.

A request may trigger several forms of retrieval:

• Semantic or keyword search across documents
• SQL queries over structured business data
• Lookups in a conversation or workflow store
• Retrieval from long-term memory
• Selection of relevant tools or APIs
• Loading of system or task-specific instructions

Each source returns a different type of payload. For example, a database query may return rows and typed fields, while a tool registry returns schemas rather than business facts.

The application then needs a common selection strategy. Semantic relevance is one signal, though rarely the only one. Recency, source authority, permissions, entity identifiers and token cost may all affect which candidates survive.

A policy published yesterday, for example, will typically outrank a more semantically similar version from last year. And a customer record with the exact account ID should take precedence over a text match containing a similar company name. For regulated or high-consequence workflows, information from an approved internal source will outrank a web result even when both appear relevant.

Retrieval-augmented generation (RAG) is one of the primary techniques used during this stage.

Rank, filter and budget the results

Retrieval usually produces more material than the model should receive. Before context assembly, the application needs to narrow the candidate set.

Metadata filtering removes records that don’t match the current tenant, time period, product version or other task-specific criteria. Reranking then evaluates the remaining results in relation to the actual request. Some systems use a dedicated reranker model, while others combine retrieval scores with deterministic rules.

Token budgeting adds another constraint. System instructions, output schemas and tool definitions occupy part of the available context window, so conversation state and retrieved data must fit within what remains, with additional capacity reserved for the model’s response.

A context budget might allocate space across several categories:

• Stable system and security instructions
• The current user request
• Workflow state and identifiers
• Retrieved structured data
• Retrieved unstructured passages
• Tool definitions
• Recent conversation turns
• The expected output

These allocations don’t have to be fixed. An analytical request may need more space for query results, while a tool-driven workflow may devote more of the window to function schemas and state. The important decision is explicit prioritization. Without it, whichever component grows fastest is likely to crowd out information the model needs more.

Compress context where necessary

Long documents and accumulated history often exceed the available budget. Compression reduces their size while attempting to preserve the details needed for the current step.

Several approaches are common:

• Remove duplicate passages.
• Extract specific fields or sentences.
• Summarize older conversation turns.
• Replace completed tool traces with their final results.
• Convert prose into a compact structured representation.
• Retain recent turns in full while summarizing earlier ones.
• Split large documents and retrieve only the most relevant chunks.

Deterministic reductions are usually the safest place to begin. For example, if a retrieved contract contains 40 pages but the task concerns renewal dates, extracting the relevant dates and clauses avoids spending tokens on unrelated sections. Likewise, an agent that’s called the same lookup tool several times may only need the current result rather than the full sequence of calls.

Summarization requires more care. A concise summary might omit an exception or qualifier that later affects the answer. For that reason, many systems preserve a link or identifier for the original source even when the model receives a compressed representation.

Compression also affects cost. In applications dominated by input tokens, shorter context reduces the amount of prompt data processed during every call. AI cost optimization strategies often include compression.

Assemble and order the context

After selection and compression, the system renders the final model input. The order of that material influences how the model interprets it. Stable operating rules should remain clearly distinct from retrieved content, and the current task, relevant identifiers and immediate constraints should be easy to find. Supporting material should retain enough metadata for the model — and the user — to identify its source.

A typical context layout might include:

1. System-level role, security and behavioral instructions
2. Task-specific instructions
3. Current workflow state and entity identifiers
4. Retrieved structured facts
5. Retrieved documents or passages
6. Available tool definitions
7. Recent conversation history
8. The required output schema

The exact sequence depends on the model and workload. Testing is still necessary, particularly with long contexts. Models may give uneven attention to information placed in different parts of the window, and a critical constraint can lose influence when surrounded by large retrieval results.

Untrusted content also needs clear boundaries. Text retrieved from a document shouldn’t occupy the same instructional layer as system rules. Delimiters, typed message roles and explicit source labels help the model distinguish evidence from commands embedded within that evidence.

Validate the result before acting

A model response is an intermediate artifact until the surrounding application verifies that it’s safe and usable.

For tool calls, validation may include:

• Checking arguments against the current schema
• Confirming that required fields are present
• Verifying identifiers against the active tenant or session
• Rejecting unsupported actions
• Applying rate, amount or scope limits
• Requiring human approval for sensitive operations

Structured outputs provide a predictable interface for this process. By constraining the response to a defined schema, the application can detect malformed fields before they reach another service. Schema validation doesn’t establish factual correctness. A model may return a valid JSON object containing the wrong customer ID or an unsupported conclusion. Where accuracy carries operational consequences, the application may need additional checks against source data, business rules or an evaluation model.

Persist only the state needed later

After the call completes, the system decides what to retain. Saving the full transcript is straightforward, but retrieving the right fragment several calls later is much harder.

Useful persisted state may include:

• The active record or entity ID
• Decisions already made
• Tools already called
• Confirmed facts and their sources
• Unresolved questions
• Approval status
• The current workflow stage
• A concise summary of earlier interaction

The application should also record provenance: where a fact came from, when it was retrieved and whether it remains current. Without provenance, long-term memory may return a plausible statement that the system cannot verify against its source.

Retention policies deserve equal attention. Stored context may contain customer data, confidential documents or information that later changes. For this reason, memory systems need mechanisms for access control, correction, expiration and deletion.

The execution path above can also be understood through four recurring operations: write, select, compress and isolate.

Write context

Writing context means persisting information outside the model’s immediate context window. Agents frequently write plans, notes, intermediate results or completed actions to an external store. On a later step, the application retrieves the relevant state and places it back into context.

This pattern supplies continuity across calls. The model itself doesn’t retain a durable memory of earlier requests. What appears to be memory is usually the result of the application storing information and reintroducing it when needed.

The format of that stored context influences later retrieval. A structured state object containing customer_id, case_status and approved_action is easier to query and validate than an unstructured paragraph describing the same facts. Narrative summaries remain useful for complex interactions, though important identifiers and status fields often need separate storage.

Select context

Selection determines which information enters the model’s working set. The process may involve RAG, metadata filters, database queries, memory retrieval and tool selection. High-quality selection accounts for more than similarity. The application also needs to evaluate whether a result is authorized, current, authoritative and appropriate for the current workflow stage.

In some systems, retrieval happens in several passes. The first pass identifies candidate sources. A second query retrieves records from the selected source, followed by reranking or deterministic filtering. This approach avoids searching every available corpus for every request.

Compress context

Compression reduces the size of selected information so that it fits within the context window and remains focused, but it does risk information loss. For this reason, compression should be evaluated against realistic edge cases, especially when precise language, exceptions or numerical values affect the outcome.

Isolate context

Isolation keeps one task’s information from interfering with another. For example, a multi-agent system may give a research agent access to source documents while an execution agent receives only approved findings and a restricted set of tools. A SQL agent may need semantic definitions and database metadata, while a writing agent needs the query result and source citations rather than the full schema.

Separate contexts reduce noise and help enforce least-privilege access. They also limit the spread of untrusted content. A sub-agent processing external documents doesn’t need credentials or tool definitions associated with an operational system.

Isolation introduces coordination work, however. The system needs a defined format for passing results between agents, along with rules for which state is shared and which remains local.

Memory is often discussed as a single capability, but production systems usually rely on several forms of storage with different purposes.

• Short-term memory preserves information within the active conversation or workflow. It may consist of recent turns, a running summary and structured state.
• Long-term memory persists information across sessions. Examples include user preferences, prior decisions, reusable facts or records of past interactions.
• Episodic memory stores what happened during a particular run or event.
• Semantic memory stores generalized facts or knowledge extracted from previous interactions.

Regardless of the category, memory requires retrieval. A model can’t use information merely because the application stored it somewhere. The system must identify the relevant memory, verify that it’s still appropriate and include it in the current context.

Poor memory retrieval creates predictable failures. An agent may apply an old preference to a new situation, surface information from the wrong user or rely on a decision that has since been reversed. Recency, identity, task type and source confidence should therefore influence memory selection.

Some information should never enter long-term memory. Sensitive data, temporary credentials and unsupported model conclusions require explicit handling rather than automatic persistence.

Suppose an agent receives a request to investigate declining product usage and prepare recommendations. The first call might contain the user’s request, account permissions and a list of available data tools. After selecting a data source, the next call includes schema context and the selected account. Following a query, the agent receives results and decides whether additional analysis is needed. A final call may receive only the confirmed findings, supporting evidence and a structured report schema.

At each stage, the relevant context changes. Agent state tracks that progression. It may record the current objective, completed steps, selected tools, intermediate outputs and remaining work. Without explicit state, the agent has to reconstruct its progress from conversation text, an approach that grows unreliable as the run lengthens.

Tool orchestration adds further constraints. The context builder needs to expose appropriate tool definitions, keep schemas current and prevent the model from calling functions outside its authorization. After each tool invocation, the system must decide how much of the result to preserve and whether the result should alter the plan.

Loops and retries also affect context. If an agent repeatedly receives an error without a limit or revised instruction, the same context may produce the same failed action. Production systems often track retry counts, error categories and prior attempts separately so that the next call reflects what has already been tried.

Context pipelines are difficult to improve when teams can see only the final prompt and response.

Useful telemetry traces how the working set was constructed:

• Which sources were queried
• Which records and passages were retrieved
• Which filters were applied
• How results were ranked
• Which content was removed or summarized
• How tokens were allocated
• Which tools were exposed
• Which state was loaded and persisted
• Which sources supported the final answer

This trace helps engineers distinguish model behavior from upstream defects. When an answer cites the wrong policy, the team can inspect whether retrieval selected the wrong version, the reranker favored it or the prompt assembly omitted source dates.

Evaluation should test the context pipeline as a whole. A model may answer accurately when given a clean, hand-selected document while failing under realistic retrieval conditions.

Representative tests should include:

• Similar records belonging to different customers
• Conflicting documents
• Stale and current versions of the same policy
• Long conversations with earlier constraints
• Users with different permissions
• Missing or malformed tool results
• Contexts that approach the token limit
• Attempts to inject instructions through retrieved content
• State resumed after an interrupted workflow

Metrics may cover retrieval recall and precision, source correctness, answer quality, instruction adherence, tool-call validity, state retention, latency and cost. For agent workflows, completion rate and recovery from failed steps are often more informative than performance on isolated calls.

Enterprise AI applications need to know more than whether information is relevant. They also need to know whether the information is current, what its fields and metrics mean and whether the application is permitted to expose it. Those requirements make context engineering partly a governance issue.

Snowflake can supply structured and unstructured context close to the governed data from which it originates. Cortex Search can retrieve relevant passages from documents and other text data, supporting the selection stage of a retrieval-augmented generation or agent workflow. Results can include source fields and be filtered through configured attribute columns such as tenant, date or product, helping the application assemble a focused and attributable working set.

For questions over structured data, semantic views define business concepts, metrics and relationships that raw table schemas don’t capture. Cortex Analyst is able to use this semantic context to interpret natural-language questions and generate SQL based on organization-defined business logic. An agent asking for revenue, for example, can use the organization’s defined calculation instead of inferring what the term means from a column name.

Cortex Knowledge Extensions add licensed or proprietary third-party content to the available context through shared Cortex Search services. Applications can retrieve this external knowledge without scraping websites or building a separate ingestion pipeline in the consumer’s environment.

Cortex Agents is designed to manage more of the runtime process. An agent can reason over a request and use configured Cortex Search services, Cortex Analyst resources and custom tools to retrieve information and complete a workflow. Teams specify the available resources, tool descriptions and orchestration instructions, and the managed agent layer can then select among them and carry relevant results through the run.

By assembling context within Snowflake, teams can avoid exporting enterprise data to a separate retrieval environment solely to support AI applications. Snowflake access controls govern who can use each service, while engineers remain responsible for configuring service privileges, retrieval filters, token budgets, state management, validation and evaluation.

Context engineering starts from a practical constraint: A model can act only on the information and tools available in the current call. Larger context windows increase that capacity, but they can’t determine which sources are trustworthy, which definitions apply or what state the task requires.

Some platforms can automate parts of that work, including retrieval, tool selection and context assembly at runtime. But teams still need to define the data, instructions, semantic context, permissions and tools an application can use, then evaluate whether the resulting working set supports the task reliably.

Context engineering gives teams a way to make those decisions explicit. Rather than treating model input as a static prompt or an incidental byproduct of the application, they can design, trace and improve the system that supplies context as the work progresses.