Large Language Models: How to Choose and Specialize LLMs for Production

Large language models (LLMs) made one part of natural language processing (NLP) dramatically easier: teams no longer need to train a new model every time they want to solve a new language task. They also moved much of the complexity into the system surrounding the model.

A general-purpose LLM model can summarize a report, extract entities from a contract, generate code, answer questions over proprietary documents and more, often without changing its weights at all. What changes from one application to the next is the context, data, tools and controls wrapped around that model capability.

This creates a crucial set of engineering decisions. Which information should enter the context window? When should an application retrieve external data rather than fine-tune the model? How much LLM inference cost and latency does the workload tolerate? How should teams evaluate outputs when a fluent response can still be wrong? What security and governance controls need to be attached to the data and tools the model can access?

An LLM is a machine learning model trained on large collections of text so it can recognize and generate patterns in natural language, code, markup, structured text and other tokenizable sequences across many tasks. Many prominent modern LLMs use the transformer architecture and contain billions or more learned parameters.

Generality is one of the key differences between LLMs and the task-specific NLP systems that dominated earlier applications. While task-specific models like sentiment classifiers and translation systems are still often the best tool for narrow workloads, an LLM can cover many tasks through the same model interface, with instructions or examples supplied when the application makes the request.

Language modeling itself is much older than the current generation of LLMs. Statistical n-gram models estimated likely words from short preceding sequences. Recurrent neural networks and long short-term memory (LSTM) networks carried learned state across longer passages. The transformer, introduced in the 2017 paper Attention Is All You Need, changed how those dependencies were represented and how efficiently training could be parallelized.

LLMs sit within a broader AI landscape. NLP is the overarching field of computational language processing, and LLMs are one class of model used within it. LLMs are also part of generative AI, a category that extends to models for images, audio, video and other content. Within the LLM category itself, model families include GPT, Llama, Claude, Gemini, Mistral and DeepSeek, among others.

The significance of LLMs, however, is that instead of building a separate model for each language task, teams can start with a broadly pretrained model and specialize its behavior through instructions, context, retrieval or additional training.

Watch Snowflake’s Arun Agarwal, Yusuf Ozuysal and James Cha-Earley discuss why AI model flexibility is crucial for real-world applications:

A general-purpose model can sit inside applications that look very different to the end user.

• Document intelligence uses models to summarize, classify or extract structured information from unstructured documents. Structured output can constrain generation to JSON or another schema so downstream software can process the result without treating unrestricted prose as an interface.
• Question answering and enterprise search commonly combine retrieval with generation. The application identifies relevant passages from a larger corpus, supplies that evidence in context and asks the model to construct an answer from it.
• Natural-language data interfaces connect language models with structured data. An application can translate a natural-language question into SQL or another query representation, while the data system handles query execution and access controls. Semantic metadata can supply the business definitions a model needs to interpret terms that a database schema alone doesn’t capture.
• Tool calling allows an LLM to select among functions exposed by the application and generate arguments for the selected operation. Application code remains responsible for performing the action. Agentic systems extend this pattern across multiple model calls, tool results and intermediate state.
• Code generation maps natural-language instructions and code context into source code, while tests, static analysis and review provide the ordinary software controls needed to determine whether the generated code is correct.

LLMs can also perform familiar NLP tasks such as sentiment analysis, classification, named entity recognition and summarization. But for narrowly defined workloads, specialized models may offer a better quality, latency or cost profile.

Before a transformer can process text, a tokenizer must divide it into units called tokens. A common word may map to one token, while an unusual word, code identifier or piece of punctuation may require several. Tokens are then represented numerically as embeddings so the network can perform mathematical operations over them.

The central mechanism is self-attention. Rather than carrying information forward through a recurrent hidden state, as a recurrent neural network does, a transformer calculates how strongly tokens in a sequence should relate to one another. Multiple attention heads run in parallel, allowing the model to represent different relationships at the same time, while positional information preserves the order in which tokens occur. These operations repeat across many layers, progressively transforming the representation available to the next stage of the network.

That architecture provides the machinery, but architecture alone doesn’t produce a general-purpose LLM. The breadth of capability develops through training: first through large-scale pretraining across varied data, then through post-training that shapes how the model uses those learned representations.

Training determines the values stored in the model’s parameters. For contemporary LLMs, training typically happens in two broad phases: pretraining, which creates a base model, and post-training, which adapts a pretrained model.

The exact pipeline varies among models, and post-training in particular now covers a growing range of techniques. Keeping the two phases separate, however, helps clarify where capabilities and behaviors are being changed.

Pretraining creates the base model

During pretraining, a model learns from very large collections of data through a self-supervised objective. For an autoregressive decoder model, the next token supplies the training target: the model predicts it, calculates the error between its prediction and the target, then uses gradient-based optimization to adjust its parameters.

Repeated across billions or trillions of tokens, those updates encode statistical regularities found throughout the training corpus. The resulting base model can generalize across many kinds of inputs even though it hasn’t been trained separately for every downstream task.

The training data has substantial influence. How data is assembled, filtered, deduplicated and sampled impacts how the model learns. And the mix of sources changes what the model encounters during training. A corpus with more code, scientific literature or multilingual material, for example, supplies a different training distribution from one dominated by general web text.

Post-training adapts the base model

A base model can contain broad learned capability but still perform poorly as an instruction-following assistant or specialized application model. Post-training adapts a pretrained base model for the kinds of interactions and tasks it will handle in practice.

A common starting point is supervised fine-tuning (SFT), where the model trains on examples that pair an input with a desired response. Instruction tuning applies that same basic method across a broad set of instructions and tasks, helping the model generalize its instruction-following behavior beyond the specific examples it saw during training.

From there, preference data can be used to refine which responses the model favors when several outputs are plausible. In an RLHF pipeline, human feedback is used to train a reward model, and reinforcement learning then optimizes the language model against that learned preference signal. Direct preference optimization (DPO) uses the same kind of preferred-versus-rejected response data more directly, without a separate reward-model-and-RL stage.

Once a pretrained model exists, teams have several ways to specialize it: prompting, retrieval-augmented generation (RAG), and fine-tuning.

Prompting

Prompting operates entirely at inference time. Instructions, schemas and examples enter the model’s context for a particular request without changing its weights. Few-shot prompting supplies a small number of example input-output pairs to demonstrate the task. One-shot prompting uses a single example, while zero-shot prompting relies on instructions alone. Examples can be particularly useful when they communicate a classification boundary or output convention more efficiently than a long set of written rules.

RAG

RAG also operates primarily at inference time, although it introduces a retrieval system around the model. The application searches an external corpus of material, selects information relevant to the current request and supplies it to the model as context.

Modern RAG systems may combine lexical search, embeddings, vector search, hybrid retrieval and reranking before the selected passages ever reach the LLM. This makes retrieval especially useful for current, proprietary or source-grounded information: source content can change independently of the model, and the application can preserve provenance for the evidence it retrieved.

Fine-tuning

Fine-tuning takes a different path because it performs additional post-training on the model. It’s worth considering when the recurring behavior or task performance needs to change in ways that prompting can’t reliably produce. Parameter-efficient methods such as LoRA can reduce the amount of state that has to be trained and stored for each adaptation.

These methods can work together. A fine-tuned model can retrieve enterprise data and operate under a system prompt. An application may also select different techniques for different parts of the same workflow.

Prompt instructions, retrieved evidence, conversation history and tool results all compete for space in the context window. Larger windows allow more information to fit into a request, but longer inputs also require more processing and may contain material that’s irrelevant to the task, making it harder for the model to discern what to use.

Context engineering addresses that problem, with the goal of supplying the smallest set of information the model needs to perform the task reliably, rather than filling the available context window simply because the capacity exists.

For enterprise applications, this is also where data architecture starts to influence model behavior directly. The quality, freshness and permissions of the information available to retrieval determine what the model can see when it responds.

The most capable available model isn’t necessarily the best production model. Higher-capability models often carry higher inference cost and latency, and a simpler workload may not benefit enough from the extra capability to justify either. A better starting point is the workload: establish the quality the application needs, evaluate candidate models against representative data, then compare the cost and latency required to reach that threshold.

Establish the quality threshold first

General benchmarks can narrow the candidate set, but they don’t tell you whether a model performs reliably on your documents, prompts and edge cases. Run candidate models against a representative evaluation set and define what acceptable performance looks like for the workload before optimizing around price or speed.

That threshold will differ by task. For example, a customer-facing answer grounded in financial documentation may require substantially stronger factual reliability than a first-pass document classification job. Once several models meet the required bar, their differences in latency, throughput and cost become much more useful for making the final choice.

Right-size the model

For predictable, high-volume workloads, a smaller model that consistently clears the quality threshold may be a better production fit. A smaller model can often be specialized for a particular task, potentially providing the required task performance at a fraction of the inference cost of a larger general-purpose model.

Small language models (SLMs) generally require less memory and compute than larger models, while mixture-of-experts (MoE) architectures may contain a very large total parameter count but activate only a subset of expert components for each token. For production planning, measured workload performance, active computation, latency and serving cost are more informative than the headline parameter number.

Let workload requirements narrow the candidates

Some requirements rule models in or out before cost optimization begins. An application working with images, audio or video, for example, needs the appropriate multimodal support. Code-heavy workloads may benefit from models trained or post-trained for programming, while an application that depends on function calling or structured JSON output should evaluate those behaviors directly rather than infer them from a general benchmark score.

Choose the deployment model at the same time

How the model is available can affect the surrounding architecture as much as its benchmark performance. Open-weights models can be deployed and adapted under the terms of their licenses, giving teams more control over hardware, model configuration, data location and inference optimization. Proprietary models generally shift more of the serving infrastructure to an API or managed service.

The trade-off depends on the workload and the organization operating it. Request volume, latency requirements, data-handling constraints, available engineering capacity and the need for customization can all influence whether running a model directly or using managed inference is the better fit.

Evaluate the model as part of the production system

Because prompt design, retrieval quality, context length and application logic can all change the quality and cost observed in production, model evaluation and optimization are increasingly treated as an iterative workflow.

Production introduces latency, throughput and cost constraints that don’t show up in a model-quality benchmark. Input length affects prefill time and memory, autoregressive generation affects response latency, and concurrent requests compete for serving capacity. Techniques such as KV caching, quantization, batching and speculative decoding can improve different parts of that serving profile.

In other words, production LLMs have workload-level trade-offs. Interactive applications may prioritize time to first token, while batch processing may optimize throughput. Managed inference commonly charges by input and output token volume, making context size and model choice part of the operating cost.

Deployment choices also shape the operational burden. Self-hosting gives teams more control over infrastructure and optimization, while managed inference shifts more of the serving stack to the provider. In either case, teams need visibility into latency, token consumption, errors and model versions as the application changes.

Evaluation should start before production, using a representative test set to establish a baseline for the model and application configuration. It should run again whenever something material changes — the model, prompt, retrieval logic, chunking strategy, tool schema or context construction — so teams can measure whether the change improved or degraded the workload. In production, ongoing evaluation and monitoring help surface regressions, shifts in request patterns and failures that offline tests didn’t capture.

The first step is to define what success looks like for the actual task. For outputs with a known target, evaluation can remain relatively conventional: classification against labeled examples, extraction against expected fields or generated SQL against whether the query returns the correct result. Open-ended generation needs criteria tied to the workload, such as factual accuracy, relevance, completeness or adherence to a required format.

The evaluation set should reflect production conditions rather than only easy or typical examples. Include routine requests alongside cases that are ambiguous, unusually long, missing information or consequential if answered incorrectly.

For RAG, evaluate retrieval and generation separately. If the application returns a bad answer, first ask whether it retrieved the evidence needed to answer the question. If the evidence was present, evaluate whether the response stayed grounded in it and whether it actually addressed the request.

Human review is useful for establishing the reference standard, particularly when several answers could reasonably be acceptable. Once a rubric is stable, LLM-as-a-judge methods can score larger evaluation sets automatically. The judge should first be validated against human labels, since an evaluator model can introduce systematic errors of its own. Research into LLM-as-judge methods has revealed that model-based judges can be more lenient than human evaluators, reinforcing the need to benchmark the evaluator before relying on its scores.

The result should be an evaluation loop rather than a onetime model score: establish a baseline, change one part of the system, rerun the same representative tests and compare the results. This makes it possible to tell whether a new model, retrieval strategy, prompt or context configuration actually improved the workload — and which component to investigate if it didn’t.

LLM applications introduce security and governance concerns through the data they retrieve, the tools they can access and the instructions they process. Those risks are manageable, but they require controls at different layers of the system rather than a single safeguard applied to the model itself.

Protect the application from adversarial inputs and actions

LLM applications accept instructions from more places than a conventional application does: user prompts, retrieved documents, tool responses and other model-generated content can all enter the context. This creates opportunities for prompt injection, where untrusted content attempts to override or redirect the application’s intended instructions.

The risk increases when the model can take actions. A compromised response is more consequential if the model can query a sensitive database, send an external message or invoke another service, so tool access should follow least-privilege principles and sensitive operations should require explicit authorization or validation outside the model. Applications can also separate trusted instructions from untrusted content and constrain which tools, arguments and actions are available to the model.

Reduce unsupported or incorrect outputs

LLMs sometimes generate responses that are fluent but unsupported or incorrect. Retrieval and grounding can reduce that risk by supplying relevant source evidence, while evaluation can test whether responses remain faithful to that evidence and meet the application’s quality requirements.

Those controls don’t eliminate hallucination entirely, however. Retrieval may return incomplete or irrelevant material, and a model can misinterpret evidence. For higher-consequence workloads, applications may also need explicit validation, citations or fallback behavior when the available material is insufficient.

Preserve governance as data enters the LLM system

Data governance concerns start with the information the application is allowed to retrieve and expose to the model. A row, document or field doesn’t drop its access requirements just because it becomes model context, so retrieval should enforce the requester’s authorization before that information reaches the LLM.

The same principle applies to sensitive data such as PII. Teams may need controls over what can enter prompts, how inputs and outputs are retained and logged and which users or services can access those records. Provenance and lineage can record where retrieved information came from and which sources contributed to a response.

Snowflake Cortex AI brings model access, retrieval and application services closer to the enterprise data those systems use. Cortex AI Functions provide access to models from providers including OpenAI, Anthropic, Meta and Mistral AI for tasks such as generation, classification and embedding, while Snowflake’s role-based privileges can control access to individual AI functions.

For RAG and other applications that need unstructured enterprise information, Cortex Search provides low-latency retrieval over data in Snowflake, including vector and lexical search capabilities. Applications can use those results as grounded context while managing access through Snowflake’s governed Cortex Search service and associated Snowflake security controls. Cortex Agents can work across semantic views, Cortex Search services and custom tools, bringing structured and unstructured sources into the same agentic workflow.

Snowflake also provides controls around those applications. Cortex AI Guardrails can protect Cortex Agents and Snowflake CoWork against prompt injection and jailbreak attacks, while usage history and monitoring features provide visibility into agent and search activity. Those controls complement the data permissions already enforced in Snowflake rather than asking the LLM itself to decide what information or actions a requester should be allowed to access.

For teams developing LLM-powered workloads, Cortex AI Function Studio brings model selection, prompt development, evaluation and optimization into the same development workflow. That connects several decisions discussed throughout this article: teams can compare candidate models, evaluate them against workload-specific data and optimize the resulting AI function before moving it into production.

LLMs changed language AI by making broad capability reusable. A team can start with one pretrained model and adapt it across many tasks through prompts, retrieved context, fine-tuning and tools rather than training a specialized language model every time the requirement changes.

But that flexibility moves more responsibility into the application. The model still determines the capabilities available at the center of the system, but retrieval determines which external facts reach it, context engineering determines what competes for its attention, inference infrastructure determines the operating cost and evaluation determines whether its outputs are useful enough to trust.

This is why comparing LLMs only by parameter count or benchmark score quickly reaches its limit. In production, practitioners are working with an assembled system. The useful question is how well that system supplies the model with the right information, constrains what it can do and measures the result under the conditions where the application will actually run.

^{1. Based on Snyk’s reported results as of Aug. 20, 2025; individual results may vary.}