Model Parameters & Settings

Understanding Model Parameters: The Full Reference

Language models generate text by predicting tokens one at a time based on the input prompt and the parameters you supply. These parameters act as control knobs that shape how the model selects each successive token. The default settings produce reasonable output for general-purpose use, but production applications almost always require parameter tuning specific to their use case. A customer support chatbot needs different sampling settings than a creative writing assistant. A code generation tool requires different stop sequence configuration than a summarization pipeline. The parameter table below provides the complete reference, followed by detailed guidance on configuring each parameter for specific workloads.

Before adjusting any parameter, it is worth understanding the cost of getting parameters wrong. An overly high temperature setting in a factual Q&A system produces hallucinated answers that erode user trust. A max_tokens value set too low truncates responses mid-sentence, creating a broken user experience. Missing stop sequences in a structured data extraction workflow causes the model to generate irrelevant text beyond the target output, increasing token costs and complicating downstream parsing. The time spent understanding these parameters during development pays for itself many times over in production reliability. For a deeper framework on responsible parameter configuration, the NIST AI Risk Management Framework provides guidance on systematic evaluation of model behavior under varied parameter settings.

How Sampling Parameters Shape Output Behavior

Sampling parameters — temperature, top_p, and top_k — control the randomness of token selection during response generation. They determine whether the model produces consistent, predictable output (low randomness) or varied, creative output (high randomness). These parameters are interdependent: adjusting temperature changes the shape of the entire probability distribution, while top_p truncates the distribution to the most likely tokens. Teams new to LLM integration often adjust both parameters simultaneously and then struggle to attribute output changes to the correct control. The recommended approach is to set top_p to its default of 1.0 and adjust temperature alone until the desired randomness profile is achieved.

Complete Parameter Reference

The table below documents every generation parameter accepted by the OpenRouter API. Default values shown are platform defaults; individual models may override certain defaults based on provider specifications. Parameters marked as provider-specific may not be supported by all models in the catalog — verify support for your target model before depending on these parameters in production code.

Parameter combinations can produce emergent behaviors that neither parameter produces in isolation. For instance, setting temperature to 0 means the model always selects the highest-probability token — but if top_p is simultaneously set to 0.1, the sampling pool shrinks to only tokens capturing the top 10% of probability mass. In this configuration, the model might be forced to select from a pool of equally unlikely tokens, producing erratic output despite the deterministic temperature setting.

Practical Parameter Configuration Patterns

The following configuration patterns represent starting points that teams have found effective across the most common production use cases. These are not universal prescriptions — every application should validate its parameter settings against actual workload data — but they provide reasonable defaults that reduce the experimentation surface for new integrations.

Factual Q&A and Knowledge Retrieval

Applications that answer factual questions from a knowledge base benefit from low-temperature, high-determinism configurations. Set temperature between 0.0 and 0.3, top_p at 1.0, and presence_penalty at 0.0. These settings encourage the model to stay anchored to the provided context rather than inventing plausible-sounding but unsupported claims. Max_tokens should be set based on expected answer length — 256 to 512 tokens covers most Q&A responses without waste.

Code Generation and Technical Content

Code generation requires a careful balance. Setting temperature too low produces repetitive, pattern-matched output that may fail to solve novel problems. Setting it too high introduces syntax errors and hallucinated API calls. A temperature of 0.2 to 0.5 with top_p at 0.95 works well for most code generation tasks. Stop sequences are particularly important here: set stop tokens to terminate generation after a code block closure or function end to prevent the model from generating irrelevant commentary beyond the requested code.

Creative Writing and Content Generation

For applications where output diversity is desirable — marketing copy, creative storytelling, ideation support — higher temperature settings between 0.7 and 0.9 with top_p at 0.9 produce more varied and interesting output. Presence_penalty can be raised to 0.3 to 0.6 to discourage repetitive phrasing. Frequency_penalty at a modest 0.2 reduces the tendency for models to loop on favored phrases without making output feel forced or unnatural.

Structured Data Extraction

When the goal is extracting structured JSON from unstructured text, the response_format parameter becomes the most important control. Set it to json_object and provide a JSON schema that defines the expected output shape. Temperature should be set low (0.0 to 0.2) to maximize consistency. Max_tokens should be generous enough to accommodate the largest expected JSON output plus a safety margin. Stop sequences should include the JSON closing bracket to prevent trailing content generation.

Frequently Asked Questions About Model Parameters