When an LLM generates text, it computes a probability for every possible next token in its vocabulary. Temperature is a scalar value that reshapes this distribution before one token is selected — controlling how conservative or exploratory the model is.
The model first computes raw scores (logits) for each token. Temperature divides all logits before they are converted to probabilities: adjusted logit = original logit ÷ temperature. A small temperature makes differences between logits larger (sharpens the distribution). A large temperature shrinks those differences (flattens the distribution).
Temperature = 0 (Greedy) — always selects the single most probable token. Fully deterministic: same prompt always produces the same output. Best for structured output and code
Temperature 0.1–0.4 — strongly prefers high-probability tokens. Very consistent across runs. Precise and confident but rarely creative. Best for factual Q&A, data extraction, JSON generation
Temperature 0.5–0.8 — balanced range that preserves learned preferences while allowing moderate variation. Responses differ slightly across runs but remain coherent. Best for general assistants and chat
Temperature 0.9–1.5 — model explores less probable tokens. Outputs vary significantly across runs. More creative but higher risk of errors. Best for creative writing and brainstorming
Temperature > 1.5 — distribution becomes so flat that outputs degrade toward random. Rarely useful outside experimentation
Distribution sharply peaks at the most probable token. The model almost always picks the obvious next word.
Same prompt → same response, run after run. Easy to test and debug.
Correct choice for: code, structured data, factual answers, customer support scripts.
Distribution is relatively flat. Lower-probability tokens can win the sample.
Same prompt → different response each time. Less predictable, harder to test.
Correct choice for: tagline generation, story writing, brainstorming diverse alternatives.
Temperature only affects the sampling step — which token is selected from the probability distribution the model already computed. It does not change the model's underlying knowledge or probability estimates. If the model has learned incorrect information, low temperature will reproduce it consistently and confidently. Low temperature makes outputs deterministic, not accurate. Hallucinations happen at every temperature.
Start with these and tune from there:
Code generation, structured output, data extraction: 0.0–0.2
Factual Q&A, summarization: 0.2–0.4
General chat, explanation: 0.5–0.8
Creative writing, brainstorming, taglines: 0.8–1.2
Ask the AI if you need help understanding temperature, how it reshapes the probability distribution, the difference between low and high temperature outputs, or when to use different temperature settings in practice.