Temperature

What Is Temperature?

When an LLM generates text, it computes a probability for every possible next token in its vocabulary. Temperature is a scalar value that reshapes this distribution before one token is selected — controlling how conservative or exploratory the model is.

The Mechanism

The model first computes raw scores (logits) for each token. Temperature divides all logits before they are converted to probabilities: adjusted logit = original logit ÷ temperature. A small temperature makes differences between logits larger (sharpens the distribution). A large temperature shrinks those differences (flattens the distribution).

Temperature Levels and Their Effects

0

Temperature = 0 (Greedy) — always selects the single most probable token. Fully deterministic: same prompt always produces the same output. Best for structured output and code

Low

Temperature 0.1–0.4 — strongly prefers high-probability tokens. Very consistent across runs. Precise and confident but rarely creative. Best for factual Q&A, data extraction, JSON generation

Mid

Temperature 0.5–0.8 — balanced range that preserves learned preferences while allowing moderate variation. Responses differ slightly across runs but remain coherent. Best for general assistants and chat

High

Temperature 0.9–1.5 — model explores less probable tokens. Outputs vary significantly across runs. More creative but higher risk of errors. Best for creative writing and brainstorming

Max

Temperature > 1.5 — distribution becomes so flat that outputs degrade toward random. Rarely useful outside experimentation

Comparing Low vs. High Temperature

Low Temperature (0.0–0.3)

Distribution sharply peaks at the most probable token. The model almost always picks the obvious next word.

Same prompt → same response, run after run. Easy to test and debug.

Correct choice for: code, structured data, factual answers, customer support scripts.

High Temperature (0.8–1.5)

Distribution is relatively flat. Lower-probability tokens can win the sample.

Same prompt → different response each time. Less predictable, harder to test.

Correct choice for: tagline generation, story writing, brainstorming diverse alternatives.

Temperature Does Not Equal Accuracy

Common Misconception

Temperature only affects the sampling step — which token is selected from the probability distribution the model already computed. It does not change the model's underlying knowledge or probability estimates. If the model has learned incorrect information, low temperature will reproduce it consistently and confidently. Low temperature makes outputs deterministic, not accurate. Hallucinations happen at every temperature.

Practical Defaults

Rule of Thumb

Start with these and tune from there:
Code generation, structured output, data extraction: 0.0–0.2
Factual Q&A, summarization: 0.2–0.4
General chat, explanation: 0.5–0.8
Creative writing, brainstorming, taglines: 0.8–1.2

Summary

Key Takeaways
  • Temperature reshapes the probability distribution before a token is sampled
  • Low temperature sharpens the distribution — the model prefers the highest-probability tokens
  • High temperature flattens the distribution — lower-probability tokens become more competitive
  • Temperature = 0 is fully deterministic greedy decoding — same prompt, same output every time
  • Temperature affects creativity and variability, not factual accuracy or knowledge
  • Hallucinations can occur at any temperature — low temperature just makes them more consistent
  • Temperature interacts with top-k and top-p sampling, which further constrain the candidate pool

Need Help?

Ask the AI if you need help understanding temperature, how it reshapes the probability distribution, the difference between low and high temperature outputs, or when to use different temperature settings in practice.