Parameter Playground

Temperature scales the logits (raw prediction scores) before the softmax function. At 0, the model always picks the most probable token. At 2.0, the probability distribution flattens dramatically, making unlikely tokens almost as probable as likely ones. Most production applications use 0.3-0.8.

Top P (nucleus sampling) considers only the smallest set of tokens whose cumulative probability exceeds P. At 0.1, only the most probable tokens (summing to 10% probability) are considered. At 1.0, all tokens are eligible. It works alongside temperature to control output diversity.

Max tokens sets an upper limit on response length. One token is roughly 3/4 of a word in English. Setting this too low truncates useful responses; too high wastes compute and can lead to rambling. Most conversational responses fit within 200-800 tokens.

The system prompt is a special instruction that shapes the model's entire personality, tone, and constraints. It's processed before the user's message and fundamentally changes how the model approaches every response. It's the most powerful "parameter" you can set.

Each model has different training data, architectures, and fine-tuning. Larger models tend to produce more nuanced and detailed output, while smaller models are faster and cheaper. The same parameters can produce noticeably different results across models.