I'm developing strong feelings about the temperature and top-p. In statistics temperature would be considered randomness or an error term and represent latent Variables in the statistical model. Top-p on the other hand would represent something like a confidence interval or statistical test. If that analogy or perspective is valid. Then the only time you would want anything like randomness or stochastic results is when "seeding" a model or to introduce "noise" intentionally. Therefore, Especially in the context of scientific research with the need for reproducible consistent results that mitigate hallucinations as much as possible, temperature should always be zero, except for defined exploration and ideation (brainsorming/mind-mapping). Likewise during defined project phases such as execution we only want a language model to return highly relevant next word/token/response completions for our tasks, equivalent to a statistically significant result. In some cases a top-p value of 0.5 which is equivalent to a logistic value in logistic regression should suffice but for very highly focused tasks such as refinement and model validation and formalism we must be absolutely certain that the values produced can be done so time and time again in repeated "trials" and therefore a top-p of 0.05 is recommended. For exploration and creative tasks the opposite is true we want a wide range of possibilities where a next token or completion that is not as common, such as in contrarian or "outside the box" thinking should be more likely to be returned. Overall for fact based scientific studies that require grounding in reality we must be very careful to have reproducible results and these are 2 settings that are vastly underappreciated but must be respected in a language models. #project #idea #explore