LLMs traditionally generate tokens by using one or more hyperparameters to filter a subset of tokens for sampling, such as in methods like top-p and top-k. However, there is a lack of robust support on reliable hyperparameter values other than empirical hyperparameter tuning or practitioner concurrence.
To address the issue, we formulate the P-less method principled in probability and statistics instead of using arbitrary hyperparameter values. We propose P-less as a hyperparameter-free and information-theoretic method for decoding LLM outputs reliably.
Experiments show our method to produce competitive or best LLM generation evaluations against other methods on several math and logical reasoning datasets such as GSM8k and creative writing datasets such as Alpaca. These results provide empirical evidence that P-less is a reliable approach to LLM output generation.
Research submission here.