Best decoding strategy for neural text generation

Determine the best decoding strategy for open-ended text generation from neural language models, identifying which decoding approach should be used to generate continuations from a language model in order to achieve the highest-quality outputs.

Background

The paper observes that maximization-based decoding methods such as beam search tend to produce degenerate outputs (e.g., repetition, blandness), while pure sampling often yields incoherent results. It introduces Nucleus Sampling as a practical improvement and shows it currently outperforms alternatives across several metrics, but does not claim it is definitively optimal.

This motivates a broader unresolved question about which decoding strategy should ultimately be considered best for open-ended generation, beyond the currently observed empirical performance.