Beyond One Output: Visualizing and Comparing Distributions of Language Model Generations
📖 Full Retelling
arXiv:2604.18724v1 Announce Type: new
Abstract: Users typically interact with and evaluate language models via single outputs, but each output is just one sample from a broad distribution of possible completions. This interaction hides distributional structure such as modes, uncommon edge cases, and sensitivity to small prompt changes, leading users to over-generalize from anecdotes when iterating on prompts for open-ended tasks. Informed by a formative study with researchers who use LMs (n=13)
Entity Intersection Graph
No entity connections available yet for this article.
Original Source
arXiv:2604.18724v1 Announce Type: new
Abstract: Users typically interact with and evaluate language models via single outputs, but each output is just one sample from a broad distribution of possible completions. This interaction hides distributional structure such as modes, uncommon edge cases, and sensitivity to small prompt changes, leading users to over-generalize from anecdotes when iterating on prompts for open-ended tasks. Informed by a formative study with researchers who use LMs (n=13)
Read full article at source