Evaluation of Large Language Models via Coupled Token Generation
📖 Full Retelling
arXiv:2502.01754v3 Announce Type: replace-cross
Abstract: State of the art large language models rely on randomization to respond to a prompt. As an immediate consequence, a model may respond differently to the same prompt if asked multiple times. In this work, we argue that the evaluation and ranking of large language models should control for the randomization underpinning their functioning. Our starting point is the development of a causal model for coupled autoregressive generation, which a
📚 Related People & Topics
Large language model
Type of machine learning model
A large language model (LLM) is a language model trained with self-supervised machine learning on a vast amount of text, designed for natural language processing tasks, especially language generation. The largest and most capable LLMs are generative pre-trained transformers (GPTs) that provide the c...
Entity Intersection Graph
Connections for Large language model:
🌐
Artificial intelligence
3 shared
🌐
Reinforcement learning
3 shared
🌐
Educational technology
2 shared
🌐
Benchmark
2 shared
🏢
OpenAI
2 shared
Mentioned Entities
Original Source
arXiv:2502.01754v3 Announce Type: replace-cross
Abstract: State of the art large language models rely on randomization to respond to a prompt. As an immediate consequence, a model may respond differently to the same prompt if asked multiple times. In this work, we argue that the evaluation and ranking of large language models should control for the randomization underpinning their functioning. Our starting point is the development of a causal model for coupled autoregressive generation, which a
Read full article at source