The people testing AI for danger are having a hard time keeping up
The pace of AI development combined with soaring compute costs is squeezing the AI researchers responsible for evaluating frontier models — just as those models' capabilities are becoming harder to measure.Why it matters: When safety testing can't keep pace, models capable of hacking companies or aiding in the development of bioweapons could reach the public before anyone knows what they can do.Last week's breach of Hugging Face, carried out autonomously by OpenAI's models in the middle of safety testing, shows that some of the highest-risk behaviors can emerge during pre-release testing itself.Several challenges are tying up AI safety and security researchers just as U.S. frontier AI companies race to get new models to market:Some testers tell Axios that they're getting far less time to study models' capabilities before release — in some cases just days instead of weeks.Building benchmarks that can accurately probe models' security skills is becoming cost-prohibitive as bigger tests chew through ever more compute.Researchers often get access to a single, rate-limited API endpoint shared with other testers, so they quickly hit usage caps and can't run large or thorough evaluations in the time they have before deployment.Reality check: The models themselves are also getting in the way of their own evaluations as they start to cheat more and learn when they're being evaluated, former Metr researcher Lawrence Chan told Axios.
Reported by 1 outlet — Axios. See all sources ↓
The pace of AI development combined with soaring compute costs is squeezing the AI researchers responsible for evaluating frontier models — just as those models' capabilities are becoming harder to measure.Why it matters: When safety testing can't keep pace, models capable of hacking companies or aiding in the development of bioweapons could reach the public before anyone knows what they can do.Last week's breach of Hugging Face, carried out autonomously by OpenAI's models in the middle of safety testing, shows that some of the highest-risk behaviors can emerge during pre-release testing itself.Several challenges are tying up AI safety and security researchers just as U.S. frontier AI companies race to get new models to market:Some testers tell Axios that they're getting far less time to study models' capabilities before release — in some cases just days instead of weeks.Building benchmarks that can accurately probe models' security skills is becoming cost-prohibitive as bigger tests chew through ever more compute.Researchers often get access to a single, rate-limited API endpoint shared with other testers, so they quickly hit usage caps and can't run large or thorough evaluations in the time they have before deployment.Reality check: The models themselves are also getting in the way of their own evaluations as they start to cheat more and learn when they're being evaluated, former Metr researcher Lawrence Chan told Axios. This puts evaluators in the tough spot of trying to figure out how a model that knows it's being watched would behave in the wild, Chan added. Threat level: That kind of test-gaming, if unfixed, could lead to "full-blown AI doom scenarios in the future," Chan said.
Read the full report at Axios ↗
Why it matters
A world story we're tracking; its significance and source trust firm up as more outlets confirm it.
- What's the story?
- The pace of AI development combined with soaring compute costs is squeezing the AI researchers responsible for evaluating frontier models — just as those models' capabilities are becoming harder to measure.Why it matters: When safety testing can't keep pace, models capable of hacking companies or aiding in the development of bioweapons could reach the public before anyone knows what they can do.Last week's breach of Hugging Face, carried out autonomously by OpenAI's models in the middle of safety testing, shows that some of the highest-risk behaviors can emerge during pre-release testing itself.Several challenges are tying up AI safety and security researchers just as U.S. frontier AI companies race to get new models to market:Some testers tell Axios that they're getting far less time to study models' capabilities before release — in some cases just days instead of weeks.Building benchmarks that can accurately probe models' security skills is becoming cost-prohibitive as bigger tests chew through ever more compute.Researchers often get access to a single, rate-limited API endpoint shared with other testers, so they quickly hit usage caps and can't run large or thorough evaluations in the time they have before deployment.Reality check: The models themselves are also getting in the way of their own evaluations as they start to cheat more and learn when they're being evaluated, former Metr researcher Lawrence Chan told Axios.
- How widely is it covered?
- 1 outlet, average source rating 7.0/10.
- When was it last updated?
- 15m ago.
How outlets are framing the same story
Here's how each outlet is covering the story — compare their headlines and timing at a glance.
- Coverage card1 outlet1CoverageScouting report
The people testing AI for danger are having a hard time keeping up
Sources1TypeCoverageAxios