Published on February 8, 2026 6:19 PM GMTWritten in personal capacityI’m proposing UtopiaBench: a benchmark for posts that describe future scenarios that are good, specific, and plausible.The AI safety community has been using vingettes to analyze and red-team threat models for a while. This is valuable because an understanding of how things can go wrong helps coordinate efforts to prevent the biggest and most urgent risks.However, visions for the future can have self-fulfilling properties. Consider a world similar to our own, but there is no widely shared belief that transformative AI is on the horizon: AI companies would not be able to raise the money they do, and therefore transformative AI would be much less likely to be developed as quickly as in our actual timeline.Currently, the AI safety community and the broader world lack a shared vision for good futures, and I think it’d be good to fix this.Three desiderata for such visions include that they describe a world that is good, are specific, and plausible. It is hard to satisfy all properties, and we should therefore aim to improve the pareto frontier of visions of utopia along these three axes.I asked Claude to create a basic PoC of such a benchmark, where these three dimensions are evaluated via elo scores: utopia.nielsrolf.com. New submissions are automatically scored by Opus 4.5. I think neither the current AI voting nor the list of submissions is amazing right now — “Machines of Loving Grace” is not a great vision of utopia in my opinion, but currently ranks as #1. Feedback, votes, submissions, or contributions are welcome.Discuss Read More
UtopiaBench
Published on February 8, 2026 6:19 PM GMTWritten in personal capacityI’m proposing UtopiaBench: a benchmark for posts that describe future scenarios that are good, specific, and plausible.The AI safety community has been using vingettes to analyze and red-team threat models for a while. This is valuable because an understanding of how things can go wrong helps coordinate efforts to prevent the biggest and most urgent risks.However, visions for the future can have self-fulfilling properties. Consider a world similar to our own, but there is no widely shared belief that transformative AI is on the horizon: AI companies would not be able to raise the money they do, and therefore transformative AI would be much less likely to be developed as quickly as in our actual timeline.Currently, the AI safety community and the broader world lack a shared vision for good futures, and I think it’d be good to fix this.Three desiderata for such visions include that they describe a world that is good, are specific, and plausible. It is hard to satisfy all properties, and we should therefore aim to improve the pareto frontier of visions of utopia along these three axes.I asked Claude to create a basic PoC of such a benchmark, where these three dimensions are evaluated via elo scores: utopia.nielsrolf.com. New submissions are automatically scored by Opus 4.5. I think neither the current AI voting nor the list of submissions is amazing right now — “Machines of Loving Grace” is not a great vision of utopia in my opinion, but currently ranks as #1. Feedback, votes, submissions, or contributions are welcome.Discuss Read More
Related Posts
–dangerously-skip-permissions
Published on January 12, 2026 7:37 AM GMTI noticed that some AI-safety-focused people are very…
It Is Reasonable To Research How To Use Model Internals In Training
Published on February 8, 2026 3:44 AM GMTThere seems to be a common belief in…
[CS 2881r] Can We Prompt Our Way to Safety? Comparing System Prompt Styles and Post-Training Effects on Safety Benchmarks
Published on October 28, 2025 2:38 AM GMTThis work was done as an experiment for…