The growing use of generative models in daily life calls for efficient mechanisms to control their generation, to e.g., produce safe content or provide users with tools to explore style changes. Ideally, such mechanisms should require low volume of unpaired data (i.e., without explicit preference), and should be cheap, both at train and inference time, while preserving output quality. Recent research has shown that such mechanisms can be obtained by intervening exclusively on model activations, with the goal of correcting distributional differences between activations seen when using prompts… Read More
LinEAS: End-to-end Learning of Activation Steering with a Distributional Loss
The growing use of generative models in daily life calls for efficient mechanisms to control their generation, to e.g., produce safe content or provide users with tools to explore style changes. Ideally, such mechanisms should require low volume of unpaired data (i.e., without explicit preference), and should be cheap, both at train and inference time, while preserving output quality. Recent research has shown that such mechanisms can be obtained by intervening exclusively on model activations, with the goal of correcting distributional differences between activations seen when using prompts…
Related Posts
PersonaTeaming: Exploring How Introducing Personas Can Improve Automated AI Red-Teaming
This paper was accepted at the Workshop on Regulatable ML (ReML) at NeurIPS 2025. Recent…
Advancing AI benchmarking with Game Arena
We’re expanding Game Arena with Poker and Werewolf, while Gemini 3 Pro and Flash top…
Control Codegen Spend
This article originally appeared on Medium. Tim O’Brien has given us permission to repost here…