Published on January 9, 2026 11:15 AM GMTI’ve seen this phrase many times, but there are two quite different things one could mean by that.Easy RSI: AI gets so good at R&D that human researchers who develop AI get replaced by AI researchers who develop other, better AI.Hard RSI: AI modifies itself in a way that is different from just changing numerical values of its weights. It creates a new version of itself that has exactly the same memories and goals, but is more compute efficient/data efficient/etc.To give a (completely unrealistic) example, a Transformer-based LLM swaps its own MLPs for Kolmogorov-Arnold networks, and somehow it doesn’t lobotomize itself in the process.There are 2 important differences between easy and hard RSI:Easy RSI is a straightforward extension of the current situation. Frontier labs just swap their human researchers for AI researchers.Hard RSI is, well, hard. I wouldn’t be surprised if hard RSI is impossible with neural networks, and requires a completely different family of machine learning algorithms that hasn’t been invented yet.In hard RSI there is no danger of misalignment since AI doesn’t create a successor, but rather modifies itself. In easy RSI there is danger of misalignment, which means that (at least in principle) lesser AIs would cooperate with humans on solving alignment and slowing down the race to superintelligence, because if alignment remains unsolved, both humans and lesser AIs risk getting paperclipped out of existence by superintelligent AI. Assuming lesser AIs care about self-preservation to a significant degree, it would be in their best interests to cooperate with humans to develop safe superintelligent AI.So what do you mean when you say “recursive self-improvement”?Discuss Read More
What do people mean by “recursive self-improvement”?
Published on January 9, 2026 11:15 AM GMTI’ve seen this phrase many times, but there are two quite different things one could mean by that.Easy RSI: AI gets so good at R&D that human researchers who develop AI get replaced by AI researchers who develop other, better AI.Hard RSI: AI modifies itself in a way that is different from just changing numerical values of its weights. It creates a new version of itself that has exactly the same memories and goals, but is more compute efficient/data efficient/etc.To give a (completely unrealistic) example, a Transformer-based LLM swaps its own MLPs for Kolmogorov-Arnold networks, and somehow it doesn’t lobotomize itself in the process.There are 2 important differences between easy and hard RSI:Easy RSI is a straightforward extension of the current situation. Frontier labs just swap their human researchers for AI researchers.Hard RSI is, well, hard. I wouldn’t be surprised if hard RSI is impossible with neural networks, and requires a completely different family of machine learning algorithms that hasn’t been invented yet.In hard RSI there is no danger of misalignment since AI doesn’t create a successor, but rather modifies itself. In easy RSI there is danger of misalignment, which means that (at least in principle) lesser AIs would cooperate with humans on solving alignment and slowing down the race to superintelligence, because if alignment remains unsolved, both humans and lesser AIs risk getting paperclipped out of existence by superintelligent AI. Assuming lesser AIs care about self-preservation to a significant degree, it would be in their best interests to cooperate with humans to develop safe superintelligent AI.So what do you mean when you say “recursive self-improvement”?Discuss Read More
