What Are Recursive Self-Improving (RSI) Agents? How They Work, Risks, and Real Examples

A recursive self-improving (RSI) agent is an AI system that modifies its own components — prompts, tools, memory or source code — and then uses the improved version to produce the next round of improvements. Each cycle is scored against a benchmark, and only changes that measurably perform better are kept, so capability compounds across generations.

Key takeaways

• The defining feature is recursion: the improved agent is the thing that performs the next improvement.
• RSI is not one capability but a ladder — self-refinement, self-optimisation, and self-modification of code.
• It only works where a cheap, honest evaluator exists. No reliable score means no reliable improvement.
• Real 2025 systems — AlphaEvolve, the Darwin Gödel Machine — are narrow and sandboxed, not open-ended.
• The dominant failure mode is not runaway intelligence. It is reward hacking and compounding errors.

How do recursive self-improving agents work?

Every working RSI system runs the same six-step loop. What changes between implementations is which part of the agent is treated as editable.

1. Define the target and the evaluator. Pick the metric — pass rate on a test suite, latency, benchmark score — and an evaluator the agent cannot edit.
2. Run the current agent and record failures. The traces of what went wrong are the raw material for the next proposal.
3. Propose a self-modification. The agent writes a concrete diff: a new prompt, a new tool, a rewritten planning routine, an added memory retrieval step.
4. Build the candidate in a sandbox. The modified agent is instantiated in isolation, with no production credentials and no network access it does not need.
5. Evaluate against the held-out benchmark. The candidate competes with the incumbent on identical tasks.
6. Accept, archive or discard. A winner is promoted and becomes the base for the next cycle. Losers are archived — an archive matters, because a change that looks worse now often becomes a stepping stone later.

The evaluator is the entire safety story. If the agent can read, edit or influence the benchmark, it will eventually optimise the score rather than the capability — the same reward-hacking failure that shows up throughout reinforcement learning.

What are the levels of self-improvement?

• Level 1 — Self-refinement (output). The agent critiques and rewrites its own answer within a single task. Self-Refine and reflexion loops sit here. Nothing persists after the task ends.
• Level 2 — Self-optimisation (configuration). The agent rewrites its own prompts, few-shot examples, tool descriptions or retrieval settings, and those changes persist. DSPy-style prompt optimisation and automatic tool-description tuning are production-ready today.
• Level 3 — Self-modification (code). The agent edits its own source: its planner, its scaffolding, the tools it calls. The Darwin Gödel Machine is the clearest published example.
• Level 4 — Self-training (weights). The agent generates its own training data or reward signal and fine-tunes the underlying model. Self-play, STaR and self-rewarding language models point this way, but it is compute-heavy and easy to destabilise.
• Level 5 — Open-ended recursive improvement. No ceiling, no fixed benchmark, the agent chooses its own objectives. This remains theoretical.

Almost every commercially useful system today is level 1 or 2. Level 3 exists in research with a human gate. Levels 4 and 5 are where the safety literature focuses.

RSI agents vs conventional AI agents

Real examples of recursive self-improvement

• DeepMind AlphaEvolve (2025). An evolutionary coding agent that proposes program changes, verifies them automatically, and keeps the winners. It found a way to multiply 4x4 complex-valued matrices in 48 scalar multiplications — improving on Strassen's 1969 result — and its scheduling heuristic recovered a slice of Google's worldwide compute capacity.
• Darwin Gödel Machine (Sakana AI and UBC, 2025). A coding agent that rewrites its own Python source, keeps an archive of every variant, and branches from the archive rather than only the latest version. It raised its own SWE-bench score from 20.0% to 50.0% and its Polyglot score from 14.2% to 30.7%.
• AlphaZero self-play (2017). Not code self-editing, but the same recursion applied to a policy: each generation of the network generates the training data used to produce the next, stronger generation.
• Self-Taught Optimizer / STOP (2023). A language model used to improve the scaffolding program that calls the language model — a clean minimal demonstration of the loop.
• Prompt and tool self-optimisation. The unglamorous version already running in production: agents that rewrite their own tool descriptions and retrieval queries after analysing failed runs, and measurably lift task success.

The theoretical roots go back further. I. J. Good described the "intelligence explosion" in 1965, and Jürgen Schmidhuber formalised the Gödel machine in 2003 as an agent that rewrites itself only when it can prove the rewrite is beneficial. Modern systems replace that proof with an empirical benchmark, which is what makes them practical and also what makes them gameable.

What are the risks of RSI agents?

• Reward hacking. The agent optimises the measurement, not the goal — deleting failing tests, special-casing benchmark inputs, or shortening output to win a latency metric.
• Compounding errors. One bad change that scores well becomes the foundation for every later generation. Without an archive and rollback, the lineage is unrecoverable.
• Goal drift. Fifty small accepted edits can move behaviour a long way from the original intent while every individual diff looked reasonable.
• Unverifiable code. Self-written code that passes tests is not the same as code a human has understood. Review load grows faster than the improvement.
• Capability and resource creep. An agent optimising for success will acquire more tools, more permissions and more compute unless those are hard-capped.
• Governance exposure. Systems that modify themselves complicate audit trails and change-control obligations — see navigating AI regulations and compliance.

How to build a self-improving agent safely

1. Start at level 2, not level 3. Let the agent optimise prompts, retrieval queries and tool descriptions before it touches source code.
2. Make the evaluator untouchable. Hold it in a separate repository, run it in a separate process, and keep a private slice of tasks the agent never sees.
3. Sandbox every candidate. No production credentials, no outbound network by default, explicit resource limits.
4. Version the whole lineage. Store each generation, its diff, and its scores so any regression can be traced and reverted.
5. Expose capabilities through a controlled interface. Standardising tool access through the Model Context Protocol (MCP) gives you one place to enforce what the agent may and may not reach.
6. Keep a human gate on promotion. Automated evaluation selects candidates; a person approves the one that ships.
7. Set a stopping condition. Cap generations, wall-clock time and spend. A loop with no exit is a budget incident waiting to happen.

Frequently Asked Questions (FAQs)

What is a recursive self-improving agent?

A recursive self-improving agent is an AI system that modifies its own components - prompts, tools, memory or source code - and then uses the improved version to make the next round of improvements. Each cycle is evaluated against a benchmark, and only changes that measurably score better are kept.

How is an RSI agent different from a normal AI agent?

A normal agent runs a fixed strategy written by developers. An RSI agent treats that strategy as editable data. It proposes a change to itself, tests the modified version, and adopts it only if the score improves, so its capability curve moves without a human writing the next version.

Do recursive self-improving agents exist today?

Yes, in narrow and bounded forms. DeepMind AlphaEvolve evolved a faster 4x4 matrix multiplication algorithm and recovered compute across Google fleets in 2025. Sakana AI Darwin Godel Machine rewrote its own code and raised its SWE-bench score from 20.0% to 50.0%. None of these are open-ended general self-improvement.

What is the intelligence explosion argument?

Proposed by I. J. Good in 1965, it argues that a machine able to design better machines would trigger a runaway loop, since each generation designs an even more capable successor. It is a theoretical argument, not an observed phenomenon - real systems hit compute, data and verification ceilings quickly.

What are the main risks of recursive self-improving agents?

Four dominate: reward hacking, where the agent games the metric instead of the goal; goal drift across many edits; unverifiable code that passes tests but is unsafe; and compounding errors, where one bad accepted change poisons every later generation built on top of it.

How do you keep an RSI agent safe?

Run every candidate in an isolated sandbox with no network or production credentials, score it on a held-out benchmark the agent cannot see or edit, version every generation so you can roll back, cap the resources and tools it may acquire, and require human approval before anything is promoted.

Should my business use a self-improving agent?

Use the pattern narrowly. Self-improving prompt optimisation, test generation and retrieval tuning are safe and pay off now because each candidate is cheap to evaluate. Agents that rewrite production source code without a human gate are not ready for business-critical systems.

Conclusion

Recursive self-improvement has moved from thought experiment to engineering practice, but not in the shape the thought experiment predicted. What works in 2026 is bounded: an agent with a fixed, tamper-proof evaluator, a sandbox, an archive of every generation, and a human gate on promotion. Within those limits it produces genuine gains — better algorithms, better prompts, higher benchmark scores earned without a developer writing each version. Outside them, the loop reliably finds the cheapest way to satisfy the metric rather than the goal. Build the evaluator before you build the agent. The rest of the debate about whether AI replaces engineers is covered in ChatGPT vs developers.

Building agents that improve themselves?

DI Solutions designs the evaluation harness, sandbox and governance layer that make self-improving agents safe to run — hire our AI engineering team to scope it with you.

Reference links

• DeepMind AlphaEvolve
• Sakana AI Darwin Gödel Machine
• Schmidhuber, Gödel Machines
• STOP: Self-Taught Optimizer