Lessons learned from applying recursive self improvement to software application

Over the past two weeks, my newsfeed has been filled with apocalyptic warnings about AI, stories of employees leaving frontier labs, and arguments about whether we’re building something we may eventually lose control of. Now, some of the people building the most powerful models are calling for third-party evaluators and urging the industry to “pace the frontier.”

One concern behind these calls is recursive self-improvement: AI helping build more capable AI, which can then accelerate further improvements. The fear is that this cycle could move faster than humans can understand or control. Dario Amodei’s recent essay puts that concern front and center.

While everyone debates doomsday scenarios, we’ve been experimenting with something practical at Bumblebee: a product improvement loop inspired by recursive self-improvement. The premise is simple: use agents to turn logs and customer feedback into improvements, then feed the results into the next cycle.

Our agents review application error logs and customer requests, then organize them into prioritized development tasks. A human reviews each proposal and its supporting evidence before triggering a development agent. That agent implements the change, runs tests, and deploys it to a test environment. A human performs final quality assurance. Regular releases generate new logs and feedback, restarting the loop.

The hardest part has been redacting production data enough to make it safe for agents while keeping it useful. That requires digging through logs, defining redaction rules, and testing them repeatedly. Remove too little, and sensitive information can slip through. Remove too much, and agents lose the context they need. This work is time-consuming, but it’s foundational to the whole process.

After running this loop for two weeks, we started to see both benefits and drawbacks. The agents surfaced bugs and gaps we had previously missed. Often, the clues were already in the logs—we just hadn’t had time to find them. But as the loop continued, familiar issues kept coming back. Agents starting with fresh context would flag problems we’d already reviewed, adding noise and extra work. These early results show that we’re finding overlooked problems. Whether that translates into sustained engineering gains depends on how many findings become useful fixes and how much review they require.

Taking a step back, we believe engineering velocity will continue to accelerate as AI models improve. As implementation becomes faster, the bottleneck shifts toward deciding what deserves to be built and verifying that it works. Generating more tasks is easy. Understanding which ones matter to customers—and avoiding repeated or unnecessary work—requires judgment.

That changes where engineers spend their time. More of the work becomes defining the problem, evaluating evidence, and reviewing outcomes. The measure of success is how much better the product gets with each cycle.