# LongArena Engineering: RSI (Recursive Self-Improvement) at Scale

> **Title**: Production-Grade Multi-Agent Autonomous Engineering
> **Canonical**: https://www.long-arena.com/engineering/rsi
> **Updated**: 2026-09-22

## From single-agent illusion to an industrial dual-worker pipeline

LongArena’s enterprise production systems are continuously maintained by an autonomous multi-agent engineering pipeline: agents diagnose, reproduce, test and resolve live production incidents. This section publishes our first-hand engineering findings, architectural evolution, and production-proven safety protocols.

Every metric below comes from blind comparison experiments on real production incidents. Case identifiers and customer information are sanitized.

## Chapter 1: Why Single Agents Fail in Production

In demos, a single model given a full repository and a complex stack trace appears to complete “analyze, fix, test” in one shot. Months of industrial production practice taught us that the single-agent closed loop hits three insurmountable bottlenecks:

### 1. Context bloat and the reasoning long tail

When one agent explores, traces, debugs and writes tests in a single session, context rapidly stacks to 100–150K tokens with cumulative usage beyond 1.6M. Single-turn latency in high-reasoning mode spikes to 4–5 minutes, while the probability of network interruption and hallucination drift rises exponentially.

### 2. The detective–craftsman cognitive conflict

Reconnaissance demands exploratory sensitivity, fuzzy reasoning and fast temporal demarcation; engineering delivery demands defensive convergence, test-driven development (TDD) and strict code boundaries. Forcing one role to do both produces either “deep investigation with sloppy code” or “rigorous code fixing the wrong root cause”.

### 3. No mechanized handover gate

A single agent easily skips tests under the illusion that it “already understands”, or silently retreats at sandbox boundaries.

To break through this bottleneck, LongArena built the Dual-Worker Pipeline.

## Chapter 2: The Dual-Worker Pipeline (DSH Scout + Codex Craftsman)

We decouple the pipeline by cognitive characteristics into two heterogeneous agents and one hard handover gate.

#### Stage 1: DSH Scout (Worker 1: Scout-Investigator)

- Duty: strictly read-only exploration, code call-chain tracing, and temporal demarcation of database tables through a secured bastion.
- Traits: lightweight headless execution, no accumulated chat history, no business code written.
- Output: an industrial-grade investigation report pinning the direct root cause, affected primary keys, the historical mutation window, and repair recommendations.
- Artifacts: `INVESTIGATION_REPORT.md`

#### Stage 1.5: Guardian Quality Gate & Session Molt

- An external guardian process runs mechanized assertions: the report exists and is non-empty (> 1024 bytes).
- The report must contain the core facts: root cause, source-code location, control-group falsification, and mutation pattern.
- A scan verifies the report carries no plaintext passwords or credential leakage.
- On pass, Termite Protocol rule 9 — Session Molt — fires, severing historical context completely.

#### Stage 2: Codex Craftsman (Worker 2: Craftsman-Solver)

- Starts in a pristine, fresh context with only the upstream investigation conclusions injected.
- TDD contract first: write the reproduction test and establish the red baseline.
- Targeted defensive programming: add integrity checks and slot guards; never wholesale-replace structures.
- Green the suite and pass the full CI gate: 100% tests passing, plus type checks and component builds.
- Close out with a lifecycle report, respecting production release boundaries.
- Artifacts: `regression-before.log`, `regression-after.log`, `LIFECYCLE_REPORT.md`

Artifact chain:

```
INVESTIGATION_REPORT.md
  -> Guardian Gate: non-empty > 1KB, facts complete, no credentials
  -> regression-before.log (RED baseline)
  -> targeted defensive fix
  -> regression-after.log (GREEN, 84/84)
  -> LIFECYCLE_REPORT.md
```

### Blind benchmark on a real production incident (sanitized)

The same production incident (scenario slot-integrity defense) executed once as a single-agent closed loop and once through the dual-worker pipeline:

| Metric | Single-Agent Closed Loop | Dual-Worker Pipeline (DSH + Codex) | Engineering Gain |
| --- | --- | --- | --- |
| Investigation Duration | ~14 min (within mixed session) | 9 min 16 sec (556 sec) | 35% faster |
| Peak Single-Turn Context | 130,711 tokens (severe bloat) | < 15,000 tokens (clean context) | 88% reduction |
| Total Session Token Usage | 1,643,347 tokens | ~180,000 tokens | 89% cost saving |
| Turn Reasoning Latency | 4 ~ 5 min / turn | 20 ~ 40 sec / turn | 85% faster response |
| Test & Verification | 84 tests passed | 84 tests passed (red baseline verified) | Zero timeout risk |

The subject is a real production data-mutation incident (case CASE-SN15, sanitized). The single-agent group ran investigation, repair and tests in one session with one model; the dual-worker group strictly split Stage 1 / 1.5 / 2. Peak context is the maximum single request; token usage is the session total; latency is single-turn reasoning wait time.

## Chapter 3: Production Data Trace — A Five-Dimension Methodology

In cloud-native, multi-end systems, data anomalies often trigger three-way blame: “Generator defect? AI tooling corrupted it? Or an operator edited it by hand?” LongArena distills a general five-dimension temporal tracing methodology:

### 1. Temporal demarcation (T0 / T1 / T2)

Compare the initial asset’s creation time (T0), the healthy in-production window (T1) and the anomaly time (T2). If data was intact with normal interactions at T1, the “initial generator defect” hypothesis is disproved on the spot.

### 2. Control-group falsification

Inspect unaffected sibling groups and NPCs under the same script and tenant. If their markers are intact and the engine triggers normally, a “global engine bug” is ruled out and the case is scoped to specific-object data damage.

### 3. Mutation-pattern diagnosis

Distinguish incremental field updates from hard delete + reinsert: auto-increment ID jumps and update timestamps reveal whether the system wholesale-rewrites entire lists when touching non-key attributes, silently zeroing metadata slots.

### 4. Actor fingerprint triangulation

Extract web access logs within the mutation window: request IPs, user agents, endpoints and operational cadence triangulate the triggering actor.

### 5. Minimal-diff idempotent recovery and contract defense

Never fake a fix by “patching front-end state” or “asking the user to replay a card”; recovery must both restore metadata slots client-side and enforce a non-bypassable slot-integrity contract at the server API.

## Chapter 4: The Safety Net and the Termite Protocol

The greatest risk of unattended RSI is not “failing to build” but “losing control”. LongArena enforces four non-negotiable safety boundaries:

### 1. Read-only bastion and credential isolation

Workers never read environment files containing raw plaintext credentials; a system-verified read-only bastion script opens short-lived channels on non-occupied ports, and no credential may leak to the public network.

### 2. Physical workspace isolation (Git Worktree)

Every incident is built in an isolated physical worktree on a dedicated branch; dependencies are shared through a protected global cache; direct work in the trunk workspace is forbidden.

### 3. Separation of duties and an independent reviewer

After the worker produces code, a separate read-only reviewer process performs security and code review; only then may the patch enter the candidate release queue. Workers may never self-merge or self-release.

### 4. The Termite Protocol self-evolving kernel

Ten evolutionary grammar rules and four safety nets: every action leaves a traceable deposit (DEPOSIT), high-weight risks must escalate (ESCALATE), and over-limit context must molt and hand off (MOLT).