opencode icon indicating copy to clipboard operation
opencode copied to clipboard

feat(tool): add rlm_repl tool for recursive LLM pattern

Open BowTiedSwan opened this issue 1 day ago • 4 comments

Summary

This PR adds a new experimental rlm_repl tool that enables true Recursive Language Model (RLM) capabilities in OpenCode.

Fixes #8554

Background

Based on the RLM paper, this addresses two key limitations of typical sub-agent implementations:

  1. Can't write O(N) sub-calls as tool calls - the model can't verbalize 10,000 explicit sub-prompts. Recursion must be symbolic through code.
  2. Long prompts can't fit in context - prompts need to be accessible through pointers for symbolic recursion.

What This PR Adds

A new built-in tool that allows the LLM to write JavaScript code that programmatically invokes sub-LLM calls:

// Instead of 10,000 tool calls, write a 3-line loop
for (const chunk of chunks) {
  results.push(await sub_llm(`Analyze: ${chunk}`))
}

Available Functions

Function Purpose
sub_llm(prompt, agent?) Invoke a sub-LLM call, returns string result
sub_llm_parallel(prompts[], agent?) Parallel sub-LLM calls
context.store(key, data) Store data externally (pointer-based)
context.load(key) Load data by pointer
context.chunk(key, size) Split stored data into chunks
context.keys() List all stored keys

Example: Processing Large Data

// Store large data externally (not in LLM context)
context.store("input", largeDataString)

// Chunk it into manageable pieces
const chunkKeys = context.chunk("input", 5000)

// Process each chunk with a sub-LLM
const results = await sub_llm_parallel(
  chunkKeys.map(k => `Analyze this chunk: ${context.load(k)}`)
)

// Return aggregated result
return results.join("\n")

Security

  • Code runs in a sandboxed environment with limited globals
  • Maximum 50 sub_llm calls per execution
  • 5 minute timeout on total execution
  • Context store limited to 10MB total
  • Sub-agents cannot spawn tasks or use rlm_repl (prevents infinite recursion)

How to Enable

OPENCODE_EXPERIMENTAL_RLM_REPL=true opencode
# or
OPENCODE_EXPERIMENTAL=true opencode

Files Changed

  • packages/opencode/src/tool/rlm-repl.ts - New tool implementation
  • packages/opencode/src/tool/registry.ts - Register the tool
  • packages/opencode/src/flag/flag.ts - Add experimental flag

Testing

Tested that the code compiles without syntax errors. The tool uses the same SessionPrompt.prompt() pattern that TaskTool uses for spawning sub-agents.

Notes

  • The sandbox uses new Function() which is a simplified approach. For production hardening, consider using isolated-vm or vm2 for stronger isolation.
  • This is gated behind an experimental flag to allow iteration before making it generally available.

BowTiedSwan avatar Jan 14 '26 23:01 BowTiedSwan