hive-mind memories are never recorded to memory.db

Open jimweller opened this issue 1 month ago • 1 comments

The latest commit on this repository is:

  Commit: 25af48cd1c01cbcc4ac6d0dc4346e956db913845
  Author: rUv [email protected]
  Date: Thursday, November 13, 2025 at 11:32:02 AM (EST)
  Message: Merge pull request #871 - 🚀 Release v2.7.34: MCP 2025-11 Compliance & Progressive Disclosure (Alpha)

  This is a merge commit that brings in MCP 2025-11 specification compliance updates and progressive disclosure features in version 2.7.34.

what triggers writting to hive-mind memory? I've been running hive-mind a lot and there are never any memories in it. They are all in swarm memory.

⏺ Let me investigate how hive-mind memory is initialized and when it should be used versus swarm memory.

⏺ Search(pattern: "new Memory(|new CollectiveMemory(|memory.*initialize|DatabaseManager") ⎿ Found 281 files (ctrl+o to expand)

⏺ Read(src/hive-mind/core/HiveMind.ts) ⎿ Read 542 lines

⏺ Read(src/cli/simple-commands/hive-mind.js) ⎿ Read 200 lines

⏺ Read(src/hive-mind/core/DatabaseManager.ts) ⎿ Read 150 lines

⏺ Search(pattern: "memory.store|await.store(|this.memory.store", glob: "src/hive-mind/**/.{ts,js}", output_mode: "content", head_limit: 30) ⎿ Found 7 lines (ctrl+o to expand)

⏺ Bash(ls -la .hive-mind/ 2>/dev/null || echo "No .hive-mind directory found") ⎿ .rw-r--r--@ 16k jimweller 17 Nov 06:46 memory.db

⏺ Bash(find . -name ".db" -type f -not -path "./node_modules/" -not -path "./dist-/" 2>/dev/null | head -20) ⎿ ./benchmark/.hive-mind/hive.db ./docker/docker-test/.swarm/memory.db ./docs/reasoningbank/models/domain-expert/memory.db … +4 lines (ctrl+o to expand)

⏺ Bash(ls -la data/ 2>/dev/null || echo "No data directory") ⎿ No data directory

⏺ Bash(sqlite3 data/hive-mind.db "SELECT name FROM sqlite_master WHERE type='table';" 2>/dev/null || echo "Database doesn't exist or can't be read") ⎿ Database doesn't exist or can't be read

⏺ Perfect! I found the issue. Here's what's happening:

The Problem: Hive-Mind Memory Never Persists

Root Cause: The hive-mind system expects to write to data/hive-mind.db, but the data/ directory doesn't exist in your project. This causes the DatabaseManager to fall back to in-memory storage (see src/hive-mind/core/DatabaseManager.ts:75-94).

Evidence:

Expected location (doesn't exist):

data/hive-mind.db ❌

What exists instead:

.hive-mind/memory.db ✓ (16KB file, but not used by hive-mind core) .swarm/memory.db ✓ (This is for swarm orchestrator, NOT hive-mind)

What's happening:

HiveMind TypeScript Core (src/hive-mind/core/DatabaseManager.ts:75): this.dbPath = path.join(process.cwd(), 'data', 'hive-mind.db'); - Tries to create database at data/hive-mind.db - Fails because data/ directory doesn't exist - Falls back to in-memory storage (line 92-94) - All memories are lost when process exits!
CLI Commands (src/cli/simple-commands/hive-mind.js:134): const dbPath = path.join(hiveMindDir, 'hive.db'); // .hive-mind/hive.db - Creates database at .hive-mind/hive.db - But the core Memory class doesn't use this database!
Swarm Memory (src/swarm/memory.ts:1189): persistencePath: './swarm-memory', // Defaults to ./swarm-memory or .swarm - This is completely separate and works fine

The Fix:

Create the data/ directory so hive-mind can persist memories:

mkdir -p data

Then when you run hive-mind commands, the DatabaseManager will successfully create data/hive-mind.db and persist memories.

Verify it works:

Create data directory

mkdir -p data

Run a hive-mind command

npx claude-flow hive-mind init

Check if database was created

ls -la data/hive-mind.db

Verify tables exist

sqlite3 data/hive-mind.db "SELECT name FROM sqlite_master WHERE type='table';"

Why memories never appeared:

The hive-mind system was using in-memory fallback storage
All memories were stored in RAM and lost when the command finished
The .hive-mind/memory.db file you see is either old/unused or created by a different process

Nov 22 '25 18:11 jimweller

Is there anything I can do on the user side, without modyfing claude-flow to get those features? Making a data folder?

⏺ Let me check if creating a data/ folder or any configuration could enable those CollectiveMemory features:

⏺ Search(pattern: "new CollectiveMemory|CollectiveMemory(", output_mode: "content", head_limit: 20) ⎿ Found 10 lines (ctrl+o to expand)

⏺ Read(src/hive-mind/core/HiveMind.ts) ⎿ Read 20 lines

⏺ Bash(grep -r "CollectiveMemory" ~/tmp/davit/.claude* 2>/dev/null | head -5) ⎿ (No content)

⏺ Bash(grep -r "enablePatternDetection|enableCompression|enableDistribution" src/hive-mind/ | head -10) ⎿ src/hive-mind/memory.js: enableCompression: config.enableCompression !== false, src/hive-mind/memory.js: enablePatternDetection: config.enablePatternDetection !== false, src/hive-mind/memory.js: const compression = options.compression !== false && this.config.enableCompression; src/hive-mind/memory.js: if (this.config.enablePatternDetection) {

⏺ Read(src/cli/simple-commands/hive-mind.js) ⎿ Read 50 lines

⏺ Bash(cd ~/tmp/davit && npx claude-flow hive-mind --help 2>&1 | grep -A 20 "SUBCOMMANDS") ⎿ Error: Exit code 127 cd:1: command not found: __zoxide_z

⏺ Bash(npx claude-flow hive-mind memory --help 2>&1 | head -30) ⎿ node:internal/modules/package_json_reader:314 throw new ERR_MODULE_NOT_FOUND(packageName, fileURLToPath(base), null); ^ … +14 lines (ctrl+o to expand)

⏺ Read(src/hive-mind/memory.js) ⎿ Read 100 lines

⏺ Based on my analysis, no, there's nothing you can do from the user side to enable these features. Here's why:

The Problem - Architectural Disconnect:

CollectiveMemory exists but isn't integrated: // src/hive-mind/memory.js:18 persistenceDir: config.persistenceDir || './data/hive-memory', // Would use data folder enablePatternDetection: config.enablePatternDetection !== false, enableCompression: config.enableCompression !== false,
But HiveMind core uses a different Memory class: // src/hive-mind/core/HiveMind.ts:78 this.memory = new Memory(this.id); // This is Memory, NOT CollectiveMemory!
CollectiveMemory is only instantiated in one place: // src/cli/simple-commands/hive-mind.js:804 (spawn wizard) const memory = new CollectiveMemory({ swarmId, maxSize: 100 }); memory.store('objective', objective, 'context'); // Just stores 4 basic values memory.store('queen_type', ...); // Then never used again!

Why Creating data/ Won't Help:

CollectiveMemory isn't wired to agents - Even when instantiated, it's not connected to the actual agent execution flow
Agents use SwarmMemoryManager - The orchestrator and agents use .swarm/memory.db, not CollectiveMemory
Methods never called - The pattern detection, relationship mapping, and sharing methods exist but are never invoked during actual swarm operations

What Would Be Needed (Code Changes Required):

// Someone would need to modify HiveMind.ts:

this.memory = new Memory(this.id);

this.memory = new CollectiveMemory({ swarmId: this.id, persistenceDir: './data/hive-memory' });

// And wire up pattern detection in agents: await this.memory.detectPatterns(); await this.memory.shareMemory(fromAgent, toAgent, key); await this.memory.analyzeRelationships();

The CollectiveMemory features are essentially "feature-complete but unused code" - they exist in the codebase but aren't integrated into the execution path.

Your current setup with 123 entries in .swarm/memory.db is working correctly for basic coordination. You're not losing critical context, just missing the advanced AI-powered memory features that would require code changes to enable.

Nov 22 '25 19:11 jimweller