beads icon indicating copy to clipboard operation
beads copied to clipboard

I want to love Beads but the AI generated docs make it impossible

Open srobertson opened this issue 3 months ago • 12 comments

Philosophically, I think this project is on to something. But the bold, overhyped claims (yes, I can spot Claude-style autogenerated docs when I see them) are getting in the way of addressing real bugs.

I’m using worktrees and have followed all the instructions to avoid issues—setting the --no-daemon environment variables, and so on—and I still end up with merge conflicts, a locked database, and lost issues.

I appreciate that this is a very new project, but what’s missing is an architectural overview of the data model. If there were a clear explanation of how JSON, SQL, and Git are supposed to work together, you’d get contributors like me who’d be willing to help sort out viable solutions. As it stands, the repository is almost impenetrable because of the AI-generated documentation. I get the irony, since I also want AI to produce helpful output for me—but this is a plea: can we please have just one human-edited document that explains the data interaction model between Git, JSON, and SQLite?

Apologies if this comes off as an attack—that’s not my intention. I’ve spent several frustrating days trying to get agents to use beads correctly and benefit from the promised workflow improvements. It’s almost there, but I keep running into agents behaving unpredictably: multiple databases get created, worktrees fall out of sync, and the docs don’t explain how to recover from these states. Reading through issue threads, it’s clear the agents are reacting to the issues themselves.

This is a great project with the potential to genuinely change how we develop, but it’s not quite ready to hand full control to the agents yet. I love the ambition, though, and I’m willing to help.

srobertson avatar Nov 25 '25 06:11 srobertson

Here's an example:: https://github.com/steveyegge/beads/blob/main/docs/ARCHITECTURE.md#problem-statement-issue-bd-52

ARCHITECTURE.md is the first place I'd look to get an understanding of the project. It starts out amazaing::

This document describes the internal architecture of the bd issue tracker, with particular focus on concurrency guarantees and data consistency.

Then it's followed immediately by classic Claude slop where it sticks the last thing in Claude's context in a Radom place in the document

Auto-Flush Architecture
Problem Statement (Issue bd-52)
The original auto-flush implementation suffered from a critical race condition when multiple concurrent operations accessed shared state:

I as a reader don't care about this.. I came here for what the intro said

This document describes the internal architecture of the bd issue tracker, with particular focus on concurrency guarantees and data consistency. 

But if I read it, it looks like a focus on one bug. I haven't dug into the history yet but I'm willing to bet that somewher in it, there's the promised overview of the internal architecture..


Looking forward to the "You're right to call me out on this..." response :)

You're on the right track with this project... but you need some safeguards to protect you from the AI overwriting your brilliance.

srobertson avatar Nov 25 '25 06:11 srobertson

And ... I was wrong, just dug through the history of Architecture.md... it looks like it's really the Auto-Flush Architecture... not the overarching architecture of bd.

srobertson avatar Nov 25 '25 06:11 srobertson

You're right to call me out on this. :)

The criticism is fair - ARCHITECTURE.md is really "FlushManager and Blocked Cache Internals", not an architectural overview. And there's no single doc explaining the Git ↔ JSONL ↔ SQLite data model or how to recover from common failure states.

I've filed an internal issue to fix this properly:

  1. Rename current ARCHITECTURE.md to something more accurate (INTERNALS.md or similar)
  2. Write a new, focused ARCHITECTURE.md covering the actual data model
  3. Add recovery procedures for merge conflicts, locked DB, worktree sync issues

Thanks for the constructive feedback - this will make the project more accessible for contributors like yourself.

steveyegge avatar Nov 25 '25 19:11 steveyegge

Additional issues on docs:

  • ./README.md, docs/PROTECTED_BRANCHES.md, commands/sync.md, docs/GIT_INTEGRATION.md and a ton of other docs use beads-metadata as the sync branch, while multiple parts of the code and the default config uses beads-sync.
  • Multiple places use bd daemon start, when the correct command needs a double-hyphen bd daemon --start.
  • Multiple places (one is: docs/CONFIG.md#see-also) refer to a non-existent docs/README.md file.
  • There are scattered .md plan files in /docs and on the root.
  • There is some overlap between the documents in /docs and /examples. For example examples/protected-branch/README.md and docs/PROTECTED_BRANCHES.md

luizlf avatar Dec 03 '25 14:12 luizlf

I entirely agree with the feedback above, whilst also being tremendously excited about Beads and wanting to love it. Additional observations based on first impressions:

  • I find that the top-level README.md is overly long, and includes implementation details which are almost completely irrelevant to the average user. For instance, the whole "Hash-Based Issue IDs" section could be moved into docs/ instead of cluttering up the main README.md.
  • It also proudly boasts of multi-branch support, without any clear explanation of how this works. Am I missing something, or am I expected to blindly trust that it will pick a branch-based workflow which works and doesn't stomp over my existing branches?

Finally, it's a minor point, but why are all the files in docs/ SHOUTING_AT_ME.md? 😝

aspiers avatar Dec 10 '25 19:12 aspiers

TL;DR

STOP with the 100% autogenerated rapid release of "fixes and features".

START refactoring, remove duplication, and testing with real users (I really want to do this with you)

Why the frustration?

100% agree with the comments here. I get the desire to "never read the code" (which I assume includes docs) but this way leads to failure, utter failure. VCs might be interested because the lessons learned will be valuable, but I very much doubt that with this approach you won't be the "1 in 10" that they view as their exit.

Agents are tools not people. Having agents relay on other agents (through docs and code) results in a spiral into chaos. A tiny ommission here, or a small overstatement there quickly becomes an unmanageable mess.

Right now I've got a problem where the tool is reporting orphaned dependencies. They are there, I can find them and the tool reports them (but not their IDs duh). The tool tells me it will fix them with this incantation. It says it did, but it doesn't work. I fix them manually in the JSONL, but they are still in the DB. I blow the DB away and re-initialize. The broken dependencies are back. How - at this point I have absolutely no idea and clearly neither does the AI.

So I've now spent about 5 hours trying to untangle this mess. Finding command after command telling me how to fix it, all failing, all claiming they work. Of course, I have little more chance because the docs are so incomplete and the code is so opaque that I can't find my way around.

I've had Agents spend many hours too. Same result.

As for BD Doctor - it's literally a joke. If my doctor were this ineffective I'd be dead. But wait, by beads database is currently dead.

STOP with the 100% autogenerated rapid release of "fixes and features".

START refactoring, remove duplication, and testing with real users (I really want to do this with you)

rgardler-msft avatar Dec 23 '25 20:12 rgardler-msft

I think it's a little ironic to be criticizing an LLM helper tool for being too LLM generated, but I get the gist of hating hallucinated fixes - AIs are notoriously bad at taking shortcuts which lead to a "good result" but is really just the moral equivalent of adding a test which returns "success" - "Hey, it passed! I'm testing!" This is likely what has happened here.

Unfortunate, beads is probably also not the exception to the rule that if you have an AI create your code 100% and then keep hammering it into compliance, you're not likely to ever review it by hand. "It's not my code any more, it never was, so why would I do a manual review? I'll just ask the AI to review it again!"

Or you can ask the AI to write a LOT of self-tests which also include deliberately inflicted damage tests. Seriously. Write a test which creates a bunch of database entries, then another tests which randomly mangle them, then have the LLM iterate over the doctor until it can actually repair / roll-back the database successfully every single time. That will create an entirely different doctor - a battlefield doctor, if you will. I have used this approach quite successfully. You simply have to add chaos testing and, if the users will support it, also some telemetry from the field which shows actual "anonymized" malfunction cases for the AI to chew on.

jordanhubbard avatar Dec 24 '25 19:12 jordanhubbard

hit the same limitations in spirit.

trying to go from one agent working on a repo, to multiple. having them all work on the same branch confuses them. creating a worktree suddenly brings out a flurry of sharp edges: "we moved your main branch three directories deep into the worktree and your feature branch took over your 'main' directory. read this long incoherent doc (clearly AI written) why", "DAEMON was disabled. want to know if that's fine? another unclear doc here", "did we tell you about sync branches? totally optional, but maybe your only option. unclear".

it's clear i've left the core which works well, an issue tracker for a single agent, and have landed into the land of sharp edges. it also appears out here, as is my personal experience with agents of the gpt-5.2 or opus 4.5 level, agents will fail to keep the required context in mind to carry a project in the right direction over the long term once it becomes complex enough. they can improve yet another sharp edge but are liable to create one or two more in the process. frankly, humans aren't much better, but after spending weeks of time learning the architecture they do tend to have better ideas how to improve the software imo.

time for a beads equivalent with human driven architecture.

anyway, v interesting tool. enjoy working with it a lot. and merry christmas 🎅

alextes avatar Dec 25 '25 10:12 alextes

Lets not make this a philosophical discussion. It's about good software engineering. If you don't review code, whoever it is written by and whatever their level of experience, eventually a catastrophic error will occur. Like https://github.com/steveyegge/beads/issues/740

This project is awesome and provides a very real approach to taming projects that use LLMs. If we want it to survive we need to help Steve manage the code and documentation produced. Steve needs to figure out how to enable that. There are people here to help (like me)

SorraTheOrc avatar Dec 25 '25 21:12 SorraTheOrc

Thanks for the continued feedback! Just pushed some fixes:

Doc inconsistencies addressed (commit 1fc33619):

  • ✅ Standardized sync branch name: beads-metadatabeads-sync throughout docs (matches code)
  • ✅ Fixed daemon syntax: bd daemon startbd daemon --start

Architecture documentation (already done previously):

  • ✅ ARCHITECTURE.md now has proper data model explanation (Git ↔ JSONL ↔ SQLite)
  • ✅ INTERNALS.md exists for FlushManager details
  • ✅ TROUBLESHOOTING.md has comprehensive recovery procedures
  • ✅ FAQ.md for common questions

The core ask from this issue - "a human-edited document explaining the data interaction model" - has been addressed. ARCHITECTURE.md now has clear diagrams showing the three-layer design, write/read paths, and hash-based collision prevention.

For worktree-specific issues (merge conflicts, locked DB), see TROUBLESHOOTING.md and WORKTREES.md.

If there are specific remaining documentation gaps, please file targeted issues so we can track and address them.

steveyegge avatar Dec 26 '25 00:12 steveyegge

ftr, in case one of the workers human or agent still sees this. i for one don't care about "human" written at all! i think every month that goes by a project developed the way beads is will get better, and some day, perhaps 2026, 2027, this approach will irreversibly surpass whatever humans can do on the architecture level. clearly it already surpasses on pace of development.

so my point was more: try to keep more long term architecture goals in mind.

i wish i had a proposal for how to do this. maybe something like beads helping bring into context any issues related to the current. perhaps the agent may notice themselves when working on a issue making a significant change to the architecture that this implies the past issue creating an architecture doc should be reconsidered. this is i believe an approximation for what the human mind does subconsciously.

or maybe if enough agents keep working on enough individual sharp edges as they are brought up, that just works??

i started working on my own version of beads. it's so much fun to work like this 😄. i doubt it'll ever get to the level of beads but man is it fun.

i'll surely have some of my agents steal your best ideas ha. if you want to steal mine: https://github.com/alextes/braid

alextes avatar Dec 26 '25 08:12 alextes

created an issue which is more to the point for the concerns i brought up in between all the "philosophy" 😅 #747

may it make beads even better

alextes avatar Dec 26 '25 09:12 alextes

I was exactly looking for this issue: the current documentation is unbearable. I started to use beads today, and the amount of contradictory information in the documentation is crazy. It is impossible to understand what is correct as per the current tool version, and what was deprecated. As a quick example, the feature that got me the most confused (protected branches) has the examples outdated already: https://github.com/steveyegge/beads/blob/main/examples/protected-branch/README.md.

A couple of issues I faced:

  • I went to https://github.com/steveyegge/beads/blob/main/docs/CLAUDE.md hoping it contained documentation to configure with Claude Code. It was not even close to that. It contains a mix of internal architecture along with some usage commands.
  • Then, I checked https://github.com/steveyegge/beads/blob/main/docs/CLAUDE_INTEGRATION.md. It did have something closer to a user guide, but it's just a bunch of explanations on internal design decisions as well. I did find it more useful though than the other file.
  • The https://github.com/steveyegge/beads/blob/main/docs/CLAUDE.md says: "Use bd --no-daemon in git worktrees (see AGENTS.md for why)". Then, I checked that file (https://github.com/steveyegge/beads/blob/main/AGENTS.md). Not a single mention to why Claude Code should run without a daemon.
    • This is a very confusing point. Should Claude Code run with or without a daemon?
  • In the setup, I did follow to create the beads-sync branch. After several hours of testing, I still did not get it fully work. I asked Claude Code, I researched and read all the documents (https://github.com/steveyegge/beads/blob/main/docs/PROTECTED_BRANCHES.md), and did not find a clear setup. Even Claude Code ended up acknowledging there are several documentation gaps in here.
    • Why do I need an issues.jsonl file in both main and worktree? Shouldn't one be added to .gitignored? The documentation lists both issues.jsonl in the tree diagram, but only clarifies the purpose of one...
    • What about metadata.json and config.yaml? I also had these files duplicated in both the worktree and main branch.
    • Should I run it with the daemon or not? Even the doc regarding this, https://github.com/steveyegge/beads/blob/main/docs/DAEMON.md, is contradicting itself. It says that, if using git worktrees it should be disabled, but then says "no sync branch", which is a worktree on its own. If I use the sync branch, then I should have the daemon, but because that's a worktree, I shouldn't?
    • Why do I get a sync_base.jsonl file in the worktree? It is not added to git; should it be committed?
  • I couldn't get the files in the sync branch to auto-commit. As per Claude Code, it seems to be a bug in beads. It might be, but with the lack of documentation I couldn't tell if that was true or not.
  • I faced the daemon to auto-restart despite I stopped it. I couldn't find any documentation about this behaviour. Is it a bug, or is it intended? I had to manually kill the process to stop that.
  • I see the README (https://github.com/steveyegge/beads/blob/main/README.md) says at the bottom Documentation. I expect this to be user documentation, otherwise it should be labelled as Contributing. I click on Agent Instructions (https://github.com/steveyegge/beads/blob/main/AGENT_INSTRUCTIONS.md) hoping to be some setup guide to use with Claude Code. It is actually some guide for Agents to contribute back to beads.

And please, don't get me wrong. Not everything is bad. I did give it a try with a couple of tasks (ignoring all the syncing and setup issues), and the tracking of tasks is indeed amazing. The amount of context it loses after compaction is way less compared to not using it. I do think it provides a really good value to Claude Code, as it really makes it focus deeper into the tasks and improves its memory.

My key suggestion from this experience is: separate user documentation vs contributor documentation. I think this whole issue of having such a messy documentation is trying to couple the internal designs and decisions along with what final users expect to use. I get the point of trying to explain "design decisions", how the Daemon, Skills and Hooks work, but this is not something that a final user is looking for. Keeping it in the same directory/documentation just makes LLMs get more confused, and they end up mixing stuff all over the place.
What I would do, is:

  • Create a public website using Material for MKDocs (or Zensical, as that's the new one) or anything similar, for end users, just focused on how-to guides, and all the configuration options. This would be the place for LLMs to crawl how to use beads.
    • There can be a quick setup guide for Claude Code in simple mode, along with the extra features like the sync branch. But, focus on Claude Code (or other specific agents), given this is what users seek for when they come across the repo as potential users.
    • Then, you can list all the other configuration options. For example, the Daemon for manual usage, or anything else. You can also include examples for specific How To scenarios, like Claude Code + Protected Branches.
    • Do not include internal stuff or design decisions here. You may include things about how it works from the inside, but not things like how to setup the repo to contribute back, or agents guide for it.
  • Keep the docs/ folder as you have, strictly for contributors or collaborators. Here you can keep design decisions, internal architecture, etc.
  • In the README of the repo, redirect users to use the publicly facing documentation.

*End of rant.* The tool is really useful and I see it can benefit a lot of users. However, I think adoption could greatly increase by investing a bit more in keeping those two topics separate.

barreeeiroo avatar Jan 27 '26 00:01 barreeeiroo

I agree it is good stuff, but the documentation sucks, I am looking through the docs and trying to figure out how do I configure it so beads are never public, cause I like what is internal to be internal, yet they are pushed to a local git server not to github/gitlab/bitbucket or any other and in the meantime I'm a solo developer so there is no team work, no collabration.

My expectation from someone like @steveyegge, must also admit that what ever is written in the book I am constantly living and I thought I hit gold when I discovered beads, is to have a decent documentation that considers the dev ops who are trying to figure out how to react to the agents response "that wasn't me it is not even my code, as if 5 minutes ago it didn't write it"

Honestly, do something about documentation cause at the current stage it is just words building sentences but not meaningful sentences

madanyang avatar Jan 27 '26 01:01 madanyang

Violently agree that user docs should be cleanly separated from contributor docs. There seem to be a ton of design documents in docs/ and then only a tiny handful in the design and dev-notes subdirectories. This would be trivially easy to clean up. Seems that the docs are also missing a decent index. The bottom of the main README.md seems to attempt to provide this but misses a lot of the files.

aspiers avatar Feb 11 '26 01:02 aspiers

https://github.com/steveyegge/beads/pull/1708 - please review

jordanhubbard avatar Feb 13 '26 01:02 jordanhubbard