Investigate "failed to fill whole buffer" when updating the workspace

[Byron]

From Discord

Hello, I've an issue that appeared today in GitButler, I cannot update the workspace with upstream commits anymore, it fails with the message: failed to fill whole buffer. I've try to make it forget the workspace, delete all the gitbutler/ branches, login again to GitHub but it does not work, I can't integrate upstream changes anymore.

---

As fetches are performed using the Git command-line, we can probably assume that this works as expected. Hence, the failed to fill whole buffer seems to come from Rust code trying to stream data using std::io::Read.

The question is where that happens, and if it happens when reading objects. We should be able to learn more about it by launching GitButler from the terminal, configured to print a backtrace and with detailed logging.

RUST_BACKTRACE=1 LOG_LEVEL=debug /Applications/GitButler\ Nightly.app/Contents/MacOS/gitbutler-tauri

[Byron]

It seems that updating the workspace is indeed a way to produce this issue, see this discord message.

[ali90h]

Thanks, @Byron — I’d like to take this from the backend side (no UI changes). Proposed plan:

Scope (Phase 1: diagnostics only)

• Add structured error context to git invocations (command, cwd, exit code, stderr) and wrap I/O with anyhow::Context, special-casing UnexpectedEof so the “failed to fill whole buffer” path carries actionable context.
• Add tracing spans around Update Workspace and the apply/unapply virtual-branch path (where git2 worktree resets happen), capturing:
  • repo path + virtual branch id (non-PII),
  • git --version, git lfs version,
  • git config --get-regexp '^filter\.lfs\.' (subset only),
  • count/total size of changed files (and top N large paths).
• No behavior change; goal is to make the next occurrence self-explanatory.

Scope (Phase 2: small fix, informed by Phase 1)

• Harden reads that can short-read under LFS/pack scenarios: replace brittle read_exact with a buffered loop/io::copy where appropriate, or retry on UnexpectedEof if safe.
• Ensure worktree resets don’t unintentionally bypass LFS clean/smudge semantics in V3 paths.
• Add a tiny integration repro: temp repo with LFS + multiple large files and an “Update Workspace” exercise.

Questions/constraints

• Is there a preferred place for a shared run_git(..) helper (common crate)?
• Any existing tracing conventions you want me to follow or limits on logging environment details?

If this approach sounds good, I’ll open PR 1: “chore(rust): richer git error context + tracing around Update Workspace” (diagnostics only). Then, based on the captured data, a focused PR 2 with the minimal fix.

[Byron]

Thanks for chiming in!

For context, we will be rewriting the update-workspace functionality which should shake things up a little, but the question remains if the Rust code is meaningfully affected. Maybe filters do play a role, and there is at least one known issue which probably isn't responsible here. However, if a filter process ends early we might see issues like these - definitely something to investigate at some point.

Instead of capturing large amounts of debug data unconditionally, I'd rather want to try to build a reproduction. Generally, object reading even with standard filters can't fail unless it should fail, i.e. if the disk genuinely fails to provide data. At least this is the assumption.

When filter processes come into play, a lot more can go wrong and maybe there is something. git2 doesn't even support these, so it must be us reading in data from the worktree and piping it to a filter. That is unrelated to the workspace update itself, and instead a step that runs to re-integrate uncommitted worktree changes.

• Harden reads that can short-read under LFS/pack scenarios: replace brittle read_exact with a buffered loop/io::copy where appropriate, or retry on UnexpectedEof if safe.

UnexpectedEof is always an error, and all read calls automatically deal with Interrupts that allow a retry.

Ensure worktree resets don’t unintentionally bypass LFS clean/smudge semantics in V3 paths.

This will be fixed once gitoxide can do worktree resets - while git2 is used, these aren't supported. That's unrelated to this issue though.

• Add a tiny integration repro: temp repo with LFS + multiple large files and an “Update Workspace” exercise.

That would be my first step, and it's likely the hardest.

[ali90h]

Thanks for the guidance, Byron.

Plan on my side:

1. Build a minimal LFS repro (no smudge/clean skipping): repo with multiple large LFS files, then add a new large uncommitted file. In GitButler V3: create a virtual branch, commit, then Apply/Unapply (and/or Project → Update workspace).
2. If I can reproduce the error, I’ll focus on the filter pipeline only (where we read from the worktree and pipe into the filter). No blanket logging—just targeted diagnostics on failure.
3. Harden reads there by replacing any brittle read_exact with a buffered loop/io::copy and surface richer error context for UnexpectedEof (read/expected sizes, filter exit code, stderr). No retries on UnexpectedEof, as discussed.
4. Provide the repro steps (or a tiny script) and open a small PR limited to that path.

If you have a preferred module/function as the filter entry point, let me know and I’ll anchor the changes there.

[Byron]

That sounds good, thanks again for your help!

3. Harden reads there by replacing any brittle read_exact with a buffered loop/io::copy and surface richer error context for UnexpectedEof (read/expected sizes, filter exit code, stderr). No retries on UnexpectedEof, as discussed.

If you have a preferred module/function as the filter entry point, let me know and I’ll anchor the changes there.

Once there is a reproduction I think it will be possible to home in the place where it happens and figure out a solution. The issue might well be in gitoxide and it needs to do something differently (or fix its bugs), as generally I don't know what's wrong with read_exact().