`gix-index` towards 1.0

[Byron]

Only interpret and make accessible all values and figure out how to bridge different formats in a single implementation ideally.

Reading

• [x] V3 reading + V4 reading with multi-threading
• read extensions
  • [x] needed for multi-threading
  • [x] TREE
  • [x] mandatory link and sdir
  • [x] remaining optional extensions
• [x] use bitflags instead of u32 for type safety
• [x] test for long paths and extended flags
• [x] publish v 0.1 of index and bitmap crates
• [x] improve performance - there seems to be a bottleneck when reading large indices, reading a 53MB file takes 2s!
• [ ] evaluate using std::thread::scope() instead of statically scoped threads, which should make it easier (needs Rust 1.63 MVP)

Instantiation

• verification/warm up
  • [x] visualize/print index content
  • [x] implement verification first (to double check what we produce from a tree)
  • [ ] handle verification on linked indices
• [x] index from tree
  • it looks like a basic tree traversal to create entries, maybe it's smart by creating them sorted right way.
  • git can actually traverse multiple trees at the same time, maybe taking advantage of them being sorted. It seems to unpack the same entry on each level for comparison.
  • it inserts without special knowledge of tree traversal, and typically does a binary search unless it naturally inserts past the end
    • It will invalidate the cache tree and update the untracked cache.
    • It checks every path and rejects .git (on macos with case-insensitivity, should be a configuration flag). Handles backslashes in paths, too, on windows, rejecting them.
    • '.git' is always 'outlawed' without case checking, everywhere.
  • The tree cache (extension) is created after inserting all entries, and thus not created on the fly
• [ ] create cache tree
  • see here - interesting is to see promisor objects and 'quick-fetches' without negotiation.

Writing

Definitely round-trip tests with what we are reading.

• [ ] when writing trees, assure these do not contains relative path components or .git named entries
• [ ] write V4
• [ ] write entire tree after clone
  • [ ] 'tree' extension
  • [ ] what about racy timestamps? Needs modification dates after checkout to learn more
  • [ ] parallelization support (via extension)
  • … see if other extensions should also get write support, even though there isn't much use in them after clone in particular.
• [ ] generalized writing - an index after various modifications
  • [ ] protections, while we don't support it
    • [ ] [CE_REMOVED flags](https://github.com/git/git/blob/master/read-cache.c#L2894:L2894
    • [ ] extended flags
  • [x] V2 index, no extensions
    • [x] automatic version - use version that makes sense (2 or 3, depends on extended flags), or preset.
    • [ ] maybe handle the above with some sort of prepare step to do the cleanup like git does, needs mutation

Plumbing

• [x] entries
• [x] info, with details for extensions…
  • [x] TREE
  • [ ] …others

Research

• There is an fs-monitor implementation in Rust, it might be a basis for a pure-Rust notify based implementation, avoiding the watchman tool.
• the ignore crate is probably interesting for .gitignore/exclude files. And it's a big dependency but most definitely worth it.