PSeitz

I'm looking for anything that would suggest you run into some border case, e.g. tokens longer than `u16::MAX` or very large memory `memory_budget_in_bytes` Can you share an example document? Term...

@neilyio Do you have the same stack trace? Can you apply the patch I posted? It would narrow it down if Term is constructed incorrectly or read incorrectly. Neither should...

Do you mean this error? ``` thread 'thrd-tantivy-index2' panicked at columnar/src/columnar/writer/column_writers.rs:192:17: assertion `left == right` failed: Input type forbidden. This column has been forced to type U64, received I64(1) left:...

Can you provide a repo? I get ``` ---- tests::test_writer_commit stdout ---- thread 'thrd-tantivy-index0' panicked at /home/pascal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tantivy-columnar-0.2.0/src/columnar/writer/column_writers.rs:192:17: assertion `left == right` failed: Input type forbidden. This column has been forced...

I think the rust docs would be a good place for this, as they don't get outdated so easily and are the entry point for documentation. Or do you think...

Yes, but we can have a walk through in the rust docs or are there features missing? > if I wanted to understand what exactly happened in the internals of...

> Fair enough, we could do it in the Rust docs itself. Do they support > > * flowcharts > > * diagrams > There's a mermaid integration, which looks...

> Yeah fair enough, we can get started on it then. Should we make a separate branch for it ? > I don't think we need a branch for this....

I like the straightforward style of Architecture.md and think it would make a good addition to the docs. Probably makes sense to identify what's missing as high level concepts or...

Indexing or search performance? What type of query?