Thorsten Klein

Currently, handling errors coming from ingesting documents within an archive can get pretty nasty. Also, the way we do it right now cannot be parallelized for speedup and first collects...

Add dataset metadata to the context, so the LLM has a basis to decide whether to call the knowledge tool or not requires #88

present the LLM with available dataset and file-level metadata for advanced hybrid search requires #94

... enabling chunk previews in the UI

Implementations: - https://github.com/afjoseph/RAKE.Go (favorite on first sight) - https://github.com/wolfgangmeyers/go-rake/blob/master/rake/rake.go - https://github.com/bohrium272/rake/blob/master/rake.go

... possible as customization option for the `merge` retriever

limit returned chunks by total token limit of the target model

https://github.com/iwilltry42/pdf/issues/4