Nachama Bar Natan issues

Repositories
Issues
Comments

Results 3 issues of


                                            Nachama Bar Natan

[Bugfix] Fix shape mismatch in cacheblend with partial HBM cache (fixes #854)

This PR fixes a shape mismatch issue in the cacheblend component when handling partial caches stored in HBM. The original code assumed that the key tensors `k` and the `old_k`...

Questions about CacheBlend Implementation

Hi, I was reading the CacheBlend (V1) code and I have a few questions and points I didn't quite understand. I’d appreciate any explanations 🙂 1. If I have prompt...

Handling Non-Prefix Chunk Caching in CacheBlend

Hi, I read your CacheBlend paper and looked into the V1 implementation code — great work! Currently, it seems that only prefix caching is supported. That is, if we have...