candle
candle copied to clipboard
Published
20 hours ago •
huggingface
huggingface
Falcon implementation issues
trafficstars
It seems that clearing cache on current Falcon model implementation is currently not working properly. Every time a second query is run, the cache is not cleared.
Actually there was no way to flush the kv cache in falcon, I've added a function for this in #2066 , you should call it on further queries.
