MDK8888
MDK8888
I definitely agree! Apologies for the late response-I will probably upgrade it in the next version, which will be out soon.
Hey, the new version does indeed have torch==2.2! Closing this now :)
Hey, I would love to pick this up!
Hey @A1Liu, thank you so much for guiding me towards that resource, I really appreciate it! I was wondering if there were additional resources to understand the error, i.e. where...
Hey @A1Liu, thanks so much for this-I will definitely check it out!
Hey David, apologies for the late response. Mixtral should support static caching natively, and a new branch should be up this weekend or early next week with the fixes.
Hey, how are you? There's going to be an update for the Llama Model soon-there is a better way to do it than how it is currently done on that...
Hey Nicolas, apologies for the late response-just did this with version 0.3.0!
Hey James, Llama actually already supports static key-value caching natively within transformers. Will put up a fix in the next few days so that models with static key-value caching natively...
Hey, apologies for the late response-that is very interesting indeed! I would have to investigate how LlamaGuard-7b works under the hood to answer :)