Daniel Han

In the works!!

No sorry currently not :(

Should be out next week :)

Automatic model support is in the works!!

Wait this is a Phi-3 model but 128K - this requires first mistral-fying it, then adding the special RoPE methodology. @gjyotin305 Yes you're more than welcome to thanks!

Oh you need to update your pytorch version to 2.2

I will check this! Sorry on the issue!

Oh why is `past_key_values = instruction_outputs.past_key_values` there? The KV Cache should be a list of 2 matrices (K and V)

Ok it seems like all past_key_values won't function - I think this will have to be a feature request

Working on it!