PowerInfer icon indicating copy to clipboard operation
PowerInfer copied to clipboard
verified publisher icon SJTU-IPADS

Please make a tinyllama v1.0 version for use.

Open kolinfluence opened this issue 1 year ago • 1 comments

curious how it performs on smaller models

kolinfluence avatar Feb 20 '24 12:02 kolinfluence

We are actively considering the support of new models, but model retraining is required to make it run on PowerInfer. Please kindly refer to our FAQ and stay tuned on our model rollout tracked at #93.

hodlen avatar Feb 22 '24 11:02 hodlen