Ammar Ahmad Awan comments

Repositories
Issues
Comments

Results 24 comments of


                                            Ammar Ahmad Awan

Add configs to run int4 inference

> > may I suggest that the just added `kwargs['quant']['weight']['num_bits']` isn't the most user-friendly API as far as `kwargs` go? > > why not have a flat structure of simple...

run-example.sh fails with urllib3.exceptions.ProtocolError: Response ended prematurely

@delock - FYI. Created this issue so we can track and fix it. Please work with folks assigned on this issue.

"RuntimeError: The size of tensor a (5120) must match the size of tensor b (20480) at non-singleton dimension 0" in step3

@oolongoo -- can you please update to the latest DeepSpeedExamples and DeepSpeed and try again? Some LoRA-related fixes have been merged today (https://github.com/microsoft/DeepSpeed/pull/3563) so please try and let us know.

Resizing model embedding when loading the model

@yaozhewei - please respond to this question.