Sourab Mangrulkar comments

Results 244 comments of


                                            Sourab Mangrulkar

Support deepspeed dynamo

Without using the env variable `TORCHDYNAMO_DEBUG_FUNCTION=forward`, I get the following error: ``` File "/raid/sourab/transformers/src/transformers/models/bert/modeling_bert.py", line 286, in forward mixed_query_layer = self.query(hidden_states)result = forward_call(*args, **kwargs) File "/raid/sourab/miniconda3/envs/hf/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl...

accelerate deepspeed stage 3 numprocess=2 : RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0!

Hello @aohan237 , DeepSpeed isn't compatible with bitsandbytes 4-bit quantization. It is also not compatible with torch compile.

[RFC] Supporting multiple models with DeepSpeed

Hello @stas00, @tjruwase, @muellerzr and @BenjaminBossan; Would be interested in knowing your thoughts.

`VLLM + Llama-Stack` fails when using local images in base64 format with Vision Llama

Gentle ping @ashwinb as you had raised the related commit. Please look into it, Thank you!