wejoncy

Results 22 comments of wejoncy

[Mobile] Supporting more operators for NNAPI and CoreML

Please reopen it if there are other ops required for CoreML

suggestion, make quantization possible to offload to disk instead of ram

Good question. We can do this definitely. The Transformer API `AutoModelForCausalLM.from_pretrain` has a args named `device_map` doing what you mentioned. It support load the weight into GRAM/VRAM/DISK seperately. It needs...

suggestion, make quantization possible to offload to disk instead of ram

Thank you.

依次按顺序点击文字

使用孪生网络还挺容易识别的，ocr 总是不准确

[build] allow MPI on Unix when NCCL is disabled

MPI is not a hard requirement for Multi-GPUs (of Nvidia or AMD). Hi @stefantalpalaru What was the case when MPI is required for CPU backend? Is there a real senario...

[build] allow MPI on Unix when NCCL is disabled

> I was packaging this software for a Gentoo overlay and I noticed USE_MPI does not enable MPI, due to what is clearly a logic error in the CMake configuration,...

`preload_module_classes` is lost in attach_execution_device_hook.

Hi @muellerzr Could you please take a look at it? Sorry if I ping the wrong contributor could you please help to recommend someone? Thanks

`preload_module_classes` is lost in attach_execution_device_hook.

Could anyone help to look at this issue？

`preload_module_classes` is lost in attach_execution_device_hook.

Hi @SunMarc Could you please take a look at this issue when you get a time? Thanks.

`preload_module_classes` is lost in attach_execution_device_hook.

Sure. I opened a PR to resolve this. https://github.com/huggingface/accelerate/pull/3248