Jiyuan Qian

> @jiyuanq thanks for the additional info. The screenshot above looks strange; seems to be a result of data corruption. Can you also share how many `track` method calls per...

Update: I switched to a new EFS with no other users, and tried 7 runs today. 2 of those runs still hit the same issue.. I'll try remote tracker as...

> > Update: I switched to a new EFS with no other users, and tried 7 runs today. 2 of those runs still hit the same issue.. I'll try remote...

@frankfliu thanks for your reply! Just to clarify, I’m already using IValue, and I’m asking if we can enhance the IValue creation api. In the existing IValue api, there’s a...

@frankfliu By “pytorch Java API” do you mean this? https://pytorch.org/javadoc/1.9.0/ it looks like it’s supported there, see “dictStringKeyFrom” I think the c++ api supports generic dict of Dict, so I’m...

@frankfliu I see. Thanks for the additional context! Is it possible to add `stringIValueMapFrom` as a new method? I took a look at the code and it looks like it...

Hi @Narsil amazing work! I tried your branch with https://huggingface.co/huggingface/falcon-40b-gptq today and was able to get 44ms per token on 4 x A10G, as opposed to 180ms per token I...

> > Wondering when this PR will be merged and whether you will be uploading a falcon-40b-instruct-gptq as well? I think many including myself don't have access to a GPU...

> Try using options here:: https://huggingface.co/docs/accelerate/usage_guides/big_modeling > > Notably `device_map = infer_auto_device_map(my_model, max_memory={0: "10GiB", 1: "10GiB", "cpu": "30GiB"})` seems like a good option to reserve enough memory on GPU0 (you...

> That means the layers wasn't loaded at all, probably disk offloaded. I'm not familiar enough with accelerate internals, but there must be some way to fetch the information of...