Luca Antiga comments

Results 171 comments of


                                            Luca Antiga

Fix adapter v2 llm.int8 inference

Great! @Diormiu we'll get this merged as soon as the fix gets, in. If you don't have time we can push this through no problem.

What about a langchain wrapper for lit-llama?

Hey @mfranzon, that would be cool! Is it something you'd be interested in contributing?

Can not finetune, OOM on v100 with batchsize 6

Changing batch size will not change the memory requirements, since we are using gradient accumulation, but changing `micro_batch_size` will. What happens is that forward / backward will be computed with...

Can not finetune, OOM on v100 with batchsize 6

Sorry what do you mean by "multiple-lan support"?

Why FSDPStrategy is so slow-down when I use multi-machine

Hi, can you post the CLI args or code you are using? Also this is with two machines and 8 GPUs per machine?

Why FSDPStrategy is so slow-down when I use multi-machine

Just to confirm: are you running the pretraining command? Maybe try to comment this line out: https://github.com/Lightning-AI/litgpt/blob/main/litgpt/pretrain.py#L174 We have bumped into issues with PyTorch 2.2 and torch.compile recently, let's take...

A potential bug for multi-GPU training

Thanks for the report. Can you try: - running without torch.compile (comment this line out https://github.com/Lightning-AI/litgpt/blob/main/litgpt/pretrain.py#L174) - running with torch.compile but on PyTorch 2.3 Thanks a lot for investigating this....

Add support for memory-efficient and faster optimizers

Agreed

add TensorBase.is_complex

Hey @khushi-411 let us know if you need help!

Move Grad AllReduce Bucketing to inside of `thunder.executors.passes.transform_for_execution` from `torch_autograd.split_forward_backward`

Should we try to get this in? @crcrpar @IvanYashchuk