hohoCode comments

Results 23 comments of


                                            hohoCode

During inference, multi-GPUs are not used with DDP strategy. Only single GPU is used.

I realized that if scripts can be rerun again, the inference will be on 8 GPUs. However, if I put the training function and the inference function together in one...

[Post 1.0] Multimodal distributed training support

Could you please also test try bigger models like Flan-t5-XXL or XL? Currently it seems weird with deepspeed on, looks like a perfect candidate for the deepspeed trainer. Thanks.

[Post 1.0] Multimodal distributed training support

> > Could you please also test try bigger models like Flan-t5-XXL or XL? Currently it seems weird with deepspeed on, looks like a perfect candidate for the deepspeed trainer....

[Post 1.0] Multimodal distributed training support

BTW, I tested run the codes with 'deepspeed_stage_3_offload' given Flan-t5-xl and "bf16" as the data type. The deepspeed shows it has the "must have the same dtype" error: File deepspeed/runtime/zero/linear.py,...

Improve ft-transformer

Also, as you have probably noticed, the new SOTA Tabular DL model seems to be "Trompt": https://arxiv.org/pdf/2305.18446.pdf One existing implementation of this model in Pytorch is here: https://github.com/pyg-team/pytorch-frame?tab=readme-ov-file#benchmark If there...

[BUG] Multi-GPU support issue with MultiModalPredictor and TensorRT

Very important issue.

[timeseries] Add support for multi-GPU training and inference

Absolutely. Thanks. Very much needed.

Help with Key Error when Running Examples from Readme

What is your pyg version? Mine is the latest version.

Extracting sections for 10Q

Thanks for the work. And look forward to this great new feature!

SELA end2end

Great work and cannot wait to try it when ready. Just a general comment: can you make sure o1 is also fully supported with SELA? Because currently it seems o1...