saiajaym

Issue with optimizer instantiation

2

Hi, I get g\the below error whenever I try to create an optimizer. Please help optimizer = driver.instantiate_optimizer(torch.optim.Adam) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/nucleus/lib/python3.11/site-packages/torchpippy-0.1.1+8f549f3-py3.11.egg/pippy/PipelineDriver.py", line 1573, in instantiate_optimizer return PipelineOptimizer( ^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/nucleus/lib/python3.11/site-packages/torchpippy-0.1.1+8f549f3-py3.11.egg/pippy/PipelineDriver.py",...

Use of left padding

2

Left padding makes more sense for auto regressive models. Also HuggingFace's implementation uses left padding for tokenization.