PiPPy

PiPPy copied to clipboard

# $ torchrun --nproc-per-node 4 pippy_llama.py import os import copy import torch from transformers import AutoModelForCausalLM, AutoTokenizer from torch.distributed.pipelining import ScheduleGPipe, PipelineStage # Grab the model whole_model = AutoModelForCausalLM.from_pretrained( "meta-llama/Llama-3.1-8B-Instruct",...

LYMDLUT

Hi, I am a student who interested in pipeline parallelism for LLM inference. I have successfully run the [example](https://github.com/pytorch/PiPPy/blob/main/examples/cpu_init/gpt2_cpu_init.py) for GPT mentioned in Pytorch document, so I just want to...

Noblezhong

[BUG] num_stages incorrect and some assertions

1

Hi, First of all, thank you for the great work. I am trying the llama example script with llama2-7b-hf and the following key packages: ``` torch 2.5.0 torchpippy 0.2.0 torchtext...

jq-wei