parallelformers
parallelformers copied to clipboard
tunib-ai
Parallelformers: An Efficient Model Parallelization Toolkit for Deployment
```python from transformers import TrainingArguments import torch # get the number of gpus num_gpus = torch.cuda.device_count() if num_gpus > 1: from parallelformers import parallelize parallelize(model, num_gpus=num_gpus, fp16=True, verbose="detail") ``` gives...
Doesn't look like this library still in development? What are some other ones you can point us to that do similar things? Or have HF integrated something similar themselves?
I tried running the example from the readme but received the above error. Does that mean that my hardware is not supported? ## Environment - OS : Ubuntu - Python...
Is there any way to perform tensor parallelism across multiple nodes instead just in a single node? Any tips would be helpful!
Dear Community, I could not find the Falcon models neither in the list of supported nor unsupported models. So are these models supported by parallelformers? If not, are there any...
Hi, Would it be possible to support LLaMA models? It is open and based model for some other large models, such as Alpaca. Here's the official doc: https://huggingface.co/docs/transformers/model_doc/llama Thanks for...
## Add policy for GPT Neox - Addresses : https://github.com/tunib-ai/parallelformers/issues/34 ## Description - Change to add Neox policy consolidated with HF NeoxPretrainedModel ## Linked Issues - Addresses: https://github.com/tunib-ai/parallelformers/issues/34
## How to reproduce ```python from transformers import AutoModelForCausalLM, AutoTokenizer if __name__ == '__main__': model_name = 'facebook/opt-30b' model = AutoModelForCausalLM.from_pretrained(model_name) tokenizer = AutoTokenizer.from_pretrained(model_name) from parallelformers import parallelize parallelize(model, num_gpus=8, fp16=True)...
## Add Bloom to the Auto Policy It would be great to see the recent [bloom](https://huggingface.co/bigscience/bloom) model from bigscience can be added to the auto policy. The Bloom model is...
