metaseq

metaseq copied to clipboard

## 🚀 Feature Request Implement `finish_reason` as in the OpenAI API specification. Currently it's default to `"length"`. ### Motivation It is useful for saving generation times and generate only until...

frankxu2004

## 🐛 Bug The `stop` parameter provided to the API is not working as expected when `stop = "\n\n"` ### To Reproduce Steps to reproduce the behavior (**always include the...

frankxu2004

How to load an OPT-IML checkpoint and use it for inference?

7

Thanks for your work on OPT-IML. But I am confused about how to load the OPT-IML checkpoints for inference. #### Code Following the instruction in Inference API, I run the...

linmou

I didn't understand the code Can you give a simple example of how to use it based on the existing model How the imported text and model are used as...

597038837

See https://arxiv.org/abs/2201.07520

suchenzang

Remove Megatron dependency - move entirely to Fairscale

1

This is to look into whether or not we can remove our Megatron dependency and rely entirely on our Fairscale dependency (model parallelism implementation seems to be identical between the...

suchenzang

I'm very excited that you built OPT-IML bench with so many tasks/datasets. It seems like a great tool to compare LLMs. Have you released the code? I may just not...

alexkreidler

Integrate LucidRain's RotaryEmbeddings

2

See https://github.com/lucidrains/rotary-embedding-torch/blob/main/rotary_embedding_torch/rotary_embedding_torch.py And from PaLM paper: > We use RoPE embeddings (Su et al., 2021) rather than absolute or relative position embeddings, since RoPE embeddings have been shown to have...

suchenzang

**Patch Description** Adding tests for sequence_parallel flag to check rough equivalence between going through the sequence-parallel code-path (say, with MP 2) vs the current non sequence-parallel run. **Testing steps** black...

bashnick

Pulling Igors barrierless init change in

1

suchenzang