pythia
pythia copied to clipboard
EleutherAI
Hi there, The deduplicated dataset has fewer sequences and to keep a consistent token count with the non-deduplicated version the models are trained for ~1.5 epochs (as discussed in the...
Hello, I see your batch_view.py, found that the data does not use a shuffle, but in the gpt-neox library, the data is shuffled. So I want to make sure that...
In the pythia-160m, 410m, 1.4b five-shot the configs say num_fewshot:0 I presume the wrong files were copied (the zero shot ones) pythia-v0/pythia-160m/five-shot
Thanks for your valuable work! I wonder if there are any results on the validation set (if used)? Such as the validation ppl values.
I only find EleutherAI/pythia-410m-seed1, EleutherAI/pythia-410m-seed2, EleutherAI/pythia-410m-seed3, EleutherAI/pythia-410m-seed4 and EleutherAI/pythia-410m-seed6 on HuggingFace. But where are the sedd5,7,8,9? Could you re-upload them?
I tried to shower events for the NLO process p p > t t~ [QCD]. But I am getting the following error. INFO: Generating events INFO: Idle: 0, Running: 3,...
Could you please open-source the training data used between two adjacent checkpoints? Thanks
I noticed that there is a dataset on Hugging Face: https://huggingface.co/datasets/EleutherAI/pythia-memorized-evals. Is there any relationship between this dataset and the project Predictable-Memorization? If so, how can we use this dataset...
```python from transformers import GPTNeoXForCausalLM, AutoTokenizer model = GPTNeoXForCausalLM.from_pretrained( "EleutherAI/pythia-12b", revision="step143000", cache_dir="./pythia-12b/step143000" ) tokenizer = AutoTokenizer.from_pretrained( "EleutherAI/pythia-12b", cache_dir="./pythia-12b/step143000" ) inputs = tokenizer("Hello, I am", return_tensors="pt") tokens = model.generate(**inputs) tokenizer.decode(tokens[0]) ```...
