Ditto P S
Ditto P S
I just cloned the repo and installed it from the main branch. But I'm still facing the error. Do I need to install it from any specific branch?
This resolved the issue of the shape error. But now it seems offloading to CPU memory and the process get killed because of CPU overloading. I have 450 GB of...
okay, so is there any suggestion to solve the problem?
I was testing this with the main branch. While loading the state dict, the CPU is taking around 550GB. Yesterday I tried that in a larger instance with 900GB CPU...
Yes, I was using zero3 and LOMO. And I was getting GPU OOM while saving
@x54-729 Please let me know if you have pushed any updates on this to the dev branch. I can try it out.
I have tested the code. I was able to train and save the model. I was testing by training a small dataset that contains the identity(English) of the model. But...
I have tried following the steps. It's creating the pca32.faisis file. But as per the main.py file, it's loading the data from "https://huggingface.co/datasets/meerkat-ml/lemma/resolve/main/filtered_08cdfa755e6d4d89b673d5bd1acee5f6.mk.tar.gz". How do I create that data which...
Getting the same issue
Yes, this is working as expected. It might be an issue with TogetherAI implementation. I'm trying to create a client similar to OpenAI for my inference engine. I would look...