swcrazyfan
swcrazyfan
Do you plan to support T5?
This is what I was about to ask! Would be amazing.
I'm working through the PyTorch Lightning docs, and I think I almost have working TPU support soon! I'll submit a pull request at that time.
To be honest, I'm pretty new to ML, so I'm not sure if I structured the data correctly or even how to tell you the way it's structured. It's the...
> That's a weird notification. There may be a bug, although it shouldn't affect the final training. > > How is your dataset structured? As you said, it doesn't seem...
Okay, makes sense! I'm getting total gibberish outputs from 125M GPT-NEO fine-tuned with on dataset, so I'm going to stick with the official GPT-NEO training for now (Despite needing conversion)....
Good news! With a point in the right direction from someone on the Eleuthera discord, I was able to convert my model and it worked flawlessly to generate text with...
Firstly, I used the Colab notebook from Eleuther's github Readme. Afterwards, I copied the checkpoints into my own colab and converted it. You can check out my colab here: https://colab.research.google.com/drive/16Mg3bc42VSni7hTJhauJBjg3kZDkgorx?usp=sharing
Diffusers officially supports TPU, so I'm guessing it's not a complete rehaul to add it. However, since it's FLAX, I'm not sure exactly how it would be done.
> An [Img2prompt](https://replicate.com/methexis-inc/img2prompt) model already exists for Stable Diffusion. Could we use this model to convert images to prompts? This is a great tool, but it takes a decent GPU...