happy-transformer
happy-transformer copied to clipboard
Add preliminary support for TensorFlow in text generation.
Added preliminary TensorFlow (and MacBook M1 GPU) support for text generation by creating two new classes: TFHappyTransformer and TFHappyGeneration. However, since I created two full new classes by copying and slightly modifying each, I'm guessing this isn't the most efficient for updating in the future. Though, it technically works.
To use TensorFlow models, simply replace HappyGeneration with TFHappyGeneration. Everything else stays the same.
Currently, only generation works. Fine-tuning does not.
I've tested adding TensorFlow support for text2text, and it worked well. However, that is not added to my repository yet.
In order to get the models to load correctly, I had to remove eval() from TFHappyTransformer. I'm not sure why, so this needs to be looked into.
Yeah the copy-pasted TFHappyTransformer
feels off to me. It shouldn't matter in terms of runtime but it will be a pain to update.
Is the usage of TFAutoModelForCausalLM
instead of AutoModelForCausalLM
the only change?
Nice work! But, version 3 involved a full refactor so this code is now longer compatible with the latest version.