happy-transformer icon indicating copy to clipboard operation
happy-transformer copied to clipboard

Add preliminary support for TensorFlow in text generation.

Open swcrazyfan opened this issue 3 years ago • 1 comments

Added preliminary TensorFlow (and MacBook M1 GPU) support for text generation by creating two new classes: TFHappyTransformer and TFHappyGeneration. However, since I created two full new classes by copying and slightly modifying each, I'm guessing this isn't the most efficient for updating in the future. Though, it technically works.

To use TensorFlow models, simply replace HappyGeneration with TFHappyGeneration. Everything else stays the same.

Currently, only generation works. Fine-tuning does not.

I've tested adding TensorFlow support for text2text, and it worked well. However, that is not added to my repository yet.

In order to get the models to load correctly, I had to remove eval() from TFHappyTransformer. I'm not sure why, so this needs to be looked into.

swcrazyfan avatar Feb 15 '22 03:02 swcrazyfan

Yeah the copy-pasted TFHappyTransformer feels off to me. It shouldn't matter in terms of runtime but it will be a pain to update.

Is the usage of TFAutoModelForCausalLM instead of AutoModelForCausalLM the only change?

ted537 avatar Feb 25 '22 15:02 ted537

Nice work! But, version 3 involved a full refactor so this code is now longer compatible with the latest version.

EricFillion avatar Aug 08 '23 02:08 EricFillion