Matt

Hi @visionscaper - this seems like something that could work! We haven't experimented with `tf.recompute_grad` in detail but the core code for training our Keras models is in the `train_step`...

@johnryan465 I just did it as an example, I'm sorry! I'm marking off the completed models now.

@robotjellyzone You can! Please note that we accepted a PR yesterday to add the TF decorator to BART, so make sure you're working on the most recent version of the...

cc @sgugger @LysandreJik this should now be ready for review! ESM-1b and ESM-2 models are both supported and the discrepancy between our output and the output from the original model...

(Note that tests will fail until I finish converting and uploading checkpoints)

@LysandreJik Everything renamed ESM -> Esm!

Hi @lexipalmer13 - that code runs fine for me locally, but we did have a lot of compatibility issues with TF 2.11. Version 4.26, which we released two days ago,...

Hi @lexipalmer13, thanks for the error traceback! I believe this error isn't related to `transformers` after all - the issue is an incompatibility specifically triggered by using XLA on TF...

@sayakpaul MLM probability added as an arg and I modularized the loading!

Hey @a8nova! I'm the TF maintainer around here, and this sounds like a great idea. Feel free to ping me if you encounter any difficulties. In general when porting code...