Matt Watson comments

Results 339 comments of


                                            Matt Watson

Integrate ELECTRA Pretrained Model into Keras_NLP Ecosystem

Electra would be a welcome addition! Thanks! As mentioned in the model contribution guide, first thing to do would be to write a backbone and a checkpoint conversion script from...

Integrate ELECTRA Pretrained Model into Keras_NLP Ecosystem

> I'm trying to implement the backbone of ELECTRA which is similar to BERT with some changes. Please let me know if I'm wrong here cause I'm pretty new to...

Add support for learnable relative position encoding

Would this be something we could use from DeBERTa @chenmoneygithub @abheesht17 if we get the right initialization? Or are the weights/graph too different? If we cannot use this for DeBERTa,...

ConverterError: 'tf.FastWordpieceTokenizeWithOffsets' op is neither a custom op nor a flex op

Thanks for checking this out! Have you tried registering the tf lite custom ops along with the other ops in your converter? That would be the first thing I try...

Port sentence embedding example to keras-core

Thanks @abuelnasr0 !

Sharding for model loading

I think @nkovela1 will be taking a look here!

@TheAthleticCoder thanks! Let us know if you have questions, this is a significant piece of work. One resource is the original BERT squad script -> https://github.com/google-research/bert/blob/master/run_squad.py There is a lot...

Add a SQuAD example

I think for now we can forgo worrying about TensorFlow ops. Let's focus on a solution that is concise and readable. Probably for now we can either: - Compute the...

Add Flash Attention 2 support

Thanks! We definitely want this, and are actively looking at how to best support this in a cross platform way for jax, torch and tf. Ideally this is something we...

high-level generation workflows (e.g. a LM class) on T5 (and Flan T5) is missing.

Thanks @debrupf2946 you are assigned!