Matt Watson
Matt Watson
Will need to step through this more carefully soon, but at first blush this looks like it would probably be quite inefficient. `py_function` is quite possible going to be slower...
Support for this at a low-level is now done with https://keras.io/api/keras_nlp/modeling_layers/reversible_embedding/ We could consider opening up a parameter for this for high level tasks for specific architectures. E.g. a way...
This is still something we should add! There's a lot of techniques, but I think the best one to start with is the one described here. https://arxiv.org/pdf/2104.08691.pdf
Sorry I missed this. But for posterity, yes this is quite supported. The easier way to do this for our pre-trained modeling flows is to just pass multiple sequences to...
Thanks! We will take a look.
Great question! and the short answer is this is on the way. There's three things that are currently in flight. 1. An easy upload API (no need for preset utils)....
Thanks! What's the main advantage to switching? I suspect we won't want to do this independently. The whole of Keras and TF, is still using setup.py for the most part,...
Closing this for now! I think the KerasNLP answer is we will follow whatever Keras does just to keep our overall infra similar. So if Keras switches we can switch...
Yes actually! We are currently working or rolling out some support for quantization generally and quantize lora. We will work on some actual docs for this soon, but for now...
Guide on this incoming... https://github.com/keras-team/keras-io/pull/1889