Add presets for Electra and checkpoint conversion script

[pranavvp16]

I have uploaded the weights on personal google cloud bucket. The from_preset method works properly in my local setup, but it throws some error in google collab notebook.

[mattdangerw]

Please format your code with ./shell/format.sh, also looks like we have a pretty simple merge conflict to resolve.

[pranavvp16]

Sorry for the delay I'll make the above following requested changes, also I have left my kaggle username above

[pranavvp16]

I have updated the script conversion file according to the new kaggle format. Will I be able to upload the weights from my kaggle account as mentioned in #1422

[mattdangerw]

@pranavvp16 yes, you should be able to create a new model (just not make it public). Please share it with matthewdwatson (me) on kaggle. Thank you!

[pranavvp16]

I uploaded the model and also added collaborator access, but when trying to download the model using .from_preset I get error from kaggle API
You don't have permission to access resource at URL: https://www.kaggle.com/api/v1/models/username/model_name/framework/variant/1/download/config.json Please make sure you are authenticated if you are trying to access a private resource or a resource requiring consent. I have setup the kaggle API credentials in .kaggle file also set the env variables. Still encountering this issue. How can I fix this ? @abuelnasr0 do you have any workaround this ??

[abuelnasr0]

@pranavvp16 Try to upgrade kagglehub to 0.1.9 (latest version). I was using kagglehub 0.1.5 and it wasn't working right, but I was trying to upload a model not using a model. but I was getting a similar error though. https://github.com/Kaggle/kagglehub

UPDATE: If that didn't go right, try to run the script in a kaggle notebook while you are signed in. It worked with me.

[pranavvp16]

@abuelnasr0 I tried running the script in kaggle notebook and still failed notebook. Was your model public while trying to load from kaggle ?

[abuelnasr0]

@pranavvp16 yes it worked with me. And I tried it now with a private model and it worked, but I didn't upgrade kagglehub. It worked with kagglehub=0.1.6 https://www.kaggle.com/mohamedabuelnasr/bloom-preset

[pranavvp16]

notebook I also added the model using the UI of kaggle, also I validated my credentials using kagglehub.login() . @mattdangerw I have added you the collaborators please can you help

[mattdangerw]

@pranavvp16 sorry for the delay! We fell off triage for a second with the Gemma release, but back to normal now.

This seems off to me You don't have permission to access resource at URL: https://www.kaggle.com/api/v1/models/username/model_name/framework/variant/1/download/config.json. We should have your actual username/model name/framework and variant in there. Can you share the keras-nlp code you were running with the presets point to Kaggle? And the link to the Kaggle model you created?

Can help debug with those two pointers. Thanks!

[pranavvp16]

@mattdangerw I have linked notebooks above and my url is also the correct format for the kaggle model. notebook.

[abuelnasr0]

@pranavvp16 Try this in a kaggle notebook

%env  KAGGLE_USERNAME=<YOUR_USERNAME>
%env  KAGGLE_KEY=<YOUR_KEY>

[mattdangerw]

Thanks @pranavvp16 could you still share the link to the model, and share it with me on kaggle? I can't find you model update to look at all the files. Here's me on Kaggle. https://www.kaggle.com/matthewdwatson

Thanks!

[mattdangerw]

Some other notes. I don't believe that you will need to authenticate with Kaggle as long as you are logged in for your Kaggle notebook with the same user you create the Kaggle model with. For things like local testing and colab, you will need to set the KAGGLE_USERNAME and KAGGLE_KEY environment variables.

All user models are private, so it is expected that you can't access https://www.kaggle.com/models/mohamedabuelnasr/bloom, I don't believe that has been shared with you. You should be able to access your own model after proper authentication.

Soon, we will all be able to create public models, and life will get easier. The whole Keras -> Kaggle flow is still under development with some rough edges.

[abuelnasr0]

Some other notes. I don't believe that you will need to authenticate with Kaggle as long as you are logged in for your Kaggle notebook with the same user you create the Kaggle model with. For things like local testing and colab, you will need to set the KAGGLE_USERNAME and KAGGLE_KEY environment variables.

@mattdangerw That should be the expected behaviour, but sometimes kagglehub library returns weird errors. for example I wanted the upload a model

handle = 'mohamedabuelnasr/bloom/keras/bloom_560m_multi'
local_model_dir = '/kaggle/working/bloom_560m_multi'
kagglehub.model_upload(handle, local_model_dir)

and the library returns that error

Model 'bloom' does not exist or access is forbidden for user 'mohamedabuelnasr'. Creating or handling Model...

And Unauthorized at the end of the error message. Although I am logged in. but when I add the env variables, everything work right.

So I suggested to add env variables as a desperate attempt to work around any potential unexpected behaviour from the kagglehub library.

@pranavvp16 Also one thing that's worthy to mention also is that the models takes time to be available may be 10-15 minutes at most, so try to wait a while after uploading it, then download it.

[pranavvp16]

@mattdangerw I tried setting up the env variables still getting the same error. Also I have added you in collaborators with administrator permissions so you can have a look at the model. If everything is fine with the model files maybe we can make it public and try downloading

[mattdangerw]

@pranavvp16 thanks for sharing! I can see the model now, and this looks like two path issue.

1. Your entire model path, for all variants, is electra_base_discriminator_en, though it looks like you have tried to rename it to electra. I am not sure if a rename is possible. You might have to create a new model called electra. The model name != the variant name. Today, if you tried ElectraTokenizer.from_preset("kaggle://pranavprajapati16/electra_base_discriminator_en/keras/electra_base_discriminator_en/2") you would see your assets attach. But the proper fix is making a new model named Electra, which you should see at the link https://www.kaggle.com/models/pranavprajapati16/electra when things are working.
2. You have an extra top-level directory which should not be there. Instead of seeing all your files inside electra_base_generator_en, the config.json, tokenizer.json, etc, should be at the toplevel of your directory.

I've shared a quick example for one variant... Can you see if this works for you? https://www.kaggle.com/models/matthewdwatson/electra https://www.kaggle.com/code/matthewdwatson/electra-preset

[pranavvp16]

Thanks it worked @mattdangerw. Now I will create a new model and then upload its variant and make updates to the electra_presets.py file. Thanks once again!

[mattdangerw]

@pranavvp16 sorry may have missed this, but is this ready for review?

[pranavvp16]

yes yes @mattdangerw

[pranavvp16]

I have made all the changes as suggested

[mattdangerw]

Actually does look like there is an error here. It looks like the tokenizer should be configured to lowercase input, but is not. This is leading to some test failures.

E.g. input_data=["The quick brown fox."], -> an UNK token at the beginning of the sequence and failing tests.

Can you take a look and confirm that we should be lowercasing input for all electra presets? If so I can go ahead an make the changes here, there will be some annoying renames to stick to our conversions--we should call the variants uncased_en instead of _en.

[mattdangerw]

Also, could you try converting the large versions? And adding presets? We should probably include those.

https://huggingface.co/collections/google/electra-release-64ff6e8b18830fabea30a1ab

[pranavvp16]

regarding the tokenizer I found this config for one of the presets. Also I have uploaded the weights for large models

[mattdangerw]

Thanks! I'll get large uploaded, and fix up the lowercasing issues.

[mattdangerw]

OK think this is working! Going to pull this in. @pranavvp16 please let us know if you spot any issues. We should probably test this out with an end to end colab to make sure things are working as intended.

[mattdangerw]

Not that I changed all the preset names to include "uncased" in keeping with bert conventions. electra_small_generator_uncased_en