zer0int
zer0int
Hey everyone, I'm the author of `ViT-L-14-BEST-smooth-GmP-ft.safetensor` @ [https://huggingface.co/zer0int/CLIP-GmP-ViT-L-14/tree/main](https://huggingface.co/zer0int/CLIP-GmP-ViT-L-14/tree/main) and code @ [https://github.com/zer0int/CLIP-fine-tune](https://github.com/zer0int/CLIP-fine-tune) that was used to fine-tune the model. I am assuming the issue may have been due to...
@FreddyFnafbear Here you go (please let me know if you encounter any issues!): - [Full model in HF format](https://huggingface.co/zer0int/CLIP-GmP-ViT-L-14/blob/main/ViT-L-14-BEST-smooth-GmP-HF-format.safetensors) - [Text-Encoder only model in HF format](https://huggingface.co/zer0int/CLIP-GmP-ViT-L-14/blob/main/ViT-L-14-BEST-smooth-GmP-TE-only-HF-format.safetensors) Even if it works...
@MatthewK78 ...and thank you very much for your work, too! 👍 Your 'quick code' is quite sophisticated; especially the resize/interpolate (as well as all the mismatch handling) could indeed come...
That's an interesting idea! I am not sure what would happen, as converting from weight matrix to theta and r affects all parameters, so what happens to those that are...
Sad update for now :( ``` peft_model = get_peft_model(modified_clip_model, peft_config) File "C:\Users\zer0int\AppData\Roaming\Python\Python310\site-packages\peft\mapping.py", line 149, in get_peft_model return MODEL_TYPE_TO_PEFT_MODEL_MAPPING[peft_config.task_type](model, peft_config, adapter_name=adapter_name) File "C:\Users\zer0int\AppData\Roaming\Python\Python310\site-packages\peft\peft_model.py", line 1170, in __init__ super().__init__(model, peft_config, adapter_name) File...
I just opened [an issue](https://github.com/huggingface/peft/issues/1863) with the PEFT folks. Who knows. Maybe they'll be interested - or they can confirm that this absolutely won't work, which would also be a...
Hi Mathieu! You can indeed use any pre-trained weights, convert them to geometric parametrization (GmP) -> fine-tune -> done -> convert back from GmP .theta and .r -> to .weight....
PS: Here's the low-down of the changes I made to CLIP, as written by GPT-4o after 'doing a diff' (I read it and can confirm there are no 'hallucinations' in...
You can just use `import gmpclip as clip` from my repo, instead of `import clip`. That will work for any code (but you may need to adjust the code itself...
Additional info: The issue was due to padding depending on which prompt is shorter (CLIP-L vs. T5). And I learned from "searching for the source of 9999999" that you don't...