ai-dev-gallery icon indicating copy to clipboard operation
ai-dev-gallery copied to clipboard
Published 1 month ago • verified publisher icon microsoft

[FEATURE] Handle cliptokenizer.onnx in Stable Diffusion sample

Open nmetulev opened this issue 1 year ago • 4 comments

Describe the bug There is no way to get the cliptokenzier.onnx that is used by the Stable Diffusion sample other than exporting the entire project

nmetulev avatar Nov 25 '24 23:11 nmetulev

@nmetulev would having an implementation in Microosft.ML.Tokenizers help here?

luisquintanilla avatar Dec 20 '24 02:12 luisquintanilla

100%. Right now, we depend on an cliptokenizer.onnx model which complicates the implementation.

nmetulev avatar Dec 20 '24 17:12 nmetulev

Great! We're tracking that here.

dotnet/machinelearning#6993

luisquintanilla avatar Dec 20 '24 20:12 luisquintanilla

In the meantime though, if you wanted a C# implementation, this might work for your scenario.

https://github.com/LittleLittleCloud/Torchsharp-stable-diffusion-2/blob/main/Tokenizer.cs

cc: @LittleLittleCloud to confirm this is all that's needed for tokenization of Stable Diffusion.

luisquintanilla avatar Dec 20 '24 21:12 luisquintanilla