Add a new blog post: How to use HF endpoints to run Concrete-ML privacy-preserving ML models.

[bcm-at-zama]

Add a new blog post: How to use HF endpoints to run Concrete-ML privacy-preserving ML models.

--

Hello Hugging Face people.

This is a proposed blog post to explain how to deploy our Concrete ML models on the HF endpoints with custom handlers. Concrete ML is about privacy-preserving ML, we already made two HF blogs (https://huggingface.co/blog/encrypted-llm and https://huggingface.co/blog/sentiment-analysis-fhe).

Here, we use your endpoints to directly deploy some models we have saved in https://huggingface.co/zama-fhe?sort_models=modified#models. It was an idea which some of you proposed and that we discussed, a long time ago (CC @philschmid). And it was finally quite easy to make, and it works very well.

[bcm-at-zama]

Sorry, there is something really strange with zh/starcoder2.md file, I don't want to modify it, but in my fork / branch, it has an issue. Maybe you renamed (capitalization) the file? Seems like it creates an issue, at least on my macOS

[bcm-at-zama]

CC the previous HF people who had a look to our previous blogs: @fxmarty @adrinjalali @osanseviero @LysandreJik

[pcuenca]

Sorry, there is something really strange with zh/starcoder2.md file, I don't want to modify it, but in my fork / branch, it has an issue. Maybe you renamed (capitalization) the file? Seems like it creates an issue, at least on my macOS

This should have been resolved :)

[bcm-at-zama]

Thanks a lot for the reviews, @pcuenca and @osanseviero. I'm away from keyboard for some days, I'll follow your remarks when I'm back.

[bcm-at-zama]

I'll have another look tomorrow, to be sure I took all your comments. Not sure what happened when I tried to directly take your proposed suggestions, it failed.

[bcm-at-zama]

As far as I can see, I have applied your proposed changes:

• 1: remains management of images: I need someone to merge https://huggingface.co/datasets/huggingface/documentation-images/tree/refs%2Fpr%2F316 please, and then maybe tell me how to use the merged images in the .md
• 2: https://github.com/huggingface/blog/pull/1885#discussion_r1534775464, nothing to do, right?
• 3: https://github.com/huggingface/blog/pull/1885#discussion_r1535328526, nothing to do, right?
• 4: test on azure, https://github.com/huggingface/blog/pull/1885#discussion_r1557090363, it will take me a bit of time

Removing the draft status, I think it can be re-reviewed.

[bcm-at-zama]

(I'll squash the commits, or if you want me to do it now, I can. )

[bcm-at-zama]

Hello. So, regarding the deployment on Azure or on Google cloud: I have tried to deploy https://huggingface.co/zama-fhe/concrete-ml-encrypted-deeplearning but both of them seem to fail, I'll contact [email protected] as it is suggested, CC @pcuenca

-- edit: mail sent

[bcm-at-zama]

To me, I've done all the changes. What do you think, @osanseviero @pcuenca? We also need to agree on a date for _blog.yml, to fix the conflict once for all

[osanseviero]

I think we can aim to release on Monday. WDYT?

[bcm-at-zama]

Fine with me @osanseviero! Just, I think someone from HF need to merge the image files (https://huggingface.co/datasets/huggingface/documentation-images/tree/refs%2Fpr%2F316) and I would need a bit of guidance on how to link to them in my .md. Thanks

[pcuenca]

Fine with me @osanseviero! Just, I think someone from HF need to merge the image files (https://huggingface.co/datasets/huggingface/documentation-images/tree/refs%2Fpr%2F316) and I would need a bit of guidance on how to link to them in my .md. Thanks

Happy to merge! However, they are still very small so they look blurred in my screen and you can't hardly read the text. Can you maybe upload larger versions, and ping me if you do? :)

[bcm-at-zama]

I don't have them in larger format, let me regenerate them!

[pcuenca]

To link the images in the dataset, you should use URLs like the ones shown in this example: https://github.com/huggingface/blog/blob/487226f09d83faaf220df38cc13dd41bbe5c5f44/vlms.md?plain=1#L18

(In this case they decided to use HTML, but the same URL structure works with markdown, of course)

[bcm-at-zama]

I wanted to redo the images, and then I realized the endpoints don't work on AWS anymore. I have asked your support, but that's a bit of a blocker here for the blog.

[bcm-at-zama]

Update: I'm discussing with HF support team, to see what happens with the breaking endpoints. We'll fix that next week, hopefully.

[bcm-at-zama]

OK I can make the EndPoints work again (it was about a python version default change, I think). Now, let me check all models and then have a look with other clouds.

[bcm-at-zama]

Fine with me @osanseviero! Just, I think someone from HF need to merge the image files (https://huggingface.co/datasets/huggingface/documentation-images/tree/refs%2Fpr%2F316) and I would need a bit of guidance on how to link to them in my .md. Thanks

Happy to merge! However, they are still very small so they look blurred in my screen and you can't hardly read the text. Can you maybe upload larger versions, and ping me if you do? :)

https://huggingface.co/datasets/huggingface/documentation-images/commit/0099707dc7484e4a445a3c203a46294afbe89057 contains now larger files. Good to merge 316? Then I can change links

[bcm-at-zama]

update:

• the blog post has been amended with images from the repo of images: need pr/316 to be merged and then someone to check my link are correct (could be me, once 316 is merged)
• the endpoints work fine on AWS: it was related to python default version to be moved to 3.10, which is not expected currently by our compiled models (we're working on it)
• I'll test with Azure and Google clouds tomorrow hopefully

[bcm-at-zama]

Maybe I should precise that users need to use python3.10: I do that tomorrow morning

[EDIT] I just did it

[bcm-at-zama]

• with Azure and Google clouds tomorrow hopefully

It has finally been done!

• google is in the [24-27] range, except exceptions
• azure is more in the [33-35] which is worth than AWS which I said averaged in 24.

Those results may change in the future, if ever better machines are added in Endpoints.

[bcm-at-zama]

Thanks a lot @osanseviero for the final touch and the merge of 316.

For image size, and in particular, the first and the fourth,

• I have no idea how it will be in the final HTML
• but they are maybe a bit too huge in the .md View

So, should I make them a bit smaller, by playing with style="width: 90%? Or, if you can see the final HTML, don't hesitate to propose me adequate widths :)

Anyway, thanks a lot once again for your help here, very appreciated

[bcm-at-zama]

Another conflict in _blog.yml. Maybe you tell me when we can merge it and I fix it one last time?

[pcuenca]

I'll be paying attention, feel free to fix and we can merge.

[bcm-at-zama]

OK, let me fix the conflict, but what do we do for the image sizes? See https://github.com/huggingface/blog/pull/1885#issuecomment-2058723253.

If you tell me nothing, good for me too!

[bcm-at-zama]

I fixed the conflict in _blog.yml

[pcuenca]

I tried to preview but couldn't, let's merge and fix later if we need to.

[pcuenca]

@bcm-at-zama can you open a PR to change the date to April 16?

[bcm-at-zama]

Absolutely!