blog
blog copied to clipboard
huggingface
Blog post on `bitsandbytes` integration on Hugging Face
A blog post explaining the method 8-bit Matrix Multiplication for transformers at scale and how this is used under the hood in transformers !
- Still a WIP PR
TODOs
- [x] change the main thumbnail
- [x] add more content
- [x] correct content
- [x] re-capture the gif
out-quant.gifbecause we can see the mouse
You might want to share in the OP the preview link that will remain fixed through this PR's mods. https://github.com/younesbelkada/blog/blob/add_bnb_inference/hf-bitsandbytes-integration.md it's much easier to read that way.
the broken images in the html header part is normal - will appear normally once published.
Great thanks for the tip! I have changed the previous names of some pictures and now all images should be rendered correctly
Thank you very much for your review! This will be extremely helpful Yes we can go for this approach, I can give you access to the google doc I used to write the text if you prefer
no google doc needed, we are already editing here, it'd be extremely difficult to keep 2 separate docs in sync. So let's switch to editing here 100%.
As I shared earlier the preview link should serve as a nice visual that's easier to read than raw MD.
I've pushed the rewrite of the first few sections. Please let me know if it feels good and then I will continue reworking the rest when you're asleep next time, so that we don't collide on our edits.
I wonder if perhaps you'd want to first finish the rough version first and I will do the heavy editing afterwards? It's your call.
Thank you very much @stas00 ! I have updated more content and tried to rephrase as much as I could! I would love to have a review pass until How to use it in transformers section 💪
I have addressed your points and comment on them! Let me know if they make more sense now 💪
Thank you very much @stevhliu for the complete review! I validated them and addressed the points that needed more clarification :pray:
I left some comments for myself to integrate @stas00 comment regarding the animations that I will process later ;)
Just finished my round of editing @stas00 ! Please let me know if the proposed modifications make sense to you 🙏 Regarding https://github.com/huggingface/blog/pull/463#discussion_r946058286 I will probably let Tim answer on that one! Again thank you for your feedback so far 🔥
Looks good to me now! cc @stevhliu I think that we are ready to merge now 💪