jan
jan copied to clipboard
feat: Import public model via Huggingface URL
Specs: https://www.notion.so/jan-ai/Model-Hub-can-Import-from-HuggingFace-225dd4da88944bc183b816922c3333a8?pvs=4
User Flow: Import Model via Huggingface URL
- Copy Huggingface URL & Paste into Search Bar: Copy the link of the model repo from Huggingface, pasting it to search bar in Hub or My models page.
-
Repo Search: Jan will automatically search the repository linked to the provided Huggingface URL.
-
Scan for GGUF Files: The application scans for available GGUF files associated with the provided Huggingface model.
- Model Card & Files Preview: You'll see a nicely-formatted model card reflecting all available model files to decide which one you want to import to Jan.
- Import and Generate
model.json: Jan proceeds to download the identified files and generates the necessarymodel.jsonfor seamless integration.
6. Design
Figma link: https://www.figma.com/file/ytn1nRZ17FUmJHTlhmZB9f/Jan-App?type=design&node-id=3009-127260&mode=design&t=ZlK57CWtREqzmEZu-4
@louis-jan @namchuai This script: https://github.com/ggerganov/llama.cpp/blob/master/scripts/server-llm.sh#L223 contains the logic to parse .gguf models as list from a given huggingface URL (We can use huggingface model handler).
Hope it helps
btw @imtuyethan - Normally people paste the HuggingFace model handler, not the full URL
e.g: The link - https://huggingface.co/TheBloke/Mistral-7B-v0.1-GGUF -> People usually use TheBloke/Mistral-7B-v0.1-GGUF instead. Technically we can construct the URL to use underneath
@namchuai
-
Correct downloaded to Use button
-
Loader icon
-
Allow pressing Enter to reload the URL to view the popup
-
UI broke due to many tags
- Duplicate HF model id input :x: need clearer msg @namchuai, thank you