jan icon indicating copy to clipboard operation
jan copied to clipboard

feat: Import public model via Huggingface URL

Open hantran-co opened this issue 1 year ago • 3 comments

Specs: https://www.notion.so/jan-ai/Model-Hub-can-Import-from-HuggingFace-225dd4da88944bc183b816922c3333a8?pvs=4


User Flow: Import Model via Huggingface URL

  1. Copy Huggingface URL & Paste into Search Bar: Copy the link of the model repo from Huggingface, pasting it to search bar in Hub or My models page.
Screenshot 2024-01-24 at 2 00 47 PM
  1. Repo Search: Jan will automatically search the repository linked to the provided Huggingface URL.

  2. Scan for GGUF Files: The application scans for available GGUF files associated with the provided Huggingface model.

Screenshot 2024-01-24 at 2 01 19 PM
  1. Model Card & Files Preview: You'll see a nicely-formatted model card reflecting all available model files to decide which one you want to import to Jan.
Screenshot 2024-01-24 at 2 02 11 PM
  1. Import and Generate model.json: Jan proceeds to download the identified files and generates the necessary model.json for seamless integration.

6. Design

Figma link: https://www.figma.com/file/ytn1nRZ17FUmJHTlhmZB9f/Jan-App?type=design&node-id=3009-127260&mode=design&t=ZlK57CWtREqzmEZu-4

hantran-co avatar Jan 24 '24 07:01 hantran-co

@louis-jan @namchuai This script: https://github.com/ggerganov/llama.cpp/blob/master/scripts/server-llm.sh#L223 contains the logic to parse .gguf models as list from a given huggingface URL (We can use huggingface model handler). Hope it helps

btw @imtuyethan - Normally people paste the HuggingFace model handler, not the full URL e.g: The link - https://huggingface.co/TheBloke/Mistral-7B-v0.1-GGUF -> People usually use TheBloke/Mistral-7B-v0.1-GGUF instead. Technically we can construct the URL to use underneath

hiro-v avatar Jan 30 '24 02:01 hiro-v

@namchuai

  1. Correct downloaded to Use button image

  2. Loader icon loadHF

  3. Allow pressing Enter to reload the URL to view the popup image

  4. UI broke due to many tags image

Van-QA avatar Apr 12 '24 10:04 Van-QA

  1. Duplicate HF model id input :x: need clearer msg @namchuai, thank you image

Van-QA avatar Apr 19 '24 04:04 Van-QA