jan
jan copied to clipboard
janhq
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)
## Kanban https://github.com/orgs/janhq/projects/5/views/7?filterQuery=milestone%3A%22Jan+has+System+Monitor%22 ## System Monitor _Questions_ - [ ] Are we able to do a Prometheus/Grafana-style time series dashboard? (How do we even persist the data on flat files?)...
- [x] Scaffolding for GPU monitoring - [x] Mac Metal monitoring backend - [ ] Mac Metal monitoring frontend - [ ] Mon plugins to ask for `sudo` permission on...
**Problem** I want to see my MacOS metal gpu monitoring (GPU utilization/ VRAM) on Jan to check the resource consumption of models use **Success Criteria** - Correctly show the utilization...
**Problem** I want to see my nvidia gpu monitoring (GPU utilization/ VRAM) on Jan to check the resource consumption of models use **Success Criteria** - Correctly show the utilization -...
 Our current Jan results in "New Thread" for each thread, as users don't title each thread manually. We should find a way to implement a ChatGPT-style Thread Title summarizer,...
**Describe the bug** LLM responses seem to terminate prematurely, do we have a max reply token setting somewhere? EDIT: Mistral likely limited to 512? **To Reproduce** Steps to reproduce the...
@Van-QA quoted this from feature request janhq/jan#2723: > **problem** > You can only use the GGUF model, not a wide range of models. So, if you can use the Transformer...
https://www.digitaltrends.com/computing/microsoft-nvidia-tensorrt-llm-update-ignite-2023/ Tasks - [ ] Step by step docs for Jan Windows TensorRT-LLM - 1 day - [ ] Updated code in `triton-tensorrt-llm` extension - 1 day Reference https://github.com/NVIDIA/trt-llm-rag-windows/blob/release/1.0/app.py#L43
WIP Spec - Need to figure out if BigDL or Intel Extensions are separate - Have Extensions for each inference engine - `model.json` should have an `engine: intel-bigdl` or `engine:...