Robert Ritz

I'm also on a 2020 Macbook Pro M1 and have not had this issue. I'm using Google Drive for Desktop (formerly Google Drive Filestream) and this mounts my Google Drive...

I'm having the same issue when trying to save or export. Environment: - Windows 10 64 bit - Jupyter core 4.4.0 - Jupyterlab 0.34.9 - Browser: Chrome 69.0.3497.92

I tried to replicate the solution later but couldn't. It appears the real issue is that I was missing libjpeg. I installed libjpeg-dev and it worked. Can anyone verify this...

Can you explain how to save the covers query to the expected cached filename? I would like to use MapIt Global but can't figure out how to add these area...

Having tried several, I think the reason no examples are shown is that their outputs are suboptimal. With lots of prompt engineering, you can get usable results but nothing as...

Yes! If quantized models are supported I will be moving away from llama.cpp and over to MLX. Prompt evaluation is still tediously slow on M series with llama.cpp and it...

I don't think it will improve overall tokens per second. But time to first token is what the issue is right now. For large prompts it can take almost 10...

To be clear I'm referring to llama.cpp (I'm using the Python bindings). For long prompts, it takes several seconds for me to get a response. Are you referring to llama.cpp?

Yep that's why I'm so interested in mlx. Slow evaluation is the main shortcoming of local LLMs on Macs right now.

@tanliboy I haven't done any testing that was recorded. But here is my anecdotal experience. When a model is first loaded with llama.cpp (not kept in RAM but lazy loaded)...