NEWbie0709

[Bug]: Image Handling with litellm Proxy for qwen-vl-plus

5

### What happened? I’m experiencing an issue when using litellm proxy to communicate with the qwen-vl-plus model for multimodal interactions. When I send an image URL directly to qwen-vl-plus, it...

[Bug]: Different Behavior with Image Input on GROQ/Llama 3.2 Vision Model vs Qwen

2

### What happened? I've encountered an issue while using LiteLLM with the GROQ/Llama 3.2 Vision model and Qwen. The problem arises specifically when providing an image input. --GROQ/Llama 3.2 Vision...

Hello, I would like to train a model specifically on MITRE ATT&CK knowledge and help it familiarize itself with MITRE ATT&CK concepts using the Zainabsa99/mitre_attack dataset. However, the documentation mainly...

Does AirLLM Support Running Quantized Models (e.g., unsloth/Qwen2-72B-bnb-4bit)?

1

Does AirLLM currently support running 4-bit quantized models like unsloth/Qwen2-72B-bnb-4bit? I’m trying to load and run this model using AirLLM, but I’m encountering the following error during generation: > RuntimeError:...

Very slow training speed with CURLoRA on Llama 3.1 8B Instruct

7

I am currently fine-tuning the Llama 3.1 8B Instruct model using CURLoRA adapters on a single RTX 4090 GPU.  Problem: - It takes ~170 seconds per step (batch) during...

Ollama uses CPU only after upgrading to CUDA 12.8

1

### What is the issue? After upgrading to CUDA 12.8 (from a previously working CUDA setup), Ollama now runs models using the CPU only, whereas it previously utilized the GPU...