mlx-llm icon indicating copy to clipboard operation
mlx-llm copied to clipboard

Published 20 hours ago • verified publisher icon riccardomusmeci
→

Metadata

Large Language Models (LLMs) applications and tools running on Apple Silicon in real-time with Apple MLX.

Results 1 mlx-llm issues
Sort by recently updated
recently updated
newest added

stupid question, any way to avoid running out of memory?

2 comment

If it put an input of 17,000 tokens into `model.generate(x, temperature)` I get `libc++abi: terminating due to uncaught exception of type std::runtime_error: Attempting to allocate 19081554496 bytes which is greater...

jmugan avatar jmugan

← Metadata

305
Stars
27
Forks
Watchers

Owner

verified publisher icon riccardomusmeci

Metadata

Large Language Models (LLMs) applications and tools running on Apple Silicon in real-time with Apple MLX.

Back