Alex Cheema comments

Results 413 comments of


                                            Alex Cheema

Support Mixture of Expert (MoE) Models

> will like to work on this :) @345ishaan that would be great - go for it

Support Mixture of Expert (MoE) Models

> @AlexCheema > > Is it possible to have the active parameters favor an Nvidia Cuda GPU and let the other nodes store the inactive parameters? Yes! We can definitely...

No models downloading

Probably duplicate of https://github.com/exo-explore/exo/issues/107 Fixed with https://github.com/exo-explore/exo/commit/76766253cde775f58f5637344dae2656f1c1447a

No models downloading

Reopening. Can you give some more info - what commit are you running on? What device? Do you get any logs? Can you try running with DEBUG=2 e.g. DEBUG=2 python3...

A module that was compiled using NumPy 1.x cannot be run in NumPy 2.0.0

It might be something about `asdf`. I'm using `source install.sh` to install from fresh (which uses venv) and works fine. Here are my installed packages: ``` (.venv) alex@Alexs-MBP exo %...

Allow LLM copy from the local network instead of downloading from Hugging Face

This would be great. I think the thing that is difficult is doing this in a way that's compatible with one of exo's core design philosophies: node equality. I don't...

Allow LLM copy from the local network instead of downloading from Hugging Face

Related: #80 #70 #16

Removing download task for Shard(model_id='mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated', start_layer=0, end_layer=31, n_layers=32):

This will be fixed with #139 with the torch inference engine

local model loading without request to huggingface

Renamed the issue. Let me know if this is accurate: we need an option to entirely disable the request to huggingface and only load from local disk.

Can a large model be successfully loaded across multiple EXO nodes with insufficient individual memory?

What device?