lu

启动两个实例，其中一个会自动挂掉

2

不知道啥原因

I have several Android device nodes that I use for model inference. When the nodes start loading the model, the first node loads the model successfully. However, the second node...

Mali GPU OpenCL does not support bfloat16

2

I'm encountering an issue with my Mali GPU. When I try to inference, I get the following error: ``` {'319cbd94-148d-4767-80af-950aa5c20-11'}}). Next partition: Partition(node_id='319cbd94-148d-4767-80af-950aa5c20-23', start=0, end=0.08333) Sending tensor_or_prompt to 319cbd94-148d-4767-80af-950aa5c20-23: user...

Can a large model be successfully loaded across multiple EXO nodes with insufficient individual memory?

6

I have three nodes, each with 3GB of memory. When I try to load the 8B LLAMA model locally on each node, I encounter an OOM (Out of Memory) error...

local model loading without request to huggingface

3

I have a large number of Mali GPU devices. I have already copied the 8B model to each of my nodes, but the latest version of EXO utilizes the download/hf...

Can I set the maximum exo memory usage for each node? Prevent oom

1

I have multiple mali gpu devices, no matter how many nodes, and have never successfully loaded a model. Always the first node oom ~

How install on android

13

my android device only has python3.11.9. i try every solutinos,cannot install python3.12 on android.

I have a system with 9 identical nodes. However, the model randomly load on one node and Out-of-Memory (OOM) on the node during loading. My device uses a Mali GPU...