How To Add Suppoerted Model

[hyperbolic-c]

@b4rtaz Hey, thank you for your wonderful work. Could you please offer some details about how to add supported model? For example, how to split the network according to structure of model. It is difficult to work without your help! THANKS!

[b4rtaz]

Hello @hyperbolic-c, sorry I don't understand your question.

how to split the network according to structure of model

You don't need to adjust the network topology to the model. Just join together 2^n computers via Ethernet (you may need a switch) and that's it. Now you need to pass only the ip addresses of worker nodes to start the root node.

./main inference --model ../dllama_llama-2-7b_q40.bin ... --workers 10.0.0.2:9998

[hyperbolic-c]

@b4rtaz Sorry for the lack of clarity. Actually, I want to say how to convert the model network layer to distributed-llama format, converting open source models other than llama2 or llama3. Thanks for your reply!