exo-explore
[BOUNTY - $100] DummyInferenceEngine
- We support several
InferenceEngineimplementations including MLX, tinygrad and PyTorch - Sometimes you want to test everything except the inference itself
- It would be helpful to have a
DummyInferenceEngineimplementation that can simulate some of the characteristics of inference without actually loading a model / running an inference - I'll leave it up to you to decide which parameters the
DummyInferenceEnginetakes but some things I think would be useful to customise are: the outputs (can be static or sampled from some distribution), the latency (can be implemented as an asyncio.sleep). InferenceEngineimplementations are required to be asynchronous so please no blocking code liketime.sleep
@ji-cryptocafe Any updates? Also, @AlexCheema is it still available If yes can you assign me this?
@ji-cryptocafe Any updates? Also, @AlexCheema is it still available If yes can you assign me this?
Assigned. I haven't seen a good implementation of this yet.
@AlexCheema submitted mine yesterday here https://github.com/exo-explore/exo/pull/331#issuecomment-2421805542 I was 3 days away due to holidays. @rahat2134 you are welcome to build upon it or make a new one. i have also seen someone else made another implementation, more complex and versatile.
@AlexCheema this issue has been resolved! How can I get the bounty? Thanks.
p.s - I have already mailed you.
@AlexCheema this issue has been resolved! How can I get the bounty? Thanks.
p.s - I have already mailed you.
@AlexCheema PTAL!
Replied over mail. Needed your ethereum address. The full bounty has been paid now.
@AlexCheema submitted mine yesterday here #331 (comment) I was 3 days away due to holidays. @rahat2134 you are welcome to build upon it or make a new one. i have also seen someone else made another implementation, more complex and versatile.
Can you please mail me to claim the bounty as you submitted a complete solution.