exo icon indicating copy to clipboard operation
exo copied to clipboard

[BOUNTY - $100] Vision Model Integration Test

Open AlexCheema opened this issue 1 year ago • 8 comments

  • As expected since we don't have tests for vision models, they broke unexpectedly: https://github.com/exo-explore/exo/issues/327
  • Create a test similar to the existing one (chatgpt_api_integration_test_mlx) we have for text in .circleci/config.yml

AlexCheema avatar Oct 10 '24 21:10 AlexCheema

I can do this

reddyn12 avatar Oct 10 '24 21:10 reddyn12

I can do this

go go go!!

AlexCheema avatar Oct 10 '24 21:10 AlexCheema

@AlexCheema this is already done in my pixtral PR (https://github.com/exo-explore/exo/pull/218/files#diff-3254d7af7026ebfe4e98753ce878db27e9d55796bec2744632a5633f4ad6a841), ill just move them to run on full pipeline.

I'm half way through with llama visual model as well, will raise a combined PR by EOW

varshith15 avatar Oct 10 '24 21:10 varshith15

Looks like @varshith15 got it. Ill go back to tinygrad llava

reddyn12 avatar Oct 10 '24 21:10 reddyn12

@AlexCheema this is already done in my pixtral PR (https://github.com/exo-explore/exo/pull/218/files#diff-3254d7af7026ebfe4e98753ce878db27e9d55796bec2744632a5633f4ad6a841), ill just move them to run on full pipeline.

I'm half way through with llama visual model as well, will raise a combined PR by EOW

Wow, awesome! Okay no need then! This bounty is yours, sir!

AlexCheema avatar Oct 10 '24 21:10 AlexCheema

@AlexCheema this is already done in my pixtral PR (https://github.com/exo-explore/exo/pull/218/files#diff-3254d7af7026ebfe4e98753ce878db27e9d55796bec2744632a5633f4ad6a841), ill just move them to run on full pipeline.

I'm half way through with llama visual model as well, will raise a combined PR by EOW

I was also hoping for a circleci integration test here.

AlexCheema avatar Oct 10 '24 21:10 AlexCheema

oh ok, that would be a very small change/addition right? - will do that

varshith15 avatar Oct 10 '24 21:10 varshith15

Since the bounty is paid out and the PR is discontinued, should the issue be closed until further notice? Unless there is a new bounty out for this issue I suppose the bounty in the title should be removed at least?

marksverdhei avatar Mar 20 '25 14:03 marksverdhei