cog
cog copied to clipboard
replicate
Local web interface
It'd be neat if Cog served a Replicate-style web interface when you ran the web server for testing locally.
User data
- As requested by @postera-james
This would be fantastic. The current testing situation is frustrating.
It's a really slow feedback loop to test with cog predict if your model's setup function is heavy. And if you're working with images, it's ginormous base64 outputs also muddy up the console.
Running the docker container and hitting the prediction endpoint is also not ideal for working with images. Those big base64 strings clog up the SwaggerUI interface (or Hopscotch or whatever) and don't mean much on their own. And you can't get at the images they represent (if they're saved to the filesystem) without mucking about more with the docker container.
And even if everything seems to be working fine locally, that's no guarantee that it will work the way you think it will on Replicate. It's an agonizingly slow feedback loop to cog push and wait for a potentially huge docker image, with a bunch of model checkpoints, to push to replicate. And then you have to wait for replicate to spin up infrastructure for the model.
If we had this, it could halve the time it takes to port a model to replicate.
Super helpful data -- thanks @nicholascelestin. I wonder if there was a way to keep a Cog model running in the background and running cog predict against that would be a small first step to avoiding running setup() each time. 🤔
Theoretically, you could have an interactive mode for cog predict such that after each prediction it immediately prompts for more input. Maybe you could start with a while loop here ? I think that would be a nice feature by itself.
But it doesn't target as many frustrations as a local replicate UI. Assuming the local UI acts like the real one, it would save you from hitting the setup function and from needing to push the model to replicate to validate things like how the output renders.
Oh yeah an interactive mode is a great idea. #666 😈
Agreed a web UI would be much better, but it's big and complicated. Trying to figure out if we can get something done quick that would make your life easier.
Feel free to submit a PR for any of these things if you feel the urge! 😄