Diwank Singh Tomer

What's the status of this, @RobertLucian ? We could really use this for our workloads. Would love to help as well

Thanks! I'll try to work on this over the weekend. In the meantime, sharing notes from my research: - You're right, a lot of the projects mentioned in the description...

It would be very interesting to see if this is indeed faster. We use fairly large images (average around 16GB) and the network overhead takes a heavy load on startup

Yup yup. Basically because we bundle ML models within the image itself so it’s not the runtime that’s causing the bloat (although that could use some optimisation too). We have...

By the way, what’s the best way to measure time spent by the service in different stages during startup? Just looking at the logs?

Tried the [ragawan/python-distroless images](https://hub.docker.com/layers/ragavan/python-distroless/3.8.8/images/sha256-34454ecb0dafc8ee7e19999c8df05d420cb11a8d4082851254225fbf31449f4d?context=explore) ?

Unable to use library with react 18.x without this patch. @KyleAMathews any particular reason for not merging this?

@zh217 this would be really useful. Happy to help out if this is something you'd like to have

huggingface/peft is the most popular choice

I'd be happy to update the docs etc. but is this the right direction for this?