huggingface
Latent Diffusion Upscaler
I'd love to see the latent upscale model ported: https://github.com/Stability-AI/stablediffusion#image-upscaling-with-stable-diffusion
The Unet architecture in x4-upscaling.yaml looks very similar to the Stable Diffusion model (v2-inference.yaml) so should be pretty straightforward but getting all the way to inferencing in Swift is a bit beyond my skill set. I'm more than willing to help and learn though.
@3DTOPO That'd be very cool indeed! Let's see if someone in the community takes up the challenge to convert the model to Core ML!
I could probably hack a conversion of the model to CoreML but what would take me a few days I bet someone could do in a few hours and I just don't want to work on something that might already be worked on.
I requested the model to be ported at the repo.
Hi there! Let me wish you a Happy new year first! I could be interested in working on that but my knowledge about Core ML and models is very limited. But, I would be happy to learn. On the other side, I have strong experience in Swift and I'm curious about Models and CoreML topics. Do you guys think it could be done without xp on that? Or it would be a waste of time?
The upscaling model architecture is very similar to the diffusion U-net model that the coreml conversion scripts currently convert so there is a good example to learn from.
If you feel like taking a stab, I'm happy to help if I can - just ask. I do have a lot of experience converting PyTorch models to MLModels.