libvips-rust-bindings
libvips-rust-bindings copied to clipboard
olxgroup-oss
VipsImage thread safety
Hi! Thank you very much for providing this very pleasant to use binding crate.
It would appear this warning is directed at me and my use case:
Be aware that the VipsImage object is not thread safe in the moment. I'll investigate what is happening and provide a solution for it in the future.
Is there an update to your thoughts on this, or your general advice on working around it?
@augustocdias I am working with a type of proxy service - I basically fan out a bunch of HTTP requests in parallel process them with vips to resize and crop as they come in. Then once they are all done, I join them in to one contiguous image with arrayjoin. As such, I need to hold on to basically Vec<VipsImage> as they come in, which requires Send.
The only "hack" I could think of was serializing them to [u8] in the worker tasks so I could move it around, and recreate a VipsImage when they are all finally ready. Obviously not very efficient..
I did try unsafe impl Send for VipsImage {} just for kicks, and saw that it indeed breaks things...
Is there maybe another approach that will let me do this in parallel? I am totally new to using vips, so I appreciate the adivce :) Let me know if there's other details I can provide.
The problem is I'm not sure if the problem is with rust or libvips. I had a similar setup as you. I didn't have any segfaults or anything, but the vips operations started to fail complaining the internal pointer from vips was invalid.
I didn't have time to investigate and I couldn't find any doc stating how am I supposed to handle ffi pointers in these situations.
If you find anything please share because I'm intrigued to know the reason for that.
Gotcha. IIRC the segfault happened with a particularly large image. Others, yes, would say something along those lines of the pointer being invalid, something from vips own logger.
The problem is I'm not sure if the problem is with rust or libvips.
Right. I get the impression from https://libvips.github.io/libvips/API/current/using-threads.html - which claims:
On startup, you need to call VIPS_INIT() single-threaded. After that, you can freely create images in any thread and read them in any other thread.
as I'm sure you've already read. So I'm inclined to take their word on it, and its something funny we're doing in Rust.. I have a few things to try today, among them possibly trying to reproduce an equivalent small program in C, or maybe seeing what I can find with a debugger.
If I find anything I will report back - thanks for your time!