image-analogies icon indicating copy to clipboard operation
image-analogies copied to clipboard

Allow for multiple source images

Open awentzonline opened this issue 9 years ago • 1 comments

This is useful for super-resolution and for allowing multiple scales/rotations of the input.

awentzonline avatar Mar 04 '16 21:03 awentzonline

I realize this issue is just a short note to yourself, but I'm curious about what you have in mind more exactly. I have a somewhat unusual use case, and I've just started working on a version of your code that works with multiple images. I'm super-interested in your input. If we figure out some principled, general way of solving both my use case and other, more common issues as well, I'll happily provide a pull request.

Here's my use case: I'd like to generate a neural doodle with a ridiculously high resolution. Fortunately, I can restrict the camera position to a few zoom positions, so in practice I'll have a big lower-resolution overview image, and some lower-resolution zoom-in images.

Here's what I plan to implement: There are several A, A', B triplets as input, and the program works on these triplets one at a time, like before. But we also provide geometric information about how the B' output images are supposed to be placed on a plane. Whenever there's an overlap between two B' image rectangles, that's a constraint on both of them. I simply plan to enforce this constraint unidirectionally, from smaller to larger rectangles, so that's just an extra synchronization loss term that I have to calculate between generating two images.

Do you think this plan could work? Do you see some better way of doing this, that might benefit other image-analogies users as well?

danielvarga avatar Apr 11 '16 13:04 danielvarga