overlay :: Image -> Image -> Image

[sullyj3]

It feels a little awkward that one can't overlay an image onto another and get back an image. If I want to create some kind of widget that has multiple layers, but treat it as a single unit, I have to carry around a list of layers where each layer is an image, deferring merging them until the very very end, when I can merge everying into a picture at once and display it.

Would it be feasible to add overlay :: Image -> Image -> Image?

skimming the code, it seems like you could add a constructor to Image

data Image =
    ...
    | Layers [Image] 

Which would allow for this. If the idea makes sense and you think it's a good idea, I'd be happy to have a crack at implementing it.

[jtdaugherty]

Hi @sullyj3, thanks for opening this. I am not sure what it would take (or what obstacles you might encounter) to do this, but I would be interested in it as a feature. I think I even opened a ticket like this back before I was the maintainer, and IIRC the previous maintainer said it would be feasible.

With that said, how do you envision this working? I can think of two ways this could behave:

1. An image Layers [a, b] is as large as its largest layer, even if that layer is a translation, and the layers a and b can never interact with layers of an adjacent image Layers [c, d]. This would probably be much simpler to implement, but it would only be useful if you don't want two adjacent images' upper layers to end up getting superimposed on top of those adjacent images' lower layers.
2. An image Layers [a, b] next to an image Layers [c, d] could potentially result in layer a being superimposed on top of c and/or b on top of d if those upper layers are sufficiently translated. This approach is way more flexible, but I am not sure how difficult it would be to implement. I think it could be difficult for users to reason about, but this approach might lead to the least surprising behavior. I'm not sure. This approach probably essentially amounts to just keeping track of the layers and then translating them when a Picture is made, keeping track of offsets as the larger image is assembled.

Thoughts?

[sullyj3]

Yeah, I had been implicitly thinking of version 2 there. 1 could be a good fallback if 2 turns out to be too tricky.

[sullyj3]

Actually, thinking about it more, I don't think 2 quite makes sense. Under interpretation 1, the images that you are <-> or <|> -ing, are the smallest rectangle that encloses all of their layers (which I guess is equivalent to the largest one of them, as you say). The boundary between the two would be the adjacent edges of their respective enclosing rectangles. By contrast, how would you decide where the boundary ought to be in the case of interpretation 2? Would you just say that you're putting the bottom-most layers of each image next to each other? But lower layers might theoretically be buried beneath more constructors, arbitrarily far down. It's not even clear you would have a canonical lowest layer, as in the following case

-- suppose these images are all 3x3, and overlay puts its left argument on top in the z axis
(translateY 2 stars `overlay` hashes) <-> (plusses `overlay` translateY 2 dashes)

Under interpretation 1, that would look like this (if I'm understanding you correctly):

### <- hashes are underneath stars
###
***
***
***
+++
+++
+++
--- <- dashes are underneath plusses
---

But I'm not sure how you'd decide in a principled way how it ought to look under interpretation 2. Both dashes and hashes are at the lowest point in the z axis. If stars and plusses should overlap each other, how should they do so?

I also like interpretation 1, because it allows for the intuitive properties that

-- ~== means "renders to the same flat grid"
a <|> b ~== a `overlay` translateX (imageWidth a) b
a <-> b ~== a `overlay` translateY (imageHeight a) b

[sullyj3]

Fat fingered it, sorry

[jtdaugherty]

Thanks for your example - yes, I think approach 2 is probably not workable. But approach 1 is not equivalent to just carrying around the layers yourself and stitching them together at the end, because doing it manually gives you way more control in how they are laid out. I just wanted that to be clear going into this. But I still think approach 1 could be a useful addition and worth exploring if you want to work on a patch!

[sullyj3]

I agree they're not equivalent - carrying around images [x,z] allows you to insert another layer y in between them later on. Calling overlay wouldn't allow this - you'd be committing to treating overlay x z as a solid unit. Additionally you've committed to their translation relative to each other. I think of it as an abstraction boundary - you call overlay if you know you don't want to insert anything in between later on. But you can still put things on top of or underneath the overlay.

It'd be similar to how eg. calling <|> means you don't intend to later insert anything between the arguments horizontally. You give up the later flexibility once you're sure about the layout.

I'll begin poking around!

[sullyj3]

For posterity, I ended up getting a bit intimidated by the idea of trying to edit this module:

https://github.com/jtdaugherty/vty/blob/c9b63ffad6b7ec0ea8509e17a5d10706034c0691/src/Graphics/Vty/PictureToSpans.hs#L67-L92

if anyone is interested in implementing this, that's where most of the implementation complexity will be, I think.

[jtdaugherty]

@sullyj3 I know it's been a while, but I am going to close this due to inactivity (and also because I don't have time to tackle it myself). If you get interested in taking a look at this again, let's re-open it.