[Feature Request] Masking

[MarkVabulas]

Preface: So this is gonna be a fun but powerful request, which has very wide-reaching implications. I imagine it to be extremely simple to implement.

Synopsis

I would like the ability to specify an image, and multiply that image pixel-by-pixel (as floating point [0.0,1.0]) with the already-rasterized lower layers.

One important use case, which is the impetus for this request, is to be able to apply a border around a card (when it's a SubLayout for example), and have everything in that non-uniform border turn transparent. Then, with a second layout, I could print that same SubLayout as it's own card, and it would have the necessary bleed for printing because the border mask is disabled outside of SubLayouts. By doing this, I could have cards printed as an array of cards with pretty borders (all together), and also printable individually with a bleed area.

Implementation Details

My idea is that a Graphics Element would have a 4th render type, which would be "Mask" or some such. That means that it would multiply the values in the "Mask" layer with what was already rasterized in the image (the rendering order could stay the same). If it was a greyscale image with transparent areas, the each pixel would be multiplied by the one brought it. White would keep whatever color was in the buffer, black would turn colors black, partially transparent mask pixels would turn the buffer partially transparent, etc. Adding a rainbow image mask, with a transparent background, over a white star-shape would effectively let someone create a star with a rainbow pattern inside it and a transparent background. The idea is to create a normal Graphics Bitmap as you typically would, but it would require a manual rasterization pixel-by-pixel in order to do the combination. But since you can make the Graphics Bitmap temporary mask the same dimensions and 32 bits per pixel using the current methods, it should be straightforward to grab the code like the ones referenced below to implement it. If you want me to do a rough pass first, I could, but I know you like to implement things in your own way; I fully support your control and decision making.

Examples of implementations which also show roughly what I'm looking for

(These use red as a mask, but I'm looking for an RGBA multiplication, not just masking) alpha masking image mask

Easy Examples

One easy example of the impetus and payoff is a game like Through the Ages Example Photo: there's a player mat which has cards burned into it with evident/clear borders, then there are other cards in the deck with an identical look, used to cover/augment the player mat, which obviously must have had print bleed. Using Sublayouts with a configurable transparency-based border mask means we could use 1 SubLayout to generate all of the cards consistently, and then use some of the cards as embeds on a player mat.

[nhmkdev]

Certainly sounds interesting! I'd probably go with a new Element type as Graphic has a lot going on that a mask wouldn't really need to care about (no color management/scaling/alignment). The mask would be used as-is with its original size.

I think the work would break down to a couple of steps:

1. Implement a new element type (doesn't do much, maybe just render the mask at first)
2. Adjust the CardRenderer to work with a GraphicsContext object that contains the Graphics, Bitmap, and any other critical information
   1. Performing any operations with the LockBits needs the Bitmap to allow access to the existing pixels that were rendered before.
3. The actual masking logic/math. (given 2 Bitmaps, make the magic happen)

I'll probably get this started soon depending on my time. Step 3 is the one I know the least about. I'll be investigating the links you sent to learn more. Any tips/suggestions/code related to doing this with black/white instead of a color (like the stackoverflow example) would be very helpful. 👍

Concerns

• Will this have any performance implications? It would definitely be too bad if it dragged down the app.

[MarkVabulas]

Graphics Element vs New

Personally, I think the Graphics element is actually the perfect one. I think that having all of the same functionality as the Graphics one, including positioning, scaling, color, etc would be great. Maybe a specialization of the Graphics type. But It could theoretically be really cool to have a mask which is modified by a color matrix coming from a spreadsheet which could selectively make different parts of the card transparent based on the pixel values, instead of having to have multiple elements and toggle them on/off. Thinks like mirroring/aspect ratio/centering/tiling etc could be useful to have, if you think about it.

There's also the added benefit that rasterizing a Graphics Element is already functional, the only part that would need to tweak is when it's actually applied to the canvas/render.

Considerations

I think there would be a performance implication if it was overused, so that's a caveat, however it should be minor since GDI+ uses the same types of algorithms for how it works internally anyway.

[MarkVabulas]

Overview

For the initial mask setup, I would run the typical Graphics loading routines, including applying positioning, colors, mirrors, rotations, opacity, etc; put this result into a 32bpp System.Graphics Graphics (like in TypeElementRenderProcessor). Then you use that Graphics result to do a specialized write into the Canvas/Export Bitmap, except instead of using System.Graphics.Bitmap.DrawImage, you use the masking function when the Graphics Type is set as Mask.

Combining the images

The math for combining two bitmaps, pixel by pixel, should be relatively straightforward. My suggestion is merely to do a floating-point multiplication of each tuple. So:

1. In your base image, get the RGBA value -> convert it to 4 floats in the range [0,1]
2. In your mask image, get the RGBA value for the same position -> convert it to 4 floats as well
3. Multiply the 4 floats with each other, red with red, green with green, blue with blue, alpha with alpha.
4. Store the resulting multiplication back into the base image

This has the effect of doing a sort of union of the mask's 4 channels with the base's 4 channels. A white circle with a transparent background would keep everything in the circle (since it's 1.0, 1.0, 1.0, 1.0) and make transparent everything outside the circle (since the alpha is multiplied by 0.0)

If you had an underlying pixel which was Blue [0.0, 0.0, 1.0, 1.0] and multiplied it by a Red half-transparent mask pixel [1.0, 0.0, 0.0, 0.5] , you'd get a black half-transparent pixel [0.0, 0.0, 0.0, 0.5]. This is why the above rainbow mask over a white star would work, at each pixel the white would be reduced/transformed to the color of the rainbow, and everything that is transparent in the base OR the mask would stay transparent [since either of them has an alpha of 0.0].

[nhmkdev]

Thanks for the details/suggestions with using multiplication. That worked easily (once I figured out marshaling to/from a bitmap).

The rest of the development is going to be working through the math of supporting element positioning and scaling (for the sake of the zoom level the user can have set in the Canvas). It's nothing too bad based on what I've looked at (just a number of little things to consider to make it all work).

[MarkVabulas]

I imagine you can co-opt the original placement code for the positioning/scaling. Basically, make a raw transparent canvas to draw on like you normally would for a typical Graphics element, as a temporary, and then use that for your pixel-for-pixel comparisons. That way, you don't have to calculate positions yourself, you just do it for the whole image. A white [1.0, 1.0, 1.0, 0.0] (or black really) transparent base image would be ideal for the temporary canvas. I'm suggesting using GDI+ for making a completely normal Graphics element in memory (except on a raw canvas [of the same final dimensions as the target] instead of direct writing to the output), and then combining that result with the canvas during the LockBits.

[nhmkdev]

That was one of my first thoughts too: just a matching full card image size with the scaled mask rendered on it 👍

I'm trying/considering a number of routes for the fun/challenge of it.

[nhmkdev]

Masking work-in-progress. https://github.com/nhmkdev/cardmaker/commit/56224466b319bb0c316141c68519549fdc62eb19 (comically large change)

It is not enabled by default (requires recompiling with a temp hack enabled)

• Then any filepath with the string mask will run through the code path to render the mask.