Add support for resource textures in D2D1 pixel shaders

[rickbrew]

You create a resource texture with ID2D1EffectContext::CreateResourceTexture() https://docs.microsoft.com/en-us/windows/win32/api/d2d1effectauthor/nf-d2d1effectauthor-id2d1effectcontext-createresourcetexture . You specify an ID as a GUID, and provide a buffer to load the data from.

You can then update the resource texture with ID2D1ResourceTexture::Update() https://docs.microsoft.com/en-us/windows/win32/api/d2d1effectauthor/nf-d2d1effectauthor-id2d1resourcetexture-update

Then you bind the resource texture to a shader with ID2D1DrawInfo::SetResourceTexture(UINT index, ID2D1ResourceTexture*) https://docs.microsoft.com/en-us/windows/win32/api/d2d1effectauthor/nf-d2d1effectauthor-id2d1drawinfo-setresourcetexture

In the shader, in HLSL, you then define a texture/sampler like so:

Texture2D GradientTexture : register(t1);
SamplerState GradientSampler : register(s1);

This is assuming you passed 1 for index when calling SetResourceTexture().

Then you read from it like so:

float4 output = GradientTexture.Sample(GradientSampler, float2(upper_t, 0.5));

I think this wouldn't be too hard to add to CS.D2D1 with something like:

partial struct MyGradientShader : ID2D1PixelShader
{
    private readonly D2D1ResourceTexture myGradientTexture = new D2D1ResourceTexture(1);

    // or maybe ...
    [D2D1ResourceTexture(1)]
    private readonly D2D1ResourceTexture myGradientTexture;
}

The [AutoConstructor] would obviously need to understand the existence of special fields like this.

You could also do something like ...

[D2DResourceTexture(1)]
partial struct MyGradientShader : ID2D1PixelShader
{
    public float4 Execute()
    {
        float4 gradientSample = D2D.SampleResourceTexture(1, x, y);
    }
}

You could provide properties on your D2D1PixelShaderEffect to support binding these. I'm not sure if there's a limit, but I suspect resource textures are part of the 8-input limit on pixel shaders.

Here's some code I found that actually uses this:

• HLSL: https://github.com/Distrotech/thunderbird/blob/38c072d9b61471857f487fe8fca5a5b79afb993b/mozilla/gfx/2d/ShadersD2D1.hlsl
• C++: https://github.com/Distrotech/thunderbird/blob/38c072d9b61471857f487fe8fca5a5b79afb993b/mozilla/gfx/2d/RadialGradientEffectD2D1.cpp

[rickbrew]

This came up because I was looking at how to use the Dents effect in Paint.NET in order to remix how the About dialog's logo banner is drawn. Right now it "fades in" from very blurry to sharp by using gaussian blur:

But, Dents currently uses a 512x1 input bitmap for the Perlin noise permute table. This means that it's dependent on its creator using ID2D1DeviceContext::CreateBitmap() and plugging that in as a shader input. To package this up as an actual ID2D1EffectImpl, it'd need to use ID2D1EffectContext::CreateResourceTexture(), which is currently a dead-end since there's no support in CS.D2D1 for resource textures.

I'm not blocked from experimenting with Dents in this case, as I can just wire it up manually, but ... would be great to add support for this!

[rickbrew]

Okay I did some prototyping and I was able to get resource textures to work! I resorted to compiling HLSL at runtime with D2D1ShaderCompiler from a private fork where I just made it public and had it return a byte[] array.

In the Initialize method for the ID2DEffectImpl, I created a 1-dimensional ID2D1ResourceTexture, width=512, using ID2D1EffectContext::CreateResourceTexture(). However, you could do this in the transform as well -- you just need an ID2D1EffectContext.

Then, I set the resource texture in SetRenderInfo() with ID2D1DrawInfo::SetResourceTexture(1, pResourceTexture). Since resource textures use texture IDs, their index values must start at inputCount. So I have 1 input, therefore the input is index 0 and the resource texture is at index 1.

In the HLSL, I declared:

Texture1DArray PermuteLookupTexture : register(t1);

I didn't need a SampleState since I used Texture1DArray. You can also use Texture1D and a SamplerState, just like D2D does for inputs, but then you have to sample the texture using UV coordinates, and you have to normalize the offset with index / length, which means you have to somehow supply the length to the shader. Simpler to just use Texture1DArray, although I could see folks wanting it either way depending on their use case.

(Also, you can use Texture2DArray or Texture2D + SamplerState for 2D resource textures, and you can also have 3D resource textures.)

I can then read from the resource texture with:

int GetPermuteLookupValue(int index)
{
    // the first value of the int2 is the index, the second is the "array slice", whatever that means
    return (int)PermuteLookupTexture[int2(index, 0)].r;
}

It looks like the [] operator is returning a float4, and I don't know if it's possible to declare the Texture1DArray as float, uint, uint4, etc. Direct2D does permit 1-channel resource textures of whatever buffer precision you want (uint8, uint16, float16, float32), but I'm not familiar enough with HLSL or D3D to know if things always get normalized to float32s or if there's flexibility there.

[rickbrew]

Texture slots do not seem to need to be contiguous. By that I mean I can specify 2 inputs, which take up slots 0 and 1, and then I can SetResourceTexture(7, ...) and it's fine as long as the index in the HLSL matches.

[rickbrew]

I was also able to do SetResourceTexture(8, ...), but I'm not currently sure if that just means the count of textures has a max of 8, or if the limitation of 8 is specifically for inputs.

[rickbrew]

I tried SetResourceTexture(99, ...) and it resulted in an SEHException. So clearly there's a limit.

[rickbrew]

SetResourceTexture(15, ...) works, while SetResourceTexture(16, ...) does not (SEHException). So the limit is 16. Also, good job on the parameter validation, Direct2D.

[rickbrew]

So here's an idea for how this could be set up in CS.D2D1. It provides a nice API for users, and avoids adding a special field type that would be complicated to work around in other areas of the source generator. The length is not specified in the shader, as it is supplied at runtime.

[D2DInputCount(2)]
...
// Texture ID is 2 because the inputs take up slots 0 and 1. Would need compile-time validation of this.
// The index does not need to be sequential/contiguous with inputs, but has a max value of 15. Not sure yet
// if the count is limited to 8 across inputs and resources.
// There is one ExtendMode for each dimension, which determines how out-of-bounds sampling is treated
// The attribute values are funneled to the parts of the code that create and update the resource texture.
// D2DResourceTexture1DArray would declare the resource texture as a `Texture1DArray` in the HLSL
// D2DResourceTexture1D would have `Texture1D` and `SamplerState`
[D2DResourceTexture1DArray(2, Filter.MinMagMipPoint, BufferPrecision.Float32, ChannelDepth.One, ExtendMode.Clamp)]
...
readonly struct MyShader : ID2D1PixelShader
{
    public float4 Execute()
    {
        ... Would need many overloads to handle the various cases. Here's 3 different syntaxes, the first one is probably easiest
        ... You'd still want an overload that took an `int2` so as to directly match what is possible in HLSL with Texture1DArray
        float4 value = D2D.ReadResourceTexture1DArray(2, x);
        float4 value = D2D.ReadResourceTexture1DArray(2)[x];
        float4 value = D2D.ReadResourceTexture1DArray[2][x];
        ...
    }
}

[rickbrew]

Some docs links:

ID2D1EffectContext::CreateResourceTexture() https://docs.microsoft.com/en-us/windows/win32/api/d2d1effectauthor/nf-d2d1effectauthor-id2d1effectcontext-createresourcetexture

ID2D1DrawInfo::SetResourceTexture() https://docs.microsoft.com/en-us/windows/win32/api/d2d1effectauthor/nf-d2d1effectauthor-id2d1drawinfo-setresourcetexture

D2D1_RESOURCE_TEXTURE_PROPERTIES https://docs.microsoft.com/en-us/windows/win32/api/d2d1effectauthor/ns-d2d1effectauthor-d2d1_resource_texture_properties

ID2D1ResourceTexture::Update() https://docs.microsoft.com/en-us/windows/win32/api/d2d1effectauthor/nf-d2d1effectauthor-id2d1resourcetexture-update

[rickbrew]

A rough proposal for how this could integrate with D2D1PixelShaderEffect. The filter and extend mode(s) are specified on an attribute, while the other information is specified via effect properties. Direct2D (or D3D?) normalizes the data to be float4s when the shader executes, so it's more important that the code setting the buffer is responsible for specifying the precision (and thus the buffer size, etc.), as opposed to the shader specifying it.

// The shader
[D2DResourceTexture1DArray(2, Filter.MinMagMipPoint, ExtendMode.Clamp)]
...
readonly struct MyShader : ID2D1PixelShader
{
    ...
}

public static class D2D1PixelShaderEffectProperty
{
    public const int ConstantsBuffer = 0;

    // <summary>
    /// Sizing information for the resource texture occupying texture slot #0.
    /// This must be of type ResourceTexture1DSize, ResourceTexture2DSize, or ResourceTexture3DSize.
    /// </summary>
    public const int ResourceTexture0Size = 1;

    /// <summary>The data to be loaded into the resource texture occupying texture slot #0.</summary>
    public const int ResourceTexture0Data = 2;

    public const int ResourceTexture1Size = 3;
    public const int ResourceTexture1Data = 4;

    public const int ResourceTexture2Size = 5;
    public const int ResourceTexture2Data = 6;

    // and so on
}

// The public-facing structs are binary compatible with this so you can Unsafe.As<ResourceTexture1DSize, 
// ResourceTextureNDSize>() once you see that dimensions=1
// Otherwise there's not really a good way to have variable length span/array for projecting 
// D2D1_RESOURCE_TEXTURE_PROPERTIES.extents
// Note that this is intentionally not including stride values. Should just assume they are equal to having 
// 0 margin (that is, for a 2D texture/array, stride = extent * sizeof(T), where T is byte, ushort, half, or float
// depending on BufferPrecision) (although I don't know how strides work for multidimensional ...)
internal readonly struct ResourceTextureNDSize
{
    public readonly D2D1BufferPrecision bufferPrecision;
    public readonly D2D1ChannelDepth channelDepth;
    public readonly int dimensions;
    public readonly int extentX;
    public readonly int extentY;
    public readonly int extentZ;
}

public readonly struct ResourceTexture1DSize
{
    public D2D1BufferPrecision BufferPrecision { get; init; }
    public D2D1ChannelDepth ChannelDepth { get; init; }

    private readonly int dimensions = 1;

    public int Length { get; init; }

    private readonly int extentY = 0;
    private readonly int extentZ = 0;
}

public readonly struct ResourceTexture2DSize
{
    public D2D1BufferPrecision BufferPrecision { get; init; }
    public D2D1ChannelDepth ChannelDepth { get; init; }

    private readonly int dimensions = 2;

    public int Width { get; init; }
    public int Height { get; init; }

    private readonly int extentZ = 0;
}

// later on, user code creates the ID2D1Effect and goes to specify the resource texture...
using ComPtr<ID2D1Effect> spEffect = ...;

// assumes a generic overload of SetValue that takes a T:unmanaged and blits it over as a blob
spEffect.Get()->SetValue(
    D2D1PixelShaderEffectProperty.ResourceTexture2Size,
    new ResourceTexture1DSize() { BufferPrecision = Float32, ChannelDepth = 1, Length = 512 });

// might need to specify the D2D1PropertyType because there's probably a template/generic overload
// that takes T[] / ROSpan<T> and marshals it as D2D1PropertyType.Array ? unclear. Need to examine
// how the native side does this, as it has a bunch of templated helper methods (they also exist in TerraFX)
spEffect.Get()->SetValue(
    D2D1PixelShaderEffectProperty.ResourceTexture2Data,
    D2D1PropertyType.Blob,
    new float[512] { 3, 6, 7, ..., 89, 24 });