Add UASTC HDR.

Draft of language to add support for UASTC HDR. KDFS and all KDFS references need updating before this can be considered ready for merge.

Draft says to use colorModel KHR_DF_MODEL_UASTC_HDR (= 167) together with vkFormat VK_FORMAT_ASTC_4x4_SFLOAT_BLOCK (= 1000066000). Any issues with that?

Draft says to use colorModel KHR_DF_MODEL_UASTC_HDR (= 167) together with vkFormat VK_FORMAT_ASTC_4x4_SFLOAT_BLOCK (= 1000066000). Any issues with that?

@lexaknyazev please let me know your opinion on this as soon as possible. Rich plans to release his code very soon. We need to decide the representation in a .ktx2 file before then.

A version of https://github.com/BinomialLLC/basis_universal/wiki/UASTC-HDR-Texture-Specification-v1.0 needs to be incorporated into the Data Format Specification. That document has two parts: a description of the UASTC HDR subset of ASTC and a description of transcoding to BC6H. I think for simplicity and ease of use whole thing should be incorporated into KDFS but an argument could be made for putting the subset description in KDFS and the transcoding description into the KTX spec.

the transcoding description into the KTX spec

The unreleased KDFS version of the original UASTC spec contains both the bitstream and the transcoding steps. I think UASTC HDR should follow the same path.

@richgel999 says we need to save in the file the scale used to scale the maximum value from .exr/.hdr down to what BC6H/ASTC can handle, "roughly ~65k". We'll need to add a new standard metadata item for this. He promised me a reference to a similar field in the .exr standard. Once I have that to refer to I'll draft a description of the item.

The range for BC6H/ASTC HDR is [0.0 … 65504.0], same as non-negative 16-bit floats.

EXR supports three data types: float16, float32, and uint32. Leaving uint32 aside, other two types are signed. It would make sense to store two values: a scale and an offset, to be able to map any source range to UASTC HDR.

HDR (Radiance) range is [0.0 … 1.698e+38]. Although these values are always non-negative, it would still be useful to support both a scale and an offset for more efficient use of the UASTC HDR range.

So my proposal would be to add a metadata entry with an 8-byte payload containing two float32 values: scale and offset, with the following usage:

• Allowed for all floating-point Vulkan formats.
• When present, the effective texture values are sampled * scale + offset.

It's important to multiply first so that decoders could use a single FMA instruction to apply the scale and the offset at once.

• When present, the effective texture values are sampled * scale + offset

Let me make sure I have this correct. During encoding the "sampled" value will be calculated as sampled = (tv - offset) /scale where tv is the original texture value. Then that value is restored during sampling by tv = sampled * scale + offset. Correct? I realize that in the second case sampled is the filtered value from the texture so "sampled" in my 2 equations are not identical quantities. I'm trying to keep things simple.

Yes, that's correct. Exact choice of scale and offset is up to the encoder but they must be finite values, i.e., neither NaN nor Inf.

Whether sampled is post- or pre-filtered is an implementation detail. Assuming that original floating-point values are linear and the filtering is linear or nearest (e.g., not cubic) the results would be the same modulo FP precision.

So my proposal would be to add a metadata entry with an 8-byte payload containing two float32 values: scale and offset, with the following usage:

How about KTXmapRange which has the benefit of being a multiple of 4 bytes long (when the terminating NUL is included).

the transcoding description into the KTX spec

The unreleased KDFS version of the original UASTC spec contains both the bitstream and the transcoding steps. I think UASTC HDR should follow the same path.

I was going to email this, but this is as good a place as any...

I agree that describing both is ideal. I've been making decent progress in merging everything for a 1.3.2 release and should have something reviewable over the weekend, but we had stalled at the point of folding in UASTC before, and I wanted clarification on the thing that had previously made me hesitate:

There was a comment that "to get 32-bpp data from a UASTC texture, it's recommended to first transcode to ASTC, then decode that by following the ASTC specification and any applicable extensions", and I don't think I ever had clarification on whether this meant to remove the entire "Decoding process" section (which describes individual pixel colour values). Is that the intent, or should I preserve both, with the note that they're supposed to be identical and that decoding via ASTC is still recommended?

There was a https://github.com/BinomialLLC/basis_universal/issues/162#issuecomment-676170081 that "to get 32-bpp data from a UASTC texture, it's recommended to first transcode to ASTC, then decode that by following the ASTC specification and any applicable extensions", and I don't think I ever had clarification on whether this meant to remove the entire "Decoding process" section (which describes individual pixel colour values). Is that the intent, or should I preserve both, with the note that they're supposed to be identical and that decoding via ASTC is still recommended?

If you're using BasisU's transcoder, you can transcode UASTC LDR to basist::cTFRGBA32 (or another uncompressed LDR format, like cTFRGB565 etc.). Internally this would skip the ASTC block packing step, and be faster. It will decode the UASTC blocks to a logical format and then immediately decode these block pixels to 32bpp.

@richgel999 lack of review and approval from you has stalled this spec update. Please do it. I estimate it should take about 15 minutes.

@richgel999 lack of review and approval from you has stalled this spec update. Please do it. I estimate it should take about 15 minutes.

I've been away on vacation and just returned. Will look at this tomorrow.

I've been away on vacation

I hope you had a wonderful vacation. It's always nice to get away from the usual routine.

I hope you had a wonderful vacation. It's always nice to get away from the usual routine.

Thanks; implementing this new codec (with both RDO and intermediate) was an exhausting amount of effort.

I've added a note to our "ASTC HDR 6x6 Support Notes" page warning the user that our intermediate files are currently only readable/writable using our library/tools.

@richgel999 says we need to save in the file the scale used to scale the maximum value from .exr/.hdr down to what BC6H/ASTC can handle, "roughly ~65k". We'll need to add a new standard metadata item for this. He promised me a reference to a similar field in the .exr standard. Once I have that to refer to I'll draft a description of the item.

Yes, we need this extra metadata. Right now, when our compressor sees values greater than ~65k anywhere in the image, it scales the entire image's RGB texel values to the maximum encodable value, then it prints a loud warning. It's expected that the user will (probably) scale the fetched RGB values back up in a shader. Most HDR images I've seen in the wild don't have this problem, but to properly handle a very wide range of HDR outdoor images it's needed.

From memory, I'm pretty sure .EXR does support this metadata value already. I'll look it up.

Yes, we need this extra metadata.

@richgel999 This PR adds a standard metadata entry with a pair of floats representing scale and offset.

A version of https://github.com/BinomialLLC/basis_universal/wiki/UASTC-HDR-Texture-Specification-v1.0 needs to be

The latest URL of the UASTC HDR 4x4 spec is here: https://github.com/BinomialLLC/basis_universal/wiki/UASTC-HDR-4x4-Texture-Specification-v1.0

@richgel999 This PR adds a standard metadata entry with a pair of floats representing scale and offset.

That's perfect.

Note Windows ARM support is checked into the Basis Universal repo. Tested on Snapdragon X.

Sorry for my long delay on this. I ended up in a much deeper hole than expected getting the KTX-Software 4.4.0 release out. It is done now and has cleared the decks for the next release to focus on HDR work.

About removing the vendor GPU Photo supercompression scheme and reverting it to a standard scheme, I have a question for @richgel999. When entropy encoding is introduced will decoders need to be aware of it or is it purely on the encoder side? If the decoder needs to be aware then I am reluctant to make it a standard scheme at this point.

Also @richgel999 any further thoughts on naming for the intermediate format? As I wrote elsewhere I find using the same name for for both the supercompression scheme and block-compressed format confusing plus "UASTC HDR 6x6 Intermediate" is long and, I feel, unintuitive.

The bitstream specs still seem to be in flux. Do the decoders need to be aware of any of the planned changes, @richgel999.

I'd like to get this wrapped up as quickly as possible so we get approval from the WG and submit the spec. for ratification, which I think is necessary given the scope of the additions.

@richgel999 we need your response so we can finish this draft. Please answer. We need to quickly get to a situation where the files your tools are producing are legitimate KTX v2 files that will be validated as such by ktx validate.

Ping @richgel999. We cannot make progress without your answers. Please take the 5 or 10 minutes it will take you to answer and provide the answers. This has dragged on far too long, nearly 6 months now. :pleading_face:

I've been away for more than 2 weeks. I had hoped to find the answers on my return. :disappointed:

About removing the vendor GPU Photo supercompression scheme and reverting it to a standard scheme, I have a question for @richgel999. When entropy encoding is introduced will decoders need to be aware of it or is it purely on the encoder side? If the decoder needs to be aware then I am reluctant to make it a standard scheme at this point.

Not having or requiring entropy coding is a modern feature, not a deficiency. Entropy coding slows decoding for relatively minor gains. The really big potential gains will come from encoder improvements that don't change the decoder or format at all: better saliency mapping (smarter perceptual weighting, deeper variance spectrum, DCT-driven analysis, perform the variance spectrum analysis in ITP colorspace vs. RGB), better adaptive block pre-filtering to reduce over-fitting on low priority high frequency features, etc. - all 100% encoder-side things. The potential gains there are large (10-30%) because the current encoder's perceptual weighting is purposely very conservative.

The format and decoder design are command driven, like many video codecs. The encoder has all the smarts - the decoder just follows the commands emitted into the stream. The commands are based off the ASTC HDR standard, but at a higher level: "repeat last block X times, emit this solid color ASTC block, emit this raw ASTC block with this mode, reuse the left block's endpoints and ASTC mode but use this weight grid, or reuse and re-quantize this previous ASTC block's endpoints but apply this RGB endpoint delta and use this weight grid upsampled", etc.

But if we do decide to introduce entropy coding - yes this will require a decoder update. When/if we do this, we'll also add in alpha and support another larger block size, too. It'll be an entirely another system, so the existing system will remain 100% unchanged.

Supporting larger block sizes also give the system gains without needing entropy coding. Internally we're already at 8x8 (2 bpp) and 10x10 (1.28 bpp) for LDR, and 8x5 (3.2 bpp) and possibly 8x6 (2.67 bpp) HDR should be achievable.

Also @richgel999 any further thoughts on naming for the intermediate format? As I wrote elsewhere I find using the same name for for both the supercompression scheme and block-compressed format confusing plus "UASTC HDR 6x6 Intermediate" is long and, I feel, unintuitive.

We're internally calling it "GPU Photo HDR 6x6" or just "GPU Photo HDR".

The bitstream specs still seem to be in flux. Do the decoders need to be aware of any of the planned changes, @richgel999.

No, the bitstream for the current 6x6 intermediate HDR format is not changing. We'll introduce another system with more features.

Thanks @richgel999. Based on your answers I will revert "Basis GPU Photo HDR 6x6" back to a standard format using that full name to avoid confusion with any possible future formats. Is the format underlying "Basis GPU Photo HDR 6x6" a standard ASTC subset or something special? Do we need to document it separately?

I will revert "Basis GPU Photo HDR 6x6" back to a standard format using that full name

Does it have to be this way? A few months ago, we discussed naming it "UASTC HDR 6x6" as it's more precise and descriptive than "GPU Photo".

Does it have to be this way? A few months ago, we discussed naming it "UASTC HDR 6x6" as it's more precise and descriptive than "GPU Photo".

"GPU Photo" is essentially "UASTC HDR 6x6" with supercompression. It is also known as "UASTC HDR 6x6 Intermediate" format. "UASTC HDR 6x6" itself is a subset of ASTC decodable by any ASTC decoder.

The Binomial's wiki (dd Feb 2025) explicitly refers to that format as "UASTC HDR 6x6 Intermediate". I suggest we do the same here wrt the KTX spec sections and enums.

"Basis GPU Photo 6x6" is mentioned as the project's internal name, we could have a note about that to avoid any confusion.

All committers have signed the CLA.

I have updated the draft with based on my conversions with Binomial and the answers to my questions.

• Basis UASTC HDR 6x6 Intermediate has been added as a standard supercompression format with links to documentation for its bitstream and global data.
• The Basis UASTC HDR section is now just Basis UASTC HDR 4x4.
• A new section Basis UASTC HDR 6x6 has been added which is described as a custom format based off UASTC HDR 6x6 that underlies Basis UASTC HDR 6x6 Intermediate supercompression.

Note that the Basis transcoders can now transcode any ASTC 6x6 HDR texture to BC6H. Their encoder limits itself to 75 ASTC modes for performance reasons in what Binomial refers to as Plain ASTC HDR 6x6 but the transcoder doesn't care about this limit. I see no reason to say anything about Plain ASTC HDR 6x6 format in the KTX specification as it is purely on the encoding side.