KTX2 for non-color data

[lilleyse]

First of all, I'm very excited to see the development on KHR_texture_basisu.

We're looking at using KTX2 as an image format for storing non-color data, specifically uint16 and floating point data. We're working on a glTF extension for adding generic per-vertex and per-texel metadata to models and KTX2 seems like a good alternative to png/jpeg since data types are more clearly defined.

One example might be a per-texel "accuracy" that describes the the error in meters along a photogrammetry model compared to the ground truth. This can be stored in a KTX2 image using the VK_FORMAT_R32_SFLOAT format.

With this use case in mind there's still rationale to support baseline KTX2 in glTF. Maybe there's a chance of reviving #1612 or creating a new extension for this use case.

Also see CesiumGS#1 for more details on the feature metadata extension.

[pjcozzi]

Maybe there's a chance of reviving #1612 or creating a new extension for this use case.

One approach is to create an extension for this per-texel accuracy use case that uses KTX2 to store the texels in VK_FORMAT_R32_SFLOAT and then see if we gain any insights on how a more generic KTX2 extension for glTF could work. The concern is that today the data types have well-defined semantics such as color and normals; KTX2 has several data types and the runtime would not know what to do with them without additional semantics.