NVIDIA on Thursday published their first public beta of their RTX Neural Texture Compression “RTX NTC” software development kit.
The NVIDIA RTXNTC v0.5 beta was published on Thursday as an early preview of this new compression scheme for Windows and Linux systems and working both with Vulkan and Direct3D 12. NVIDIA’s Neural Texture Compression is self-described as:
“Neural Texture Compression (NTC) is an algorithm designed to compress all PBR textures used for a single material together. It works best when the texture channels are correlated with each other, for example, detail in the albedo texture corresponds to detail in the normal texture. Up to 16 texture channels can be compressed into one NTC texture set. Typical PBR materials have 9-10 channels: 3x albedo, 3x normal, metalness, roughness, ambient occlusion, opacity.
During compression, the original texture data is transformed into a combination of weights for a small neural network (decoder) and a tensor of latents or features that are sampled and passed through the decoder to reconstruct the texture colors, as is illustrated below. The sampling and decoding processes are fast enough to use them directly in the shaders that normally sample the material textures, such as base pass pixel shaders or ray tracing hit shaders. However, the decoder produces unfiltered data for only one texel, and in order to get filtered textures, we suggest using NTC in combination with Stochastic Texture Filtering (STF). For renderers targeting lower-end hardware, we suggest implementing the “Inference on Load” mode where NTC textures are decompressed when the game or map is loaded, and transcoded to one of the block-compressed formats (BCn) at the same time.”
NVIDIA Neural Texture Compression comes out of their graphics research. This SDK initially features a library for neural texture compression/decompressiion, an “ntc-cli” command-line tool for handling compression and decompression, an app for interactively exploring NTC compression and viewing NTC files, and some example code. This code is available under the NVIDIA RTX SDKs license.
Those wanting to learn more about this initial public beta of the NVIDIA RTX Neural Texture Compression software can find it on GitHub.
