Code Poetry
and Text Adventures

by catid posted (>30 days ago) 2:49am Fri. Aug 16th 2013 PDT
So I recognize that my benchmarks for GCIF + lossy formats were flawed, since the diff tool was not being used correctly, and so the GCIF residual inputs are not in any way representative of the actual data.  After testing correctly, the residuals are actually harder to compress as you may expect since they're somewhat unpredictable.

But ASTC is interesting on its own:

I just finished reading everything I can get my hands on for ASTC including all the source code.  It's really fantastic code and I want to study parts of it more to pick up some new software engineering tricks.

To decode ASTC, you break the image up into blocks.  In each block there are a variable number of color pairs, 2-4, and variable precision for these.  The first 10 bits of each block determines how the color pairs are distributed around the block in a guided-pseudo-random pattern.  Then each texel is roughly assigned a number from [0..1] to represent how close to each color pair it is, also with variable precision.

This seems a little gimmicky to me.  It's definitely an improvement on PVRTC but it still seems best suited for game textures, where the art is hand-drawn and lacks the mix of color seen in natural photography.  Hand-drawn art tends to use a few distinct colors in brush strokes, and ASTC seems designed to take advantage of this kind of decomposition.  It can definitely handle 2 colors being blended together really well, but not 3 colors blended together.

ASTC has a *ton* of tuning knobs and the compressor could probably be doing a far better job than it currently does.  This is a great feature of a compressor design.

Because an exhaustive search can be performed to match texels to colors, ASTC can handle rough edges much better than other lossy formats if it is given time to explore all the options.  The "search space" is so large that the best compression mode is exceptionally slow.  It also has a mode to weight block edges higher to avoid block artifacts (-b flag), though it is a block-based compressor so will necessarily have some discontinuities as a result.

The ASTC decompressor needs to decode the block, generate the color assignments for each texel, decode the texels by interpolating between colors, and then interpolate them into pixels.  This process is fairly processor intensive rather than memory intensive, which is fantastic for a GPU-based approach.  It seems like the tables required by a software decoder are all fairly small.  In comparison, GCIF is memory-intensive in that it builds its predictions from earlier pixel data and other intermediate data in memory.

Still thinking a lot about ASTC, PVRTC and how it relates to other image compression schemes.  Hopefully I can learn something that will lead to a better lossless compressor.

The difference between a lossless and lossy compressor seems to be mainly made up of the residuals left over.  Filter data takes less than 2 bits per pixel, reducing almost all of the redundancy of the image.  What's left over is the residual details, which can be, incredibly, left out or approximated for lossy image compression.  The perceived result is still "pretty good."

Optimizing the filters and tightening up the filter zones can squeeze a lot out of the data.  But what really seems to give GCIF (and WebP) a boost in compression is the entropy encoding of residuals.  By trying a lot of options, both GCIF and WebP seek to reduce the entropy of these residuals.  That doesn't just mean targeting small residuals -7..7, but actually means increasing the random chance of two residuals having the same value, which means they can use the same smaller Huffman code.

ASTC and PVRTC haven't given me any new tools to use, but has made me interested in trying to add another filter layer that can nudge residuals into more similar values and group together more of the same larger values.
last edit by catid edited (>30 days ago) 12:46pm Fri. Aug 16th 2013 PDT