Code Poetry
and Text Adventures

by catid posted (>30 days ago) 1:00pm Sat. Oct 24th 2015 PDT
I put up this library today on github:

It's built on Wirehair (hybrid GF(256)+LDPC) and my new CM256 reed solomon block codec.  The idea is to use RS for small numbers of blocks where its O(N^2) performance is still practically fast, and switch to Wirehair for larger block counts.  The switch-over seems to happen at about N=32.

I also optimized Wirehair and improved its heavy GF(256) matrix based on the fast CM256 software.

At this time it's not usable because I haven't finished crunching all the seeds for all 64000 values of N (block count).  I have a computer working on it right now, and that should be done by the end of the week.
Also the code isn't well tested and I want to improve a few things, fix the code comments, document it, etc.

One other interesting avenue on the decoder side is to try to reduce the amount of work by only solving the data needed to recover the blocks that are missing.

A recovered block is an XOR combination of just 5 intermediate blocks + a random number of other blocks.  If only one block is missing, then it suffices to produce just the intermediate blocks necessary for solution.  And in fact this is a very common case, as losses are usually random and low probability events for many applications.

If the useful intermediate blocks are known apriori, then perhaps the whole decoding process can skip a lot of work and go faster while using less memory... =)