Done some further optimization of the algorithm and I'm now hitting 6-9 MB/sec encoding speeds. This means that for example it can convert a 700 MB movie to images in 1 to 2 minutes. Also reduced the memory footprint 10-fold.

I think I'm gonna write the decoder today and make a basic app available for public testing.