The field of lossless compression is not maxed out. I found I could reduce the size of GWT JS 3% over gzip -9 by using 7-zip as a DEFLATE compressor. bzip2 using BWT is a nice improvement over gzip. But take a look at this: http://uclc.info/gimp_so... and compare modern lossless compressors, 2x-3x gains! Memory/CPU are issues.
jho,
Richard Walker,
Amund Tveit,
Matt M (inactive),
Mo Kargas,
Micah,
Tracy,
and
Joel Webber
liked this
Check out the AdvanceCOMP suite as well. The advpng tool can re-deflate PNG files, usually chopping off a few extra %. Makes me wonder- should GWT output the gzipped scripts by default using a Java port of a 7zip-tuned deflate?
- Matt M (inactive)
I've been talking privately about doing just that, a post-linker to output .cache.(js|html).gz files using 7zip (or a custom hacked pure-java deflate), and potentially a ServletFilter for content-negotiation for those who don't want to muck with apache/nginx configs. Adding a PNG optimizer for ClientBundle would be good too, since the dataURL expansion would benefit a lot from every byte saved. Any voltuneers? Look at JzLib and gzip1.2.4-hack if you want to see how to hack deflate to do 7-zip style optimization
- Ray Cromwell
On a related note, it would be nice if the W3C/IETF would take a serious look at adding LZMA support to their specs. Unlike PPM(n) and the really great lossless compressors, it features reasonable memory and CPU performance and does quite a bit better than deflate and deflate + BWT (as in 40% better), plus it seems unencumbered by patents.
- Ray Cromwell
"seems unencumbered by patents" - XOR cursor sure "seemed" "obvious to anyone in the field" of computer graphics. Don't get me started :/
- Richard Walker
I'd be happy with Accept-Encoding: bzip2 being turned on everywhere. Apparently it caused some issues with poorly written proxies though. Surprised there's not an Accept-Encoding: lzma proposal floating out there already!
- Matt M (inactive)
Let's sponsor an entry to this iana officially registered HTTP transport encoding parameters. After that, someone can file bugs on all the major browsers complaining that they don't support the standard formats: http://www.iana.org/assignm...
- Matt M (inactive)
bzip2 is nice, but it's alot slower with only marginal gains. The lowest level of LZMA is faster than the highest level gzip/bzip2 but compresses significantly more. More importantly, gzip/lzma decompression time is not affected by compression level, but bzip2 gets slower at higher levels. See http://tukaani.org/lzma... CPU consumption/speed on decompress is far more important than slowness on compression, since most of the data you want to compress is/can be cached. This is especially important for mobile. In fact, lzma seems really good for mobile optimization.
- Ray Cromwell
I downloaded the AdvanceCOMP source last night and tried to see what it might take to automate a C++ to Java conversion on their improved deflater. I quickly remembered why I love Java so much...
- Matt M (inactive)