Compressing Human Knowledge

Marcus Hutter, a German computer scientist and professor at the Australian National University, funded in August 2006 a 50.000 euros cash prize, which rewards data compression improvements on the first 100.000.000 characters of a specific version of English Wikipedia (envik8). Specifically, the prize awards 500 euros for each percent improvement in the compressed size of the file enwik8. The prize baseline was 18,324,887 bytes, achieved by PAQ8F, a free lossless data compression archiver. The contest is open ended and is open to everyone. The ongoing competition is organized by Marcus Hutter, Matt Mahoney and Jim Bowery.

The goal of the Hutter Prize is to encourage research in artificial intelligence (AI). The organizers believe that text compression and AI are equivalent problems. There is no general solution to achieve this goal because the descriptive complexity is not computable. In algorithmic information theory the measure of the computational resources needed to specify an object is called the Kolmogorov complexity, named after Andrey Nikolaevich Kolmogorov, a Soviet mathematician.