Common ZIP

Common ZIP is an open source specification for the ZIP file format we've all been using. This project is not affiliated with PKWARE, Inc., the original creators of the ZIP file format.

Although PKWARE publishes a specification called APPNOTE, it has numerous problems. This blog post is a good start: ZIP File Format Considered Harmful . The Common ZIP project was created to address these problems. The community of open-source ZIP software creators needs technical leadership that will listen and address the 35-year-old design flaws in the ZIP file format.

Common ZIP Spec

The Common ZIP Spec is a complete re-specification of the ZIP file format from scratch, excluding DEFLATE, CRC32, and UTF-8. It is a guide to:

• creating ZIP files that can be read by most existing implementations,
• reading ZIP files created by most implementations,
• and guarding against surprising behavior from untrusted inputs.

If you are writing a ZIP file implementation, this document is for you.

Tests

The test suite is under development and will be published soon. So far, it examines 11 existing implementations such as Info-ZIP (command line), archive/zip (Go), zipfile (Python), and ZipFile (Java), runs 93 test cases, and has already found numerous interesting bugs and oddities.

Unfortunately, communicating what exactly these bugs and oddities are is very difficult and nuanced, and I (Josh Wolfe) don't want to publish these results until I am confident I can communicate it all effectively, which also requires a deeper understanding of it. Additionally, due diligence is needed to verify that none of the found bugs are security critical, which would warrant disclosing the vulnerability to the authors and allowing time for response and action before publishing the results to the general public.

An important goal of this test suite is to provide evidence backing up the guidance in the Common ZIP Spec. For example, extra field 0x7875 is not important to support, because almost no implementations support it today. The test suite will provide clear evidence of that.

A major limitation of the test suite is that it is only testing interesting ZIP files given to ZIP reading implementations (e.g. extractors); it does not test ZIP file creators. It will be a profoundly different effort to create a test suite for ZIP file creators.

Contributing

To contribute to the Common ZIP project, see the source code on GitHub.

Support

Thank you for your interest in supporting the Common ZIP project! This is a strictly volunteer operation fueled by enthusiasm. Please leave a ❤️ and optionally a comment on Issue #1 to cheer us on! We're all working to make the world a better place, and it keeps the faith alive to receive expressions of support from said world. :)

If you have money you'd like to part with, please consider donating to the charities listed in the above linked issue, and optionally leaving a comment saying that you did. Good will is contagious! Thank you!