[sc34-wg1]

Mon Nov 1 14:37:59 CET 2010

sc34wg1study-bounces at vse.cz wrote on 11/01/2010 06:57:54 AM:

> 
> Many thanks for doing this – this looks like an excellent basis for 
> moving forward.
> 
> One thing that caught my eye is this opening words, which I hope 
> this group will be able to work on unpacking a bit: “ZIP archive 
> compression is a long-standing, widely-adopted technology described 
> by PKWARE in their ZIP Application Note”.
> 
> So, breaking this down:
> 
> > ZIP archive compression is a long-standing, widely-adopted technology 
…
> 
> Might it be more correct to call Zip a “family” of technologies 
> since, as we discussed on our last call, many variants are in use at
> any one time? Granted, it may be a closely related technology 
> family, and may well exhibit a good deal of forwards-compatibility –
> but for newcomers coming to our work I think it’s important to 
> understand that we’re not talking about “one spec; one 
> implementation” but something a little subtler than that.
> 

One could make a similar argument about HTML, couldn't one?  You have the 
W3C version, you have the ISO version, you have what individual browsers 
support, what individuals authoring tools write out.  I'm pretty sure you 
could complicate the discussion of almost any standard implemented by 
multiple vendors that way.  Even standards implemented by a simple vendor 
could be made impressively complicated this way, say ISO/IEC 29500.  But 
what's the point?

> > … described by PKWARE in their ZIP Application Note
> 
> The PKWare appnote is one description of Zip, a couple of others I 
> can think of are the (derivative) Info-Zip series of app notes; and 
> a further description of Zip is what existing application *do*, and 
> what existing archives *contain* (and, in any contest between the 
> written spec and existing practice, it is probably the case that the
> latter “wins”). Does anybody know of any other sources of Zip 
specification?
> 

PKWare was the company formed by the undisputed inventor and author of the 
ZIP technology, Phil Katz.  PKWare has maintained the ZIP Application 
Note, in consultation with other vendors, for over 20 years.  I'd 
challenge anyone to name another specification that has a greater claim.

> You mention in particular the Java Zip ecosystem. If we look at:
> 
> http://download.oracle.com/javase/1.4.2/docs/api/java/util/jar/
> package-summary.html
> 
> We see that “JAR format is based on the Info-ZIP file format” and 
> from there we can follow a link to the Info-Zip application note 
> 970331 (the link is dead, but I’ve put a live link on the Wiki). 
> Unsurprisingly, this is the very same document which specifies the 
> “Zip” of ISO/IEC 26300 (ODF 1.0). Later versions of the Info-Zip 
> note describe the FOSS “Zip” tools that enjoy very wide use 
> throughout the industry.
> 

But look here and you see Oracle says, "Jar consists of a zip archive, as 
defined by PKWARE".

http://download.oracle.com/javase/1.4.2/docs/guide/jar/jarGuide.html

As for ODF, as you certainly know, ODF 1.0 only added a reference to the 
InfoZip at the request of the UK in their DIS ballot comments.  The 
original OASIS version referred to the ZIP Application Note.  The 
reference to InfoZip provided by the UK disappeared when the referenced 
web site retired.  So we've changed the reference back to the PKWARE 
Application Note in ODF 1.2.  I think stability of reference is important.

> So, as I understand it, it is possible to create Zip archives that 
> conform to PKWare’s latest appnote that will not interoperate with 
> tools/archives from (in this example) the Java Zip ecosystem … this 
> is why I think we need to be very careful about asserting that “Zip 
> == PKWare appnote” without qualification – my Zip and your Zip may 
> be different.
> 

And it is possible to create OOXML documents that MS Office cannot read. 
So?  Does this really mean that we're discussing two different standards? 
Or a single standard that has partial implementations?  Let's not make 
things more complicated than they are.  Certainly no simpler, either. 

I think we can all acknowledge that even if we could snap our fingers and 
magically produce an ISO ZIP that was equivalent to Info ZIP or to the 
Application Note, or to any level in between, that exactly the same 
situation in the real world would exist, namely that there are some 
archives that will not be compatible with some tools, especially if the 
tool implements only a subset.  The real world appears to have managed 
with this state of affairs for over 20 years, without any evident 
problems.  So before we try to improve things, let's first pledge not to 
make things worse.

> Does anybody have information about what the .NET ecosystem expects 
> in its Zips? I imagine C/C++ programmers most usually reach for the 
> Info-Zip libraries when using Zip …
> 

No idea.

> The other issue here is the mention of “*their* ZIP Application 
> Note” in relation to PKWare. In the Tokyo WG 1 meeting there was 
> quite some interest in the topic of whether/how the Zip 
> specification was in the public domain, and in particular the 
> wording of the press release copied at:
> 
> http://cd.textfiles.com/pcmedic9310/MAIN/MISC/COMPRESS/ZIP.PRS
> 
> in which PKWare and Infinity Design Concepts Inc. wrote (rather 
> wonderfully): “[t]he ZIP file format is given freely into the public
> domain and can be claimed neither legally nor morally by any 
> individual, entity or company (or any other sentient creature in the
> universe.)” So I also think we need to be careful about asserting 
> ownership of the format – might it be more correct (and neutral) to 
> say that PKWare has played a major role in maintaining and extending
> a specification of the format?
> 

If you talk to a lawyer, you'll be told that "public domain" has no set 
meaning, except for creative works that are older than their statutory 
copyright terms.  So this press release -- if genuine -- is at best 
ambiguous.  As you know, it was "discovered" by one SC34 "invited expert" 
who then put on the ZIP Wikipedia page to give it credence.  The page does 
not even load for me right now.  And it is certainly not at a site that 
immediately suggests authenticity. 

From the quotes I've seen, it seems clear that they are talking about the 
rights needed to implement ZIP.  However, note that the rights to 
implement a specification and the rights to create derived specifications 
are entirely different species of IP, and typically have entirely 
different terms.  For example, most OASIS standards (and W3C standards and 
even some ISO standards) are free to implement.  And although some of the 
specifications (those from OASIS and the W3C, for example) can be freely 
copied, redistributed, translated, etc., if you read the copyright notice 
carefully you will see that they do not include the rights needed to 
publish a new specification or standard based on them, the so-called 
rights to create derivative works. This "anti-forking" provision is very 
common in SDOs.  There are many open standards, but putting the 
specification itself into the public domain is almost unheard of.

If you look carefully, you'll see that even the Info-Zip specification 
correctly bears a PKWARE copyright notice, since it is a derived work.  So 
I don't think the Info Zip specification gets you around any copyright 
concerns.

So I think we need to be very careful here.  Unless one is willing to 
create a clean room, reverse-engineered version of the ZIP specification, 
with a team who has read neither the PKWARE Application Note nor the 
Info-Zip derived version, then I don't see how you can go forward with 
this project without agreement from PKWARE.  Copyright means something, 
even in Geneva.

I think the compromise position here is quite clear. 

1) Encourage PKWARE to update their Application Note to make it easier to 
profile it.

2) Encourage PKWARE to define a public defect submission and resolution 
process.

3) ISO standards that reference ZIP Application Note directly would use 
the RER process. 

4) WG1 creates a profile standard based on the revised Application Note. 
This would require an RER as well.  IPR and maintenance would be clarified 
in the RER.

Very simple.  It uses SC34 experts within their area of expertise, and 
PKWARE within their area of expertise.  It solves all problems.  We 
declare victory.  Everyone is happy.  And no one steps on anyone's IPR.

Regards,

-Rob

> - Alex.
> 
> 
> From: sc34wg1study-bounces at vse.cz [mailto:sc34wg1study-bounces at vse.cz] 
> On Behalf Of Andrew Rist
> Sent: 29 October 2010 21:44
> To: ISO Zip
> Subject: [sc34-wg1]
> 
> I have taken as my task to list the root problems associated with 
> the use of the ZIP format in document format standards.
> This task does not involve the suggestion or prescription of any 
> particular solution, and includes the understanding that WG1, SC34, 
> or even ISO may not be the suitable forum to resolve these issues. 
> 
> The following is what I have put up on the WG1 wiki, we can develop 
> it further there.  Please limit edits to focus on identifying the 
> problem and not to define solutions.  (for this section of the wiki,of 
course)
> 
> 
> ZIP archive compression is a long-standing, widely-adopted 
> technology described by PKWARE in their ZIP Application Note.  This 
> technology is used for many purposes in the industry, from the 
> archiving and compression of files to packaging of applications 
> (Java jar/ear/war). 
> 
> Additionally, in recent years there have been a number of document 
> format standards that use ZIP as a "container file" for storing XML 
> and related resources.  For example, OASIS ODF (ISO/IEC 26300), Ecma
> OOXML (ISO/IEC 29500), IDPF EPUB and W3C Widgets. 
> 
> This use of the ZIP Application Note, as an external normative 
> reference, by International Standards or specifications which may be
> on the track to become International Standards, presents the 
> following problems:
> There is no Standards Compatible Reference associated with the .ZIP 
> Application Note.
> There is an ambiguous IPR landscape, especially related to the IPR 
> referenced in the .ZIP Application Note.
> There are technical issues related to the use of ZIP as a document 
> package which are not covered by the .ZIP Application Note
> ZIP is a ubiquitous and highly interoperable technology, and any 
> standards activity relating to ZIP should not negatively impact the 
> current use of ZIP or its current use in standards.
> 
> 
> As back up to the main problems identified above, here are a set of 
> more specific issues that fall under these categories:
> No Standards Compatible Reference 
> Current Application Note is not referenceable by International Standards
> There is no mechanism to reference parts of the Application Note
> There is ambiguity in terms of the future maintenance of the 
> Application Note (e.g. feedback procedures and transparency)
> Ambiguous IPR Landscape
> It is not possible to identify which parts of the Application Note 
> are subject to IP and which are not
> The use of ZIP in Open Standards (which are implementable in all 
> Open Source) requires any IP used in the standards to be licensed 
> under RF terms. 
> Technical Issues related to the use of ZIP as a Document Package
> Minimum feature set relevant to all document packaging use of ZIP
> additional syntax
> additional objects and metadata
> signatures and encryption
> ZIP URL protocol & fragment identifiers
> Do No Harm to the Current Usage of ZIP
> Ubiquitous nature of ZIP creates wide reaching benefits (utilities 
> built into all development platforms and OSes)
> Currently in use by wide range of standards
> ODF, OOXML, EPUB
> W3C widgets
> Java (jar, war, ear, java.util.zip)
> XPI 
> ADL-SCORM
> 
> 
> 
> 
> 

> -- 
> 
> 
> [image removed] 
> Andrew Rist | Interoperability Architect
> Oracle Corporate Architecture Group
> Redwood Shores, CA | 650.506.9847 
> 
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email 
> ______________________________________________________________________
> _______________________________________________
> sc34wg1study mailing list
> sc34wg1study at vse.cz
> http://mailman.vse.cz/mailman/listinfo/sc34wg1study