Upgrading the 29500 DR Processing System (was: Following up on my action item to give feedback on my DR Log V2 Plan)

[Jesper Lund Stocholm]

Hi Rex,

I think that sounds like a good idea - although I think a reasonable
compromise should be made in having you using Office 2007 (I presume) to
maintain DR-files and index while you use a SVN-client to commit changes
to SVN in assembla. I have just tested this, and it is simply a
two-click operation to add new files or upload modified files to SVN.

If you (or anyone else in the group) hasn't worked with SVN/TortoiseSVN
before, I'd be happy to write up a small step-by-step-manual for us.

I'd also be very interested in the c#-code you have written to help you
(but that's more a personal thing)

:o)

Jesper Lund Stocholm
ciber Danmark A/S

> -----Original Message-----
> From: Rex Jaeschke [mailto:rex at RexJaeschke.com]
> Sent: Thursday, October 01, 2009 3:30 AM
> To: 'SC 34 WG4'
> Subject: RE: Upgrading the 29500 DR Processing System (was: Following
> up on my action item to give feedback on my DR Log V2 Plan)
> 
> If we go the "1 docx file per DR" route, then, yes, I was planning to
> make
> all individual DR docx files available. And also the summary xlsx DR
> Log
> Index, and, if anyone wants, the C#/.NET source I use to generate the
> SML
> from the WML file set. Then, as we get new DRs, or I update existing
DR
> files, I'd distribute only the new/changed DR docx files.
> 
> At any time, it would be trivial for me to zip up the complete set of
> current DR docx files for distribution.
> 
> Rex
> 
> 
> > -----Original Message-----
> > From: Jesper Lund Stocholm [mailto:jesper.stocholm at ciber.dk]
> > Sent: Wednesday, September 30, 2009 3:03 AM
> > To: Rex Jaeschke; SC 34 WG4
> > Subject: RE: Upgrading the 29500 DR Processing System (was:
Following
> > up on my action item to give feedback on my DR Log V2 Plan)
> >
> > Hi Rex,
> >
> > Thank you for your elaborate reply to Murata-san's proposal. It has
> > certainly made me reconsider my position on the usage of assembla.
> >
> > From the beginning, I have stated clearly, that I do not want to
> impose
> > unnecessary tasks on the project editor (you). Since you clearly
feel
> > that you have a better proposal, I'd say we go with your proposal -
> and
> > maybe using SVN to maintain electronic versions of the schemas.
> >
> > I think it is extremely important for our work that we only have
"one
> > version of the truth". If you are comfortable with this "one
version"
> > being in multiple DOCX-files, I am quite happy with this.
> >
> > I do have one question: the WG4 website holds a copy of the DR-log
> and
> > that copy is being routinely updated (by Murata-san, I presume). Do
> you
> > think it would be possible to have the WG4-website hold copies of
the
> > individual DOCX-files as well as they are updated? This would enable
> > (anyone of) us to use these files in any sort of way to fit each of
> our
> > specific needs.
> >
> > And in the spirit of "bringing home the deer I shot", I'd be happy
to
> > take the task of being the one to update the (large) list of
> individual
> > files on WG4 website.
> >
> > Again, thank you for your response :o)
> >
> > Jesper Lund Stocholm
> > ciber Danmark A/S
> >
> > > -----Original Message-----
> > > From: Rex Jaeschke [mailto:rex at RexJaeschke.com]
> > > Sent: Wednesday, September 30, 2009 4:12 AM
> > > To: SC 34 WG4
> > > Subject: Upgrading the 29500 DR Processing System (was: Following
> up
> > on
> > > my action item to give feedback on my DR Log V2 Plan)
> > >
> > > Murata-san has posted a reply to my proposed DR Log V2, and made
> his
> > > own
> > > proposal. However, his posting still does not address my main
> > concerns,
> > > which are as follows:
> > >
> > > A. What is the problem he is trying to solve?
> > > B. Does WG4 agree that problem is real and actually *needs*
> solving?
> > > C. Who will provide the resources for the work that would be
needed
> > to
> > > implement the solution?
> > >
> > >
> > > Before I address Murata-san's specific points, here is some of my
> > > thinking
> > > with respect to 29500 DR Processing:
> > >
> > > 1. I had a detailed look at our email archive, and here's what I
> > found.
> > > For
> > > the 9 months between October 23, 2008 (the first message), until
> July
> > > 31,
> > > 2009 (when we closed-out the DCOR1 and FPDAM1 sets)
> > >
> > > * There were 650 emails posted, of which I estimate at least 20%
> > [130]
> > > were
> > > administrative in nature and/or not directly connected to DR
> > > processing.
> > > That leaves 520 DR-related messages, which comes to approximately
> > > 60/month
> > > and 15/week.
> > > * We disposed of 293 DRs
> > > * 85 DRs were still open.
> > >
> > > Based on my experience, we made *very* good progress.
> > >
> > > Note that there are 64 members of WG registered to access the
email
> > > list.
> > > That means that the average number of postings per member per
month
> > is
> > > 1!
> > >
> > > 2. Given that 29500 provides a rich set of facilities to help in
> > > publishing
> > > documents, it seems like a *really* good idea to use those as much
> as
> > > possible to help manage the 29500 standardization process. After
> all,
> > > shouldn't we be promoting the use of our own standard? (I use some
> of
> > > these
> > > facilities already, and my recent proposal uses more.)
> > >
> > > 3. I am not opposed to using Assembla, or something like it, to
> > *help*
> > > with
> > > DR processing. (One example appears to be to maintain various
> > versions
> > > of
> > > the *electronic* schema.)
> > >
> > > 4. As editor, I believe strongly that the format of any text in
> > interim
> > > "spec" releases should be the same as that intended for the final
> > > version,
> > > so readers get comfortable with, and can gain confidence in, the
> work
> > > of the
> > > author. And since our main goal is to produce CORs and Amds (and,
> > > eventually, a revised standard), then we should use a consistent
> > style
> > > at
> > > every step along the way. (That is, not only do we need support
for
> > > rich
> > > text, we need support for the styles actually used in 29500).
> > >
> > > 5. In a world of finite (and currently diminishing) resources,
when
> > it
> > > comes
> > > to developing tools to do a particular job, I'm at the minimalist
> end
> > > of the
> > > spectrum. From that comes my long-held attitude of "It might not
be
> > the
> > > best
> > > that is possible, but it sure looks good enough." This does *not"
> > mean
> > > I'm
> > > in favor of doing a poor job; quite the opposite, I'm very big on
> > > quality
> > > assurance. It simply means that we almost certainly do not *need*
> the
> > > best
> > > system, we almost never have the resources to build it anyway or
to
> > > deliver
> > > it in a timely fashion.
> > >
> > > 6. The fact that one can access something on-line, at any hour of
> the
> > > day,
> > > is not justification in itself for having such access. And that
> goes
> > > the
> > > same for *wanting* all kinds of statistics.
> > >
> > > 7. Let's be realistic about the usage we have made of the existing
> DR
> > > Log
> > > system before we start claiming that we need "something better".
> The
> > > existing system is email-based, with no registration or education
> > > required
> > > to use it, and yet we've averaged roughly one participation per
WG4
> > > member
> > > per month.  If we haven't used it all that much, why is that?
> > Perhaps,
> > > we
> > > have used the existing system "enough", and "more" is not
> necessary.
> > >
> > > 8. We have a master DR Log. It is intended to contain all
> information
> > > relevant to any DR, from its initial submission, to its discussion
> on
> > > the
> > > email list, and in meetings, through to the final resolution.
> > > (Supporting
> > > documents are not included in the DR Log itself, but are made
> > committee
> > > documents that are referenced in that Log.) If anything is
missing,
> > > then
> > > members should be telling me, so I can rectify the situation.
> > >
> > > 9. But, the DR Log is large and growing: In my recent proposal I
> > > suggested
> > > breaking it into individual DR chunks. That said, so what if it is
> > > large? So
> > > long as you can download a copy and load it into some tool to use
> it,
> > > then
> > > who cares what its size is? Can you do real work with it? If not,
> say
> > > what
> > > the problem is, and let's address that.
> > >
> > > 10. The DR Log is sometimes out of date: I have issued a revised
> Log
> > on
> > > a
> > > regular basis, typically soon after each meeting or call, which
has
> > > been
> > > every 2-4 weeks. In-between times, we all have access to the
emails
> > and
> > > attachments that members have posted, so we do have access to
*all*
> > > contributions, in one of two places: the DR Log or our email in-
> > trays.
> > >
> > > 11. An on-line system will make it easier for (more) members to
> > > participate:
> > > I don't believe that for an instant! Right now, the way to
> > participate
> > > is by
> > > posting a new email or responding to a previous one. What can be
> > easier
> > > than
> > > that?
> > >
> > > 12. We really need facility xxx: Don't confuse *need* with *want*.
> > And
> > > ask
> > > yourself if *you* are willing to pay for its implementation.
> > >
> > > 13. In the 4 months assembla has been "on the table", no-one has
> > > objected to
> > > using it: Having been on standards' committees for 25 years, I've
> > found
> > > that
> > > if you ask members "Who is in favor of someone else doing some
> > work?",
> > > there
> > > are rarely objections. From that, one might conclude there is a
lot
> > of
> > > support for that work. However, if you ask the question, "Who is
in
> > > favor of
> > > going in some given direction if *they* had to fund their share of
> > the
> > > time/effort needed to implement it?", then you often get the
> complete
> > > opposite result. So, to make it quite clear, I'm objecting to any
> > sort
> > > of
> > > wholesale move to using assembla anytime soon.
> > >
> > >
> > >
> > > BTW, in more than 3 years, Ecma TC45 has received only 3 or 4
> emails
> > > from
> > > members of the public, asking about details in the standard. I
have
> > > received
> > > no correspondence from the public via JTC 1, and I don't recall
> > anyone
> > > within WG4 saying they were contacted by the public and asked to
> > submit
> > > any
> > > DRs on their behalf. In short, the level of direct technical
> inquiry
> > > from
> > > the public has been negligible. And think of all the implementers
> out
> > > there
> > > who are *not* directly represented within WG4, yet, presumably
need
> > to
> > > understand the standard; they seem to be getting along just fine
> > > without
> > > contacting us at all. So, who then is clamoring for more
> information?
> > > Murata
> > > already makes the DR Log available publically. I would say that
> most
> > > people
> > > involved in OOXML-related projects are more interested in the
> > published
> > > standard and published CORs and Amds, than in what WG4 is doing
> > > internally.
> > > Besides, pretty much anyone who *really* has a stake in 29500
> (rather
> > > than
> > > just "being interested") has a way to participate through their
> NBs,
> > a
> > > liaison, or as an invited guest. (I know, because as an
independent
> > > consultant I funded myself to participate for 15 years in the
> > ANSI/ISO
> > > standardization for the C language, which typically had 4.5-day
> face-
> > > to-face
> > > meetings 4 times per year.)
> > >
> > >
> > > Ok, let's move on to specific points raised by Murata-san (see my
> > > replies
> > > in-line):
> > >
> > >
> > > -----Original Message-----
> > > From: MURATA Makoto (FAMILY Given) [mailto:eb2m-mrt at asahi-
> net.or.jp]
> > > Sent: Monday, September 28, 2009 1:34 AM
> > > To: SC 34 WG4
> > > Subject: Re: Following up on my action item to give feedback on my
> DR
> > > Log V2
> > > Plan
> > >
> > > Dear colleagues,
> > >
> > > Here is my proposal.  I strongly advocate the use of Assembla.
> > >
> > > 1. Problems of the current approach
> > >
> > > 1.1 Schema maintenance problems
> > >
> > > 1) Different specfications (e.g., FPDAM1 and DCOR1) developed
> > >    in parallel share a single set of schemas.  What happens if
> > >    one of them is disapproved?
> > >
> > > Rex> This could be a problem. And as I stated above, I am not
> opposed
> > > to
> > > using assembla to help solve certain parts of the maintenance
> > problem.
> > > Note,
> > > however, in the DCOR1 and FPDAM1 sets (which closed 293 DRs),
there
> > was
> > > only
> > > one instance of overlap with respect to changes to the same lines
> of
> > > schema.
> > > In any event, the same issue exists with changes to the same
> sentence
> > > or
> > > paragraph by multiple DR resolutions. In both cases, when I
> produced
> > > the
> > > DCOR1 and FPDAM1 sets, I noticed this and made appropriate
> > adjustments.
> > >
> > >
> > > 2) WG4 members cannot obtain schema files and point out problems
> > >    (including syntactical errors) before schemas
> > >    are generated from change-tracked Word files.
> > >
> > > Rex> In my V2 proposal I suggested that we not actually mark a DR
> > > closed
> > > until its schema changes have been applied to the electronic
> version
> > > and
> > > validated. As to how this is done, I have no preference at this
> time.
> > >
> > >
> > > 3) RELAX NG schemas cannot be created before XSD schemas are
> > >    generated from change-tracked Word files.
> > >
> > > Rex> See previous response.
> > >
> > >
> > > 1.2 DR maintenance problems
> > >
> > > 1) The DR Log file is bulky and cumbersome.
> > >
> > > Rex> These are not meaningful characterizations. Please be
specific
> > > about
> > > how this impairs maintenance. Also, see my earlier response.
> > >
> > >
> > > 2) The task of maintaining the DR Log is a too siginifant
> > >    burden on the project editor(s), and thus the the DR Log
> > >    does not always contain all the relevant e-mails or attachment
> > > files.
> > >
> > > Rex> Perhaps you should have asked the project editor first before
> > > making
> > > such a bold statement! Let me state for the record that I have
> never
> > > claimed, not at any time do I recall having thought that was true.
> As
> > > to the
> > > possibility of there being missing information, what steps has
> anyone
> > > taken
> > > to notify me of that? I'm not aware of any substantive problem
> here.
> > >
> > >
> > > 3) Maitaining the consistency between the DR index (as an Excel
> file)
> > >    and the DR Log is also a siginifant burden on the project
> > editor(s).
> > >
> > > Rex> Once again, you are speaking on my behalf without checking
> with
> > > me.
> > > Because I started the spreadsheet Index well after I started the
DR
> > > Log,
> > > yes, it took a couple of hours of cutting and pasting to populate
> the
> > > sheet,
> > > but, now, it's trivial to create a new row for each DR I add to
the
> > > Log. In
> > > any event, my V2 proposal makes the generation of the Index
> > automatic.
> > >
> > >
> > > 4) Statistics are published only occasionally (by hand), and they
> are
> > >    not very flexible.  For example, we cannot count the number of
> > >    MIME-related DRs.
> > >
> > > Rex> As far as I know, other SCs and WGs generate statistics by
> hand
> > > too,
> > > since statistics are only needed for occasional summaries of the
> > status
> > > of
> > > work. In any case, you can take the DR Log PDF and search for a
> > string,
> > > such
> > > as "MIME", and find the number of occurrences. Although a rough
> > gauge,
> > > I
> > > expect it will be good enough for most purposes. I can't say that
> > I've
> > > had
> > > the need to produce any other statistics. What else do members
> > *need*,
> > > how
> > > often, and why?
> > >
> > >
> > > 1.3 DR submission problems
> > >
> > > When we developed the web form for submitting DRs, we thought that
> > > this will become by far the most important channel.  Apparently,
it
> > is
> > > not.  Since some editorial defects are best described as changes
in
> > > the DOCX file rather than filling out flat-text-only forms, some
> > > member bodies continue to use DOCX files for submitting DRs.  The
> web
> > > form does not cause any troubles, however.
> > >
> > > Rex> Correct, the web form does not cause any "trouble", but it
> does
> > > make
> > > work for me. However, let's be careful about the use of "we". I
> > argued
> > > against it from the start. It is only able to capture initial
> > > submissions,
> > > not track anything beyond that, and it has no support for rich
> text.
> > As
> > > for
> > > others using an alternate approach to DR submission, that's
because
> I
> > > have
> > > openly encouraged them to do so by providing them with a "blank"
> DOCX
> > > file
> > > for them to "fill in". 50% of the DRs come to me via that method,
> > which
> > > makes it trivial for me to integrate them into the DR Log. So, if
> > > others
> > > would submit their DRs that way too, that would reduce my
workload.
> > >
> > >
> > > 2. My proposal
> > >
> > > The proposal by Rex addresses 1), 3) and 4) in 1.2.  However,
> > > the other problems are not solved.  I would like to provide
> > > some additions and changes to the proposed plan.
> > >
> > > Rex> Since 2) was about the current system's burden on the project
> > > editor,
> > > and as the project editor I've explained that this was an
incorrect
> > > assumption, it appears that my proposal does, in fact, address all
> > the
> > > problems you identify.
> > >
> > >
> > > I would like to advocate the use of Assembla.  In particular,
> > > subversion can easily addresses 1), 2), and 3) in 1.1.
> > >
> > > 2.1 Schema maintenance
> > >
> > > Schemas should be maintained in the subversion repository of
> Assembla
> > > rather than changed-tracked MS-Word documents.
> > >
> > > Rex> So long as the schemas are printed in annexes, I *need* to
> > publish
> > > tracked changes in CORs and Amds to show how those annexes
contents
> > are
> > > changed. So, it is not an "either/or" situation; we need to deal
> with
> > > changes in the printed annex versions, and we need to deal with
the
> > > electronic versions. Maybe my current approach can be used for the
> > > annexes,
> > > and assembla for the xsd/rnc file versions.
> > >
> > >
> > > Schema files can be accessed via the web-based user interface of
> > > Assembla, but schema experts and the project editor will require
> > > front-end sytems (such as Tortoise) for schema authoring.
> > >
> > > Rex> At this time, I am open to how changes to the electronic
> version
> > > of the
> > > schemas are handled.
> > >
> > >
> > > - For each specification (AM or DCOR), we create a subversion
> > >   branch.  This branch contains those schema changes required for
> the
> > >   amendment or DCOR.  Note that different specifications developed
> in
> > >   parallel have different schemas.
> > >
> > > - Whenever a WG4 member proposes a schema change in reply to a DR,
> > >   that member should store the change into the subversion
> repository,
> > >   and should create a link to the DR at the same time.  This
> > >   action requires understanding of subversion and the front end
> > system.
> > >
> > > - Everybody can view schemas stored in the Assembla subversion
> > >   repository at any time.
> > >
> > > An issue in the subversion approach (which maintains schema files
> > > only) is the creation of changed-tracked Word files.  I believe
> that
> > > this issue can be addressed by using Word for comparing two DOCX
> > > documents.
> > >
> > > - Create a DOCX document containg old schemas.
> > >
> > > - Create another DOCX document containing new schemas for the
> > >   speficiation in question.
> > >
> > > - Use the file comparison feature of MS Word for generating
change-
> > >   tracked DOCX documents.
> > >
> > > - Incorporaate the change-tracked DOCX documents as part of the
> > >   (F)PDAM or DCOR.
> > >
> > > Rex> At a glance, this sounds like a non-trivial process. When
> > > considering
> > > Murata-san's proposal, keep in mind that of the 213 DRs we closed
> in
> > > the
> > > DCOR1/FPDAM1 sets, only 37 (that is, 17%) involved changes to
> schema.
> > > So, we
> > > need to keep in perspective how sophisticated a system we really
> > *need*
> > > to
> > > manage this. In any event, I don't recall any other members
raising
> > any
> > > concerns about their inability to do anything schema-related in
> terms
> > > of the
> > > DR Log machinery.
> > >
> > >
> > > 2.2 DR maintenance
> > >
> > > Although I strongly advocate the use of Assembla, we should
> > > not force the project editor to use the web-based user interface
> > > of Assembla for DR maintenance.   This is because he might
> > > not be always able to access Assembla and also because MS Office
> > > is sometimes more convenient than web-based user interfaces.
> > >
> > > - For each DR, the project editor creates a DR file as a DOCX
> > document,
> > >   as proposed by Rex.  Whenever the status of the DR changes, the
> > >   project editor revise the DR file.
> > >
> > > - The project editor uses a program for automatically creating an
> > >   Assembla ticket from the DR file.  Custom xml or smart tags
> > embedded
> > >   in the DR file allow data to be easily extracted.  The program
> also
> > >   store the DR file as an attachment file in Assembla.
> > >
> > > - The project editor uses another program for creating the DR Log
> > >   Index from the Assembla tickets (not from the DR files) as an
> XLSX
> > >   document.  Links from the DR Log Index to DR files use the URI
of
> > >   the Assembla repository as a base URI.  The DR Log Index also
> > >   contains the summaries and statistics on a new Sheet.
> > >
> > > - The project editor then stores the DR Log Index in three
formats,
> > >   namely XLSX, TSV, and PDF, as attachment files, and create links
> > from
> > >   the Assembla wiki.
> > >
> > > - Each WG4 member receives a notification mail for each newly-
> created
> > > DR.
> > >
> > > - WG4 members discuss about each DR by using the comment feature
of
> > >   Assembla.  All comments on DRs are stored in the Assembla
> > >   repository, and are accessible via the web-based Assembla user
> > >   interface.  Attachment files are also stored in Assembla.
> > >
> > > - If necessary, we develop a program for generating a single file
> > (zip?
> > > pdf?)
> > >   for each DR or all DRs.  This program would help those who do
not
> > > always
> > >   have access to the Internet.
> > >
> > > - As described in 2.1, schema changes required for any DR should
be
> > >   stored in the Assembla subversion repository, and ech commit
> action
> > >   should provide a link to the relevant DR.  Such links make it
> easy
> > >   to maintain the relationship between schema changes and DRs.
> > >
> > > The project editor will be freed from two works.
> > >
> > > - He does not have to copy all e-mails and attachment files to DR
> Log
> > > files.
> > >
> > > - He does not have to maintain the DR Log and the DR Log Index
> > >   in sync.
> > >
> > > Note: I do not have an opinion about the DR history log.  Is it
> hart
> > > to maintain?  I have almost never read the DR history log.
> > >
> > > 2.3 Milestones
> > >
> > > Whenever a new amendment project is created, we create an Assembla
> > > milestone and a branch in the subversion repository.  Whenever we
> > > start to create a DCOR, we do the same thing.
> > >
> > >
> > >
> > > Regards,
> > >
> > > SC34/WG4 Convenor
> > > MURATA Makoto (FAMILY Given)
> > >
> > >
> > > Rex> As far as I can tell, with respect to 29500 maintenance, the
> > > assembla
> > > solution currently being proposed appears to me largely to be a
> > > solution in
> > > search of a problem to solve. I believe that the use of it would
> make
> > > the
> > > project editor's work more time-consuming.  It might also reduce
> > > participation due the administrative and training overhead
> involved,
> > > and the
> > > time and resources needed to move to assembla would distract WG4
> from
> > > the
> > > more important work of processing defect reports.  WG4 has moved
> > > rapidly
> > > through a large number of DRs already, and I think we should build
> on
> > > that
> > > success.  I'm open to fine-tuning the process, but I don't believe
> > > anything
> > > has happened to justify a wholesale change to a radically
different
> > > approach.
> > >
> > > Rex Jaeschke
> > > 29500 Project Editor
> > >
> > >
> > >
> > >
> 
> 
>