DR-16-0018: WML: need sort method for special characters [for today's call]

Tue Dec 20 17:09:41 CET 2016

HI Keld

My main concern is whether the sort method, if specified, affects what the
user sees when they open a document. Suppose that, for the sake of argument,
the default sort method of implementation A sorts accented text according to
the "normal" approach (first accent difference determines the order), while
implementation B sorts accented text according to the "French" approach
(last accent difference determines the order). Suppose a spreadsheet is
created by implementation A and has an auto-filter applied that sorts
according to its default sort method.  If this spreadsheet is subsequently
opened by implementation B, will it appear the same as if it was re-opened
by implementation A, or will implementation B's default sort method be
automatically applied? My guess is that generally the latter will be the
case, but this is obviously implementation-dependent. 

I suspect that this is a case where implementations must be free to choose
alternative approaches that aren't fully interoperable. Both implementations
A and B in my example will have had good reasons for choosing different sort
methods, e.g. based upon market demands.

It might be nice if the sort method were spelt out in the document, but this
would definitely be a new feature. For now, in response to DR 16-0018, I
think we should simply ensure that the specification is consistent with the
schema and is clear about what are meant by the existing values of
ST_SortMethod, i.e. 'none', 'pinYin' and 'stroke'.

Interestingly, I note that the content model of sortState includes the
application-defined extension element extLst, which could be used to specify
the sort state in more detail, using MCE. So, in theory, it would be
possible to define an extension to OOXML without having to amend the base
standard. However, I'm not sure that there'd be sufficient demand for this.

Kind regards,

Francis

-----Original Message-----
From: keld at keldix.com [mailto:keld at keldix.com] 
Sent: 20 December 2016 09:37
To: Francis Cave <francis at franciscave.com>
Cc: 'MURATA Makoto' <eb2m-mrt at asahi-net.or.jp>; 'SC 34 WG4'
<e-SC34-WG4 at ecma-international.org>
Subject: Re: DR-16-0018: WML: need sort method for special characters [for
today's call]

Hi Francis

Yes, I understand that new requirements need an amendment or revision.

However, this is not what I recommend as the immediate solution to sorting
special characters. I just propose to use the ISO 14651/UTS#10 kind of
default spec.
And no locale choice. 

Best regards
keld

On Mon, Dec 19, 2016 at 05:27:25PM -0000, Francis Cave wrote:
> 
> Try again...
> 
> In the first paragraph, for "cannot be done" read "can only be done".
> Clearly a serious finger malfunction...
> 
> Francis
> 
> 
> 
> -----Original Message-----
> From: Francis Cave [mailto:francis at franciscave.com]
> Sent: 19 December 2016 16:41
> To: keld at keldix.com; 'MURATA Makoto' <eb2m-mrt at asahi-net.or.jp>
> Cc: 'SC 34 WG4' <e-SC34-WG4 at ecma-international.org>
> Subject: RE: DR-16-0018: WML: need sort method for special characters 
> [for today's call]
> 
> 
> In the first paragraph or "cannot be done" ready "can only be done".
Sorry!
> 
> Francis
> 
> 
> 
> -----Original Message-----
> From: Francis Cave [mailto:francis at franciscave.com]
> Sent: 19 December 2016 16:33
> To: keld at keldix.com; 'MURATA Makoto' <eb2m-mrt at asahi-net.or.jp>
> Cc: 'SC 34 WG4' <e-SC34-WG4 at ecma-international.org>
> Subject: RE: DR-16-0018: WML: need sort method for special characters 
> [for today's call]
> 
> 
> Keld
> 
> You may or may not be correct in your view that there is a user 
> requirement that fields be sortable according to the current locale. 
> However, if there is such a user requirement, it isn't currently met 
> by OOXML or by implementations. Support for specifying the current 
> locale as the sort method would involve an extension to OOXML and as 
> such cannot be done by amendment or revision of the standard, not by 
> Technical Corrigendum, so this is a big deal. Although this might be a 
> relatively simple change, e.g. by changing ST_SortMethod to allow any 
> string (§18.18.73 of ISO/IEC 29500-1:2016), it could not be made 
> mandatory for implementations to support these new values without 
> breaking existing implementations, and if they do not already do so, 
> that suggests that there hasn't be much market pressure that would
persuade them to implement the enhancement.
> 
> I note that ODF doesn't appear to have this feature either (although 
> it has a similar feature – §19.865 text:sort-algorithm – for sorting text,
e.g.
> bibliographies), and in §19.685 table:order there is the following note:
> 
> 	Note: Sorting is locale and implementation-dependent.
> 
> It is hard to avoid the conclusion that demand for this feature is 
> very limited, at least in office document applications.
> 
> Francis
> 
> 
> 
> -----Original Message-----
> From: keld at keldix.com [mailto:keld at keldix.com]
> Sent: 18 December 2016 20:19
> To: MURATA Makoto <eb2m-mrt at asahi-net.or.jp>
> Cc: SC 34 WG4 <e-SC34-WG4 at ecma-international.org>
> Subject: Re: DR-16-0018: WML: need sort method for special characters 
> [for today's call]
> 
> I am not aware of the reason why this is so.
> 
> But anyway, why not then use the 14651 tailorable ordering in its 
> template form, which is equivalent to UTS#10 - as the universal sorting in
OOXML?
> 
> I would think it was a user requirement that fields are sortable 
> according to the current locale, eg a list of names.
> 
> Having the sorting order not being changeable creates troubles for 
> users too.
> 
> best regards
> keld
> 
> 
> On Mon, Dec 19, 2016 at 04:48:21AM +0900, MURATA Makoto wrote:
> > The sort order of Excel cannot be changed without causing troubles 
> > to users.
> > 
> > Regards,
> > Makoto
> > 
> > 2016-12-08 3:17 GMT+09:00 Keld Simonsen <keld at keldix.com>:
> > 
> > > I propose that you use the locale of the current process, and the 
> > > implied sorting sequence for special cheracters there, Or at least 
> > > the sorting specified in ISO/IEC
> > > 14651
> > > or the equivalent Unicode specifcation.
> > >
> > > best regards
> > > keld Simonsen
> > >
> > 
> > 
> > 
> > --
> > 
> > Praying for the victims of the Japan Tohoku earthquake
> > 
> > Makoto
> 
> 
> 
> 
> 