Subject: Re: Atom syndication schema

From: David Powell <[email-masked]>
Date: Tue, 14 Mar 2006 23:42:03 +0000

Not sure if this is a known bug, but I just noticed that the RelaxNG
grammar doesn't accept "atomCommonAttributes" (eg xml:lang) on the
"atom:name" and "atom:uri" and "atom:email" elements used within
Person constructs.

-- 
Dave

---

From: Martin Duerst <[email-masked]>
Date: Wed, 15 Mar 2006 12:21:08 +0900

>"atom:name" and "atom:uri" and "atom:email" elements used within

For atom:uri and atom:email at least, not having xml:lang may
be seen as a feature. While these often contain pieces from one
language or another, they are not really in a language.

Regards,   Martin.

---

From: Eric Scheid <[email-masked]>
Date: Wed, 15 Mar 2006 16:25:40 +1100

> language or another, they are not really in a language.

Since the original discussion I've stumbled across something extra that
makes xml:lang relevant for atom:name.

Seems that in writing Hungarian names, the pattern is always surname
followed by forename - e.g. Bartók Béla, where Béla is the personal name and
Bartók is the family name.

While common western names (eg. Eric Scheid) would be indexed as Scheid,
Eric; a comma is instead simply added between the Hungarian surname and
forename, making Hungarian names indistinguishable from other Western-style
names. For example: Bartók Béla is indexed as Bartók, Béla.

Icelandic names are another game altogether.

e.

---

From: "M. David Peterson" <[email-masked]>
Date: Tue, 14 Mar 2006 22:36:36 -0700

For Latin-based languages, your point is well taken. For non-latin,
its all about character sets.  As long as your character set for any
given feed is properly set, it seems to me then all the information
necessary to properly decode the email and URI (in which the work
continues to  integrate support for non-latin based languages, such as
Mandarin, etc... if I understand things correctly, full support for
Mandarin Chinese-based domains in not far off (speaking in terms of
DNS support and such).

Actually, the only reason for writing this response was to point out
the fact that we are entering a world in which China will continue to
play a dominant role in both our online and offline worlds, so
beginning to learn as much as possible in terms of how to properly
handle URI's and email adresses encoded as mentioned seems like it
would be a pretty good idea.

Couldnt hurt. :)

--
<M:D/>

M. David Peterson
http://www.xsltblog.com/

---

From: "M. David Peterson" <[email-masked]>
Date: Tue, 14 Mar 2006 22:51:01 -0700

Hmmm... some interesting points you've brought out.

Q: Does anyone know how many of the languages that fall into a similar
category, can be determined by character set alone?  I realize that
theres no  way that every single case could be covered using the
character set alone, and as such realize this does present an area of
considerable thought.

Unless, of course, someone already knows the answer, or better said,
how does one properly handle the various situations where te character
set in and of itself doesn't provide enough information.  Can a parent
element with an xml:lang attribute be enough?  It seems that in the
case of name, email and uri, the containing author element, which as
long as I'm not mistaken, does allow xml:lang should be enough to make
the assumption that the children elements also whould be treated as
the same language specified in the value of this attribute.

The only element (I think) that might be of concern is the name
element, as the email and URI should be handled with the character
encoding.

Does any of this even sound remotely on target?

--
<M:D/>

M. David Peterson
http://www.xsltblog.com/

---

From: Walter Underwood <[email-masked]>
Date: Tue, 14 Mar 2006 22:23:55 -0800

> Bartók is the family name.

Or Margittai Neumann János vs. John von Neumann. It can be more complicated
than first/last or last/first.

I'm pretty sure that I brought this up and the WG decided to punt.

Representing personal names well means starting with X.500 and asking
around to see what could be improved. That is well outside the Atom charter.
Punting was the right thing to do, but it means that atom:name is minimal.

xml:lang isn't enough information to sort out given name and family name.
About all you can do with atom:name is print it out.

xml:lang could be useful in deciding between Chinese and Japanese variants
of a character for names. 

wunder
--
Walter Underwood
Principal Software Architect, Autonomy

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Wed, 15 Mar 2006 09:33:49 +0100

On Tue, Mar 14, 2006 at 10:23:55PM -0800,
 Walter Underwood <[email-masked]> wrote 

> xml:lang isn't enough information to sort out given name and family

Yes! In many countries, the order changed. In France, one century ago
"Family-name Given-name" was common in official papers but is now
deprecated. In Algeria, a former french colony, the two usages still
prevail ("Matoub Lounes" or "Lounes Matoub"?)

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Wed, 15 Mar 2006 10:01:30 +0100

On Tue, Mar 14, 2006 at 10:36:36PM -0700,
 M. David Peterson <[email-masked]> wrote 
 a message of 43 lines which said:

> As long as your character set for any given feed is properly set, it
> seems to me then all the information necessary to properly decode
> the email and URI (in which the work continues to integrate support
> for non-latin based languages, such as Mandarin, etc...

Just to be pedantic, URIs (RFC 3986) are in pure US-ASCII. IRIs (RFC
3987) are in Unicode and are accepted by Atom (so, Atom's URIs seem to
be actually IRIs). The standard says:

   # Unconstrained; it's not entirely clear how IRI fit into
   # xsd:anyURI so let's not try to constrain it here
   atomUri = text

> if I understand things correctly, full support for Mandarin
> Chinese-based domains in not far off (speaking in terms of DNS
> support and such).

It is quite old, RFC 3490 (issued three years ago and implemented even
before).

> email adresses encoded as mentioned 

There is not yet any standard for Unicode email addresses (work is
going on, see the very recent IETF Working Group EAI
http://www.ietf.org/html.charters/eai-charter.html).

---

From: "M. David Peterson" <[email-masked]>
Date: Wed, 15 Mar 2006 03:21:46 -0700

Excellent!  Thanks for the info :)

I will have to go back and see what exactly the article I read was
refering to, but if I remember correctly they seemed to mention that
support for the top level domains, specifically .com, .net, and .cn
had not been implemented using the Mandarin character set.

Either way, obviously there are a lot of considerations that need to
be allowed for when developing from an interanational perspective,
something that should be part of the defaul process instead of the
exception... of course, I say this mainly because this is really the
first time I have put any real thought into this area, and obviously I
need to put a lot more.

Thanks again for clearing things up and providing the links!

--
<M:D/>

M. David Peterson
http://www.xsltblog.com/

---

From: David Powell <[email-masked]>
Date: Wed, 15 Mar 2006 12:19:52 +0000

Wednesday, March 15, 2006, 3:21:08 AM, Martin Duerst wrote:

> be seen as a feature.

The spec says that "Any element defined by this specification MAY have
an xml:lang attribute". We chose to limit the effects of xml:lang,
rather than the occurrence of it. Eg: atom:published is allowed
xml:lang, even though it is meaningless. The spec includes a sentence
about element xxx being "Language-Sensitive" when we consider the
language to be relevant. The idea is, if a feed reading framework such
as Microsoft's Windows/IE7 feed platform doesn't preserve xml:lang on
elements that aren't "Language-Sensitive", then they are doing nothing
wrong. Same for, eg: an Atom publishing server backed by a legacy CMS.

> While these often contain pieces from one language or another, they
> are not really in a language.

I agree. Note that this is the case in Atom, because those two
elements are not "Language-Sensitive".

Also note, that atom:uri is an IRI-reference, so it is affected by any
xml:base attributes on that element.

And that atomCommonAttributes also covers extension attributes, which
are also allowed anywhere. They are "undefined", which *I* think means
that implementations need not feel bad about dropping them on the
floor. The official meaning is, er, undefined.

-- 
Dave

---

From: "M. David Peterson" <[email-masked]>
Date: Wed, 15 Mar 2006 05:37:32 -0700

> Also note, that atom:uri is an IRI-reference, so it is affected by any
xml:base attributes on that element. <

Until now, I had no idea this was the case... WOW!!! Amazing the
things you can learn around people who know what theyre talking about.
:)

--
<M:D/>

M. David Peterson
http://www.xsltblog.com/

---

From: "Thomas Broyer" <[email-masked]>
Date: Wed, 15 Mar 2006 14:36:19 +0100

> be actually IRIs). The standard says:

Well, not really the "standard" actually, since the RNC is not normative...

>    # xsd:anyURI so let's not try to constrain it here

RFC 3987 says (section 1.2 Applicability):
       For example, XML schema [XMLSchema] has an explicit type
       "anyURI" that includes IRIs and IRI references. Therefore, IRIs
       and IRI references can be in attributes and elements of type
       "anyURI".

So, actually, it seems that the Atom RNC could say "atomUri = xs:anyURI".

...or RFC 3987 is wrong... (I didn't check XMLSchema to try to figure
it out myself)

--
Thomas Broyer

---

From: Norman Walsh <[email-masked]>
Date: Thu, 16 Mar 2006 10:42:53 -0500

/ "Thomas Broyer" <[email-masked]> was heard to say:
| RFC 3987 says (section 1.2 Applicability):
|        For example, XML schema [XMLSchema] has an explicit type
|        "anyURI" that includes IRIs and IRI references. Therefore, IRIs
|        and IRI references can be in attributes and elements of type
|        "anyURI".
| So, actually, it seems that the Atom RNC could say "atomUri = xs:anyURI".

Yes, I think that's the case.

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <[email-masked]> | Science is organized knowledge. Wisdom
http://nwalsh.com/            | is organized life.--Immanuel Kant

---

From: Uche Ogbuji <[email-masked]>
Date: Thu, 16 Mar 2006 12:31:08 -0700

> "atom:name" and "atom:uri" and "atom:email" elements used within

Did you cc me because of my coverage of the matter?

http://copia.ogbuji.net/blog/2006-02-06/Small_fix_

If so, I think I said all I have to say about it there.  My fixed RNG is 
still available.

-- 
Uche Ogbuji                               Fourthought, Inc.
http://uche.ogbuji.net                    http://fourthought.com
http://copia.ogbuji.net                   http://4Suite.org
Articles: http://uche.ogbuji.net/tech/publications/

---

From: David Powell <[email-masked]>
Date: Thu, 16 Mar 2006 21:41:56 +0000

> If so, I think I said all I have to say about it there.  My fixed RNG is

Er, I hadn't read the thread properly, and posted to it having
independently discovered the same bug when I was doing some hard-core
XSLT-ing of Atom.  Doh.

Could you post the errata to the rfc-editor, via:
http://www.rfc-editor.org/errata.html

-- 
Dave

---

From: Martin Duerst <[email-masked]>
Date: Fri, 17 Mar 2006 17:41:22 +0900

>Yes, I think that's the case.

Some details (from the primary editor of RFC 3987 :-):
 From a mile high viewpoint, and even from much lower,
it is definitely the case. From the viewpoint of intent, it is
also definitely the case.

When looking with a microscope, you will find some little
differences, because xs:anyURI was described before the IRI
spec (RFC 3987) was approved. These differences are:

1) xs:aryURI also allows spaces and a few other ASCII characters
    that are not allowed in URIs nor in IRIs (but the IRI spec has
    an escape hatch for such cases).
2) The IRI spec contains many more details than the xs:anyURI
    description, in particular also some requirements re.
    normalization. However, some of the requirements in this
    area of the IRI spec may be lowered or removed in the future
    because we have received feedback from implementers that
    there are difficulties to implement these.

Regards,    Martin.

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 17 Mar 2006 10:49:54 +0100

>    there are difficulties to implement these.

I agree with Martin that it would be incorrect to use xsd:anyURI here.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Martin Duerst <[email-masked]>
Date: Sun, 19 Mar 2006 16:33:32 +0900

>I agree with Martin that it would be incorrect to use xsd:anyURI here.

Sorry, but I never said that it would be incorrect to use
xsd:anyURI. I personally think that it should be okay to
use xsd:anyURI. The differences are microscopic, and they should
become even smaller, or hopefully go away completely, over time.
It does not make sense to perpetuate minor differences for
something that was and is supposed to be one and the same
thing.

Regards,    Martin.

---

From: Henri Sivonen <[email-masked]>
Date: Sun, 19 Mar 2006 17:40:33 +0200

(Discussion started on atom-syntax, but this is a more general RELAX  
NG issue, so cross-posting to rng-users.)

> become even smaller, or hopefully go away completely, over time.

I need datatypes for IRIs in general (relative, absolute or just  
fragment identifiers) and for absolute IRIs (possibly with a fragment  
id) in a RELAX NG schema.

Is it really the best practice to use xsd:anyURI and sweep the  
discrepancies under the rug in the hope that future definitions of  
xsd:anyURI change the meaning of the schema later? Can xsd:anyURI be  
augmented with a regexp pattern to restrict "spaces and a few other  
ASCII characters" in such a way that the resulting datatype  
restriction matches the definition of IRI? Has anyone implemented a  
strictly correct IRI datatype in a Java datatype library (for Jing  
and MSV)?

-- 
Henri Sivonen
[email-masked]
http://hsivonen.iki.fi/

---

From: John Cowan <[email-masked]>
Date: Sun, 19 Mar 2006 10:52:26 -0500

> strictly correct IRI datatype in a Java datatype library (for Jing  

It's certainly possible to construct a regular expression, a long and complex
one, that will match all IRIs and only IRIs (note that "IRI" by itself
means "absolute IRI with or without fragment identifier").  The question
is whether it's really worth doing so.  If you feel you need it,
by all means go ahead.

-- 
LEAR: Dost thou call me fool, boy?      John Cowan
FOOL: All thy other titles              http://www.ccil.org/~cowan
             thou hast given away:      [email-masked]
      That thou wast born with.         http://www.ap.org

---

From: Elliotte Harold <[email-masked]>
Date: Sun, 19 Mar 2006 12:08:21 -0500

I would recommend against using xsd:anyURI for IRIs. A URI is much more 
restrictive than an IRI, and one of the easiest things for a schema 
validator to check about an xsd:anyURI is that it only contains 
URI-legal ASCII characters. I think a new type is necessary if you do 
want to allow IRIs instead of simple URIs. I suspect you could do it 
with a regular expression but the syntax would be really hairy.

-- 
Elliotte Rusty Harold  [email-masked]
XML in a Nutshell 3rd Edition Just Published!
http://www.cafeconleche.org/books/xian3/
http://www.amazon.com/exec/obidos/ISBN=0596007647/cafeaulaitA/ref=nosim

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sun, 19 Mar 2006 18:30:18 +0100

>with a regular expression but the syntax would be really hairy.

In Schema 1.1 it is not possible for a xsd:string to be no xsd:anyURI.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Martin Duerst <[email-masked]>
Date: Tue, 21 Mar 2006 15:34:54 +0900

restrictive than an IRI, and one of the easiest things for a schema 
validator to check about an xsd:anyURI is that it only contains URI-legal 
ASCII characters.

This is indeed one of the easiest things, but it would be TOTALLY
wrong.

http://www.w3.org/TR/xmlschema-2/datatypes.html#anyURI says, among else:

    The mapping from anyURI values to URIs is as defined by the URI reference
    escaping procedure defined in Section 5.4 Locator Attribute of [XML
    Linking Language] (see also Section 8 Character Encoding in URI References
    of [Character Model]). This means that a wide range of internationalized
    resource identifiers can be specified when an anyURI is called for, and
    still be understood as URIs per [RFC 2396], as amended by [RFC 2732],
    where appropriate to identify resources.

If there is confusion in other venues about this issue, please help
to make sure it gets fixed.

Regards,    Martin.

---

From: Martin Duerst <[email-masked]>
Date: Tue, 21 Mar 2006 17:07:08 +0900

>In Schema 1.1 it is not possible for a xsd:string to be no xsd:anyURI.

Can you explain? It seems you are saying that all xsd:strings are
also xsd:anyURIs, but that seems going a bit too far.

Regards,    Martin.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 21 Mar 2006 10:29:43 +0100

Martin Duerst wrote:
>  >I would recommend against using xsd:anyURI for IRIs. A URI is much 
> more restrictive than an IRI, and one of the easiest things for a schema 
> URI-legal ASCII characters.
> 

Well,

maybe it's time that *some* specification adds new datatypes that do 
*exactly* what RFC3986 and RFC3987 ask for :-)

Best regards, Julian

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 21 Mar 2006 13:23:13 +0100

>also xsd:anyURIs, but that seems going a bit too far.

Yes, that's exactly what the XML Schema 1.1 Last Call Working Draft
implies as far as I can tell.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/