Subject: Re: partial xml in atom:content ?

From: James M Snell <[email-masked]>
Date: Sun, 15 Jan 2006 16:13:51 -0800

per rfc4287 section 4.1.3.3

    4.  If the value of "type" is an XML media type [RFC3023] or ends
        with "+xml" or "/xml" (case insensitive), the content of
        atom:content MAY include child elements and SHOULD be suitable
        for handling as the indicated media type.  If the "src" attribute
        is not provided, this would normally mean that the "atom:content"
        element would contain a single child element that would serve as
        the root element of the XML document of the indicated type.

given the last sentence in the snippet above, the answer is yes, it is 
valid.

Eric Scheid wrote:
> Is this a valid atom entry?
> 
>     <entry>
>         [...elided...]
>         <summary>a snippet of foo xml</summary>
>         <content type="application/foo+xml">
>             <foo:thing xmlns:foo="http://xmlns.com/foo/0.1/">
>                 <foo:name>King George</foo:name>
>             </foo:Person>
>         </content>
>     </entry>
> 
> That is, is a partial xml document valid inside the atom:content element?
> What about xml formats whose specs state that there must be a certain root
> element (similar to how atom documents must have either <feed> or <entry> as
> root elements)?
> 
> e.
> 
>

---

From: Eric Scheid <[email-masked]>
Date: Mon, 16 Jan 2006 11:04:20 +1100

<entry>
        [...elided...]

e.

---

From: "James Holderness" <[email-masked]>
Date: Mon, 16 Jan 2006 00:53:29 -0000

>                <foo:name>King George</foo:name>

I'm not sure if I misunderstood your question, or the mismatched tags are a 
typo, but that's not even valid XML let alone valid atom.

> That is, is a partial xml document valid inside the atom:content element?

Assuming that foo:Person close tag was meant to be a foo:thing close tag, 
then if you can extract the section from <foo:thing> to </foo:thing> into a 
file and view it successfully in a foo file viewer then I'd say it's a valid 
Atom entry (assuming also that the foo file viewer only opens strictly valid 
foo documents).

> element (similar to how atom documents must have either <feed> or <entry> 

To be honest, I didn't know you could get XML formats that didn't have an 
explicit root element. Nevertheless I think the same tests apply. Can the 
embedded XML section open successfully in a foo file viewer?

A more concrete example would be the use of application/xhtml+xml content 
starting with a div tag rather than an html tag.

   <entry>
        [...elided...]
        <summary>a snippet of xhtml</summary>
        <content type="application/xhtml+xml">
            <xh:div xmlns:xh="http://www.w3.org/1999/xhtml">
                <xh:p>Not strictly valid Atom content.</xh:p>
            </xh:div>
        </content>
    </entry>

While I'd expect most Atom clients to process such an entry successfully 
(assuming they're capable of handling XML at all) I don't think it's 
strictly valid Atom since you couldn't save that section to a file and 
expect it to open successfully in a validating XHTML file viewer.

Regards
James

---

From: Elliotte Harold <[email-masked]>
Date: Sun, 15 Jan 2006 20:40:12 -0500

>                 <foo:name>King George</foo:name>

No. It's not even well-formed much less valid.

-- 
Elliotte Rusty Harold  [email-masked]
XML in a Nutshell 3rd Edition Just Published!
http://www.cafeconleche.org/books/xian3/
http://www.amazon.com/exec/obidos/ISBN=0596007647/cafeaulaitA/ref=nosim

---

From: James M Snell <[email-masked]>
Date: Sun, 15 Jan 2006 18:07:14 -0800

> No. It's not even well-formed much less valid.

I personally assumed that the </foo:Person> was an unintended error.. 
after accounting for such, the example is valid.

- James

---

From: Eric Scheid <[email-masked]>
Date: Mon, 16 Jan 2006 13:17:05 +1100

> No. It's not even well-formed much less valid.

ignore the typo.

e.

---

From: Eric Scheid <[email-masked]>
Date: Mon, 16 Jan 2006 13:32:57 +1100

> after accounting for such, the example is valid.

Yes, that was a typo.

But I'm not so sure it's valid now, because of the SHOULD clause below:

Assume foo xml requires foo:basket as the root element. Is it valid to have
atom:content with foo:thing as the immediate child?

For example, an OPML document requires <opml> as the root, which contains
<head> and <body>, the latter containing <outline> elements ... so can an
atom entry contain just an <opml:outline> element?

  <entry>
      ...
      <entry type="whatever OPML's type is">
        <opml:outline xmlns:opml="whatever.." ... />
      </entry>
  </entry>

Thus, can atom be used to ship around parcels of xml snippets? I suppose it
could, but only so long as both ends knew what was going on, and knew naïve
atom processors might barf on the incomplete xml, right?

e.

---

From: James M Snell <[email-masked]>
Date: Sun, 15 Jan 2006 18:57:43 -0800

Eric Scheid wrote:
>  [snip]

Hey, now you're changing the scenario ;-) Item #4 of 4.1.3.3 is quite 
clear to me.  Specifically, "If the "src" attribute is not provided, 
this would normally mean that the "atom:content" element would contain a 
single child element that would serve as the root element of the XML 
document of the indicated type"

To me that says that while the example you give may be perfectly valid 
Atom, it may not be a valid instance of the application/foo+xml media type.

>         <opml:outline xmlns:opml="whatever.." ... />

Heh.. another typo methinks ;-)

Given that OPML has no media type, the XML media type would be 
appropriate.  Regardless, this would be perfectly valid Atom even if 
it's not perfectly valid OPML.

    <entry>
        ...
        <content type="application/xml">
          <opml:outline xmlns:opml="whatever.." ... />
        </content>
    </entry>

> atom processors might barf on the incomplete xml, right?

It's not really about "incomplete" XML as much as invalid XML ( in the 
sense that the xml does not validate to a DTD or schema ).  Atom does 
not require that content be valid, just that it be well-formed.

- James

---

From: "James Holderness" <[email-masked]>
Date: Mon, 16 Jan 2006 03:09:54 -0000

> But I'm not so sure it's valid now, because of the SHOULD clause below:

I suppose technically it is valid since a SHOULD is a recommendation not a 
requirement, but you'd need a very good reason for not following that 
recommendation.

>        <opml:outline xmlns:opml="whatever.." ... />

As I said above, it's probably technically valid, but not advisable. When 
you think about it, the most obvious behaviour for an Atom processor 
encountering a document type that it doesn't understand would be to write it 
to a file and fire up whatever application has been registered with the 
operating system for handling that type. In most cases that will only work 
if the file is complete.

> atom processors might barf on the incomplete xml, right?

The one time I'd think it might be safe is with XHTML (as I mentioned in a 
previous message) since Atom processors are already required to handle XHTML 
fragments in the content element. Anything else would be highly risky unless 
it was a proprietary feed communicating between two known applications.

Regards
James

---

From: Eric Scheid <[email-masked]>
Date: Mon, 16 Jan 2006 15:09:27 +1100

> Heh.. another typo methinks ;-)

grrr! TooEarly - SufficientCoffee = Typos

e.

---

From: Eric Scheid <[email-masked]>
Date: Mon, 16 Jan 2006 15:14:03 +1100

> it was a proprietary feed communicating between two known applications.

then you're not going to like what I was thinking of doing ... posting
<atom:category /> chunks to an APP/media collection, such that the media
collection entry might look like this (no typos this time, I hope)...

<entry>
    ...
    <content type="application/atom+xml">
        <category .../>
    </content>
</entry>

Thus, in the same sense I might have elsewhere a collection of images, I
have here a collection of atom categories.

I can hear the screaming already.

e.

---

From: "A. Pagaltzis" <[email-masked]>
Date: Mon, 16 Jan 2006 05:46:36 +0100

Hi Eric,

* Eric Scheid <[email-masked]> [2006-01-16 05:20]:
>then you're not going to like what I was thinking of doing ...
>posting <atom:category /> chunks to an APP/media collection,
>such that the media collection entry might look like this (no
>typos this time, I hope)...
>
><entry>
>    ...
>Thus, in the same sense I might have elsewhere a collection of
>images, I have here a collection of atom categories.

That doesn’t even work, because this is invalid:

    <entry>
        <!-- ... -->

An atom:content with an XML media type MUST have exactly one
child element, so you can’t do that. You need to wrap those
categories in something. The manifest choice is atom:entry…

Which just so happens to be a valid root element for
`application/atom+xml`. Of course, you’re pulling in an entire
train of metadata elements at that point, which will probably be
completely redundant, so whether that is any help I don’t know.

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>

---

From: Eric Scheid <[email-masked]>
Date: Mon, 16 Jan 2006 17:05:16 +1100

On 16/1/06 3:46 PM, "A. Pagaltzis" <[email-masked]> wrote:

> That doesn¹t even work, because this is invalid:
> 
>   <entry>
>       <!-- ... -->

No, that isn't one entry in a collection of atom categories, that is one
entry in a collection of *bundles* of atom categories. That's as wrong
headed as trying to squeeze *three* images into the one atom media entry.

If I have three categories in the collection, then they would be in three
individual entries.

> child element, so you can¹t do that. You need to wrap those
> categories in something. The manifest choice is atom:entryŠ

If I have a collection of atom:categories, the atomic unit is one
atom:category. So having a collection where each entry has *one*
atom:category in the atom:content element is what I would be trying to do.

If I wanted to have a collection of _bundles_ of atom:categories, I'd
probably need some foreign element to act as the bundling element (eg.
<foo:categories>)

> `application/atom+xml`. Of course, you¹re pulling in an entire
> completely redundant, so whether that is any help I don¹t know.

I don't want that train of metadata. I already have an atom:entry with which
to carry metadata for that content. (Heh, I could even apply atom:categories
to that content, same as any other content ;-)

The last niggling concern is with atom-format: although atom:content can
contain XML, can that be *any* XML data, or does the spec forbid atom+xml
from there? What will the validator do?

e.

---

From: Graham Parks <[email-masked]>
Date: Mon, 16 Jan 2006 06:29:48 +0000

On 16 Jan 2006, at 3:09 am, James Holderness wrote:

> The one time I'd think it might be safe is with XHTML (as I  
> mentioned in a previous message) since Atom processors are already  
> required to handle XHTML fragments in the content element. Anything  
> else would be highly risky unless it was a proprietary feed  
> communicating between two known applications.

Except no one bothered to tell the aggregator writers they'd have to  
implement this, so no it's not safe.

Graham

---

From: "A. Pagaltzis" <[email-masked]>
Date: Mon, 16 Jan 2006 07:50:02 +0100

Hi Eric,

>    <content type="application/atom+xml">

okay, another thrust, taking into account the things you said in
your reply to the first thrust: is anything wrong with this?

    <entry>
        <!-- ... -->

This is valid Atom.

(I get a growing feeling that I should really get on the APP
list…)

Regards,

---

From: Graham Parks <[email-masked]>
Date: Mon, 16 Jan 2006 06:57:39 +0000

> your reply to the first thrust: is anything wrong with this?

Two options:
1) Valid but completely meaningless, since category Atom documents  
aren't defined anywhere.
2) Invalid, since category Atom documents aren't defined anywhere.

Graham

---

From: Manoj <[email-masked]>
Date: Mon, 16 Jan 2006 12:27:58 +0530

I aggree to this point

-Manoj

On Mon, 16 Jan 2006 11:59:48 +0530, Graham Parks  
<[email-masked]> wrote:

>> The one time I'd think it might be safe is with XHTML (as I mentioned  
>> in a previous message) since Atom processors are already required to  
>> handle XHTML fragments in the content element. Anything else would be  
>> highly risky unless it was a proprietary feed communicating between two  
>> known applications.
>

---

From: Eric Scheid <[email-masked]>
Date: Mon, 16 Jan 2006 19:55:58 +1100

> implement this, so no it's not safe.

so atom is good for schlepping documents about, but not for elements of
documents (with two exceptions) :-(

e.

---

From: "James Holderness" <[email-masked]>
Date: Mon, 16 Jan 2006 15:21:06 -0000

>    <content type="application/atom+xml">

Now that we're talking about APP things are slightly different. I wouldn't 
necessarily expect an Atom aggregator to be able to handle APP media feeds 
in a useful way, so if APP decided to do something wierd with atom:content 
elements I wouldn't be particularly concerned as long as it were documented 
well enough that APP clients would know how to handle it.

However, as it stands this wouldn't be a valid media collection according to 
the APP spec since the media resource is supposed to be stored external to 
the collection feed and referenced via a content src link. Even then, I'd 
think you'd need a different type to signfy that the content was in fact a 
collection element and not an Atom document.

Regards
James

---

From: Antone Roundy <[email-masked]>
Date: Mon, 16 Jan 2006 08:32:20 -0700

> communicating between two known applications.

Processing type="xhtml" and type="application/xhtml+xml" are very  
different beasts.  Say your application converts Atom feeds to HTML  
to display in webpages.  With type="xhtml", the data could just be  
dumped into the webpage (after appropriate stripping of nasty tags  
and CSS and such).  With type="application/xhtml+xml", you'd have to  
figure out to do with everything outside of the <body> element.  If  
there's CSS involved for example, simply throwing it away could lead  
to some very messed up display.  But assuming your application is  
being called from within the webpage, it's not going to have the  
opportunity to add a <style> section to the document's <head>.  So to  
avoid losing the styling, for example, it would have to replace all  
id="foo" and class="bar" attributes with style="all of the styling  
for the id and class and parent classes, etc., with all cascading  
applied".  In other words, it's not going to happen.  Given the  
tremendously increased complexity involved, some apps are likely to  
refuse to process anything that's not one of Atom's three special types.

---

From: Eric Scheid <[email-masked]>
Date: Tue, 17 Jan 2006 02:33:34 +1100

On 17/1/06 2:21 AM, "James Holderness" <[email-masked]> wrote:

> the media resource is supposed to be stored external to
> the collection feed and referenced via a content src link

hmmm ... premature optimisation due to common use case of binary content?

e.

---

From: "A. Pagaltzis" <[email-masked]>
Date: Mon, 16 Jan 2006 16:53:52 +0100

* Antone Roundy <[email-masked]> [2006-01-16 16:40]:
>With type="xhtml", the data could just be  dumped into the
>webpage (after appropriate stripping of nasty tags  and CSS and
>such).  With type="application/xhtml+xml", you'd have to  figure
>out to do with everything outside of the <body> element.

I figure the canonical approach is going to be an iframe.

Regards,

---

From: "James Holderness" <[email-masked]>
Date: Mon, 16 Jan 2006 16:59:39 -0000

Antone Roundy wrote:
> different beasts.

In theory, yes. In practice, no. Bare in mind that the 0.3 Atom spec had 
type="application/xhtml+xml" with basically the same functionality as the 
current type="xhtml". Now since Atom 0.3 is still a whole lot more widely 
used than Atom 1.0, you can be fairly sure than anyone that bothers to 
handle XML content at all will be capable of handling 
type="application/xhtml+xml" containing xhtml fragments.

It's possible that an aggregator might specifically check for version 1.0 of 
Atom and choose to process type="application/xhtml+xml" in a different way, 
but that is extremely unlikely. Every Atom aggregator that I've tested that 
was capable of processing XML content (admittedly not that many) has been 
able to process xhtml fragments when labeled with 
type="application/xhtml+xml".

> With type="application/xhtml+xml", you'd have to  figure out to do with 
> everything outside of the <body> element.  If  there's CSS involved for 
> example, simply throwing it away could lead  to some very messed up 
> display.

If an aggregator is filtering (x)html content to strip nasty tags (and I'd 
hope that most are) then they should already be stripping css. And since 
there's nothing stopping a malicious content provider from including a full 
xhtml document as part of type="xhtml" content (yes, it's illegal, but that 
doesn't stop it), a careful aggregator should already be stripping off any 
other sub-elements of <head> that might cause problems.

> Given the  tremendously increased complexity involved, some apps are 
> likely to  refuse to process anything that's not one of Atom's three 
> special types.

It doesn't have to be more complex. If you are already handling type="xhtml" 
safely, it's perfectly feasible to treat type="application/xhtml+xml" in the 
same way (and most aggregators do). And it certainly isn't advisable to only 
support Atom's three special types when there are considerably more feeds 
using MIME types than there are feeds using the basic three.

Regards
James

---

From: "James Holderness" <[email-masked]>
Date: Mon, 16 Jan 2006 17:19:20 -0000

> hmmm ... premature optimisation due to common use case of binary content?

Possibly. But technically even entry collections store their content 
externally. If you look at section 9 of the APP spec (Listing Collections) 
they sneak in a note saying that an atom:entry in an entry collection is not 
guaranteed to be a full representation of the data either. A client is 
expected to download the full entry from the "edit" link before making any 
attempt to update it.

Regards
James

---

From: Graham Parks <[email-masked]>
Date: Mon, 16 Jan 2006 20:26:10 +0000

On 16 Jan 2006, at 4:59 pm, James Holderness wrote:

> In theory, yes. In practice, no. Bare in mind that the 0.3 Atom  
> spec had type="application/xhtml+xml" with basically the same  
> functionality as the current type="xhtml". Now since Atom 0.3 is  
> still a whole lot more widely used than Atom 1.0, you can be fairly  
> sure than anyone that bothers to handle XML content at all will be  
> capable of handling type="application/xhtml+xml" containing xhtml  
> fragments.

Speak for yourself. It's crappy assumptions like this that made RSS  
hellish to work with. Atom is unambiguous. "application/xhtml+xml"  
means the page content is a full standalone web page.

Graham

---

From: "James Holderness" <[email-masked]>
Date: Mon, 16 Jan 2006 23:21:19 -0000

Graham Parks wrote:
>> Now since Atom 0.3 is  still a whole lot more widely used than Atom 1.0, 
>> you can be fairly  sure than anyone that bothers to handle XML content at 
>> all will be  capable of handling type="application/xhtml+xml" containing 
>> xhtml  fragments.
>
> Speak for yourself.

I was speaking from experience having tested several different Atom 
aggregators. And I wasn't making any guarantees about anything. I just said 
you could be *fairly* sure that applications would support xhtml fragments.

For example, below are the results of some tests I've run on 15 aggregators. 
The tests included the use of a <div> tag as the root element, a <p> tag as 
the root element, and an <html> tag as the root element (i.e. a complete 
xhtml document).

The following applications worked with all three tests:
BlogBridge 2.7
Bloglines
BottomFeeder 4.1
Google Reader
Snarfer 0.1.2

The following applications worked with the <div> tag and the <p> tag, but 
failed to handle a full document (the <html> tag):
FeedDemon 1.5
GreatNews 1.0.0.354
Newz Crawler 1.8.0
RSS Bandit 1.3.0.38
SharpReader 0.9.6.0

The following applications couldn't display any of the entries:
JetBrains Omea 2.0
Netvibes
Newsgator Online
RSSOwl 1.2
Thunderbird 1.6a1 (wouldn't subscribe)

Draw your own conclusions.

> It's crappy assumptions like this that made RSS  hellish to work with. 
> Atom is unambiguous. "application/xhtml+xml"  means the page content is a 
> full standalone web page.

Not true. Atom *recommends* that the page content is a full standalone web 
page. It's not a requirement. Clearly Atom isn't as free of assumptions as 
you might have hoped. It sure isn't any less hellish to work with.

Regards
James

---

From: Antone Roundy <[email-masked]>
Date: Mon, 16 Jan 2006 16:45:01 -0700

On Jan 16, 2006, at 4:21 PM, James Holderness wrote:
> For example, below are the results of some tests I've run on 15  
> aggregators. The tests included the use of a <div> tag as the root  
> element, a <p> tag as the root element, and an <html> tag as the  
> root element (i.e. a complete xhtml document).
>
> The following applications worked with the <div> tag and the <p>  
> tag, but failed to handle a full document (the <html> tag):
> FeedDemon 1.5
> GreatNews 1.0.0.354
> Newz Crawler 1.8.0
> RSS Bandit 1.3.0.38
> SharpReader 0.9.6.0

Out of curiosity, what constitutes success in the <html> case?  I'm  
mostly curious about the browser based readers.  If they displayed  
the content within a webpage, but failed to strip out the <html>, </ 
html>, <body> and </body> tags and <head> section (assuming the test  
feed contained one), would that be a success or failure?  What did  
the apps that failed do in the <html> case?

---

From: "James Holderness" <[email-masked]>
Date: Tue, 17 Jan 2006 01:05:30 -0000

Antone Roundy wrote:
> mostly curious about the browser based readers.

It was very clear when something had failed - the message content would just 
be blank (or in one case a message saying that there was no content). There 
weren't any partial failures (like not showing the markup, or showing 
extraneous markup, or anything like that).

It's possible that those that succeeded could have been made to produce a 
less than perfect display by doing weird stuff in the head section of the 
<html> example, but I wasn't trying to make things complicated. I just went 
with the simplest valid example I could.

> If they displayed  the content within a webpage, but failed to strip out 
> the <html>, </ html>, <body> and </body> tags and <head> section (assuming 
> the test  feed contained one), would that be a success or failure?

In one case I did a "view|source" which showed that they were nesting the 
entire document (from <html> to </html>) in the middle of the rest of their 
markup. I'm not sure why, but that resulted in a blank message body. I can't 
remember if that was a browser aggregator or stand-alone client but either 
way it was using browser technology to display the content.

If you want to do any further tests or check out my test feed you can find 
it here:
http://216.93.169.119/atomtests/applicationxml/everything.atom

I don't *think* I screwed anything up. It validates in feedvalidator 
(obviously warning about the two entries that used document fragments), and 
the content from the full xhtml test validates in the W3C validator (using 
XHTML 1.0 Strict for the doctype).

Regards
James

---

From: "A. Pagaltzis" <[email-masked]>
Date: Tue, 17 Jan 2006 16:08:00 +0100

Hi James,

I am afraid I have to side with Graham.

* James Holderness <[email-masked]> [2006-01-17 00:30]:
>>It's crappy assumptions like this that made RSS  hellish to
>>work with. Atom is unambiguous. "application/xhtml+xml"  means
>>the page content is a full standalone web page.
>
>Not true. Atom *recommends* that the page content is a full
>standalone web page. It's not a requirement.

No. RFC4287 does not merely recommend it, it RECOMMENDS it.

`application/xhtml+xml` is not mentioned in the document at
all, but the pertinent section in 4.1.3.3 says:

    4.  If the value of "type" is an XML media type [RFC3023] or
        ends with "+xml" or "/xml" (case insensitive), the
        content of atom:content MAY include child elements and
        SHOULD be suitable for handling as the indicated media
        type.  If the "src" attribute is not provided, this would
        normally mean that the "atom:content" element would
        contain a single child element that would serve as the
        root element of the XML document of the indicated type.

I don’t know about you, but I consider a SHOULD to be pretty
strong language.

Aggregators which process @type='application/xhtml+xml' as if it
was @type='xhtml' are in error. Period.

Now, realworld constraints may mean that aggregators are forced
to do this, because too few people who made the transition from
0.3 to 1.0 took the time to implement the new spec properly.
That’s another matter.

But *the spec* leaves not a shadow of a doubt about what it means
when the type of content is `application/xhtml+xml`.

Thus, content producers who still follow the 0.3 custom are
clearly in error too. Period.

Atom is unambiguous. People may misuse it. That’s the extent of
the story.

Regards,

---

From: "James Holderness" <[email-masked]>
Date: Tue, 17 Jan 2006 18:04:00 -0000

>  I don’t know about you, but I consider a SHOULD to be pretty

I consider it as strong as its definition which clearly says: "there may 
exist valid reasons in particular circumstances to ignore a particular item, 
but the full implications must be understood and carefully weighed before 
choosing a different course."

I don't take SHOULD recommendations lightly, but I think I've shown some 
pretty compelling reasons why a producer (if they really absolutely have to 
use "application/xhtml+xml"), would be wiser to use an xhtml document 
fragment than a complete xhml document. Of course the wisest move would be 
to not use any form of xml at all, but that's not what was being discussed.

> was @type='xhtml' are in error. Period.

This argument I don't understand. The spec, while recommending (super 
strongly recommending, if you will) that the content should be a full xhtml 
document, clearly also allows that "there may exist valid reasons in 
particular circumstances [for a producer] to ignore [that recommendation]". 
How can an aggregator that makes allowances for such an eventuallity be in 
error?

Or are you suggesting that RFC4287 is making some other requirement of Atom 
aggregators that I've missed? According to Rob, the spec doesn't actually 
require that Atom aggregators do anything with MIME types. I'm not sure I 
agree with him completely, but section 4.1.3.3 only tells you what to expect 
to find in the content, not what you are required to do with it.

> clearly in error too. Period.

This argument I understand, but I still disagree. I'm willing to accept that 
this behaviour is highly discouraged, but it's not a REQUIREMENT and it's 
not an error. Try persauding anyone on the feedvalidator mailing list that 
SHOULD recommendations should all be flagged as errors and see how far you 
get. I'm pretty sure I'm not the only one that will disagree with you.

Regards
James

---

From: Antone Roundy <[email-masked]>
Date: Tue, 17 Jan 2006 11:45:51 -0700

On Jan 17, 2006, at 11:04 AM, James Holderness wrote:
> but I think I've shown some pretty compelling reasons why a  
> producer (if they really absolutely have to use "application/xhtml 
> +xml"), would be wiser to use an xhtml document fragment than a  
> complete xhml document.

I'm all for consuming applications that want to be really smart  
checking whether the content of <content type="application/xhtml 
+xml"> is a fragment or a complete document and handling either, but  
if your content is an xhtml document fragment, is there any reason at  
all to publish type="application/xhtml+xml" rather than  
type="xhtml"?  The only justification that comes to mind is if you  
want to make a political protest statement against the required  
wrapper div.  But unless you prominently warn your users that your  
app is doing this, you're doing them a grave disservice by making  
their feed content less likely to be seen.

---

From: "A. Pagaltzis" <[email-masked]>
Date: Tue, 17 Jan 2006 20:21:13 +0100

Hi James,

* James Holderness <[email-masked]> [2006-01-17 19:10]:
>>Aggregators which process @type='application/xhtml+xml' as if
>>it was @type='xhtml' are in error. Period.
>
>This argument I don't understand. The spec, while recommending
>(super strongly recommending, if you will) that the content
>should be a full xhtml document, clearly also allows that "there
>may exist valid reasons in particular circumstances [for a
>producer] to ignore [that recommendation]". How can an
>aggregator that makes allowances for such an eventuallity be in
>error?

To recommend conflating `xhtml` and `application/xhtml+xml` is to
deprive content producers of precise expressibility of intent.

The question is, what does the document mean?

RFC 4287 states that saying something is `application/xhtml+xml`
means that it is `application/xhtml+xml`. If I ship entries in a
feed that are marked `application/xhtml+xml` then I really *mean*
they are `application/xhtml+xml` and I don’t want the processor
at the other end second-guessing me, thanksverymuch.

Now, if you save the string on the following line in a file:
<p>Hello world!</p>
with an extension of `.html` and open the file in a webbrowser,
you are unlikely to a complaint. (Even if it’s `.xhtml`.) This is
the point at which the question of whether `application/xhtml+xml`
permits document fragments becomes one-third academical.

But when I ship `application/xhtml+xml`, I want it treated as a
full document, not inline content. That’s why I’m using the MIME
type in the first place, instead of saying it’s `xhtml`. I don’t
want aggregators to be guessing whether an escaped left angle
bracket in my title marks a tag when I’ve clearly labelled the
thing @type='text' either.

>This argument I understand, but I still disagree. I'm willing to
>accept that this behaviour is highly discouraged, but it's not a
>REQUIREMENT and it's not an error. Try persauding anyone on the
>feedvalidator mailing list that SHOULD recommendations should
>all be flagged as errors and see how far you get. I'm pretty
>sure I'm not the only one that will disagree with you.

Of course. Please don’t put words in my mouth; a SHOULD is not a
MUST. In fact I myself argued back then that this requirement can
only be a SHOULD, because expecting Atom processors to enforce a
MUST is nonsensical – no Atom processor is ever going to know
whether some content labelled with a MIME type that the processor
does not support is valid as the type of content it claims to be.

Regards,

---

From: "James Holderness" <[email-masked]>
Date: Tue, 17 Jan 2006 20:39:54 -0000

> deprive content producers of precise expressibility of intent.

So what is your intent? What do you expect aggregators to do with that 
content? Show it as some sort of document attachment that the user would 
have to save to disk and then open up in a separate browser window? Right 
now I'm displaying "appliction/xhtml+xml" documents inline (as do many other 
aggregators). That happens to involve exactly the same process as displaying 
type="xhtml" content inline. You're saying that that is an error, but I 
don't see anything in the spec that substantiates that assertion.

> at the other end second-guessing me, thanksverymuch.

This has got nothing to do with second-guessing. Just pretend for a moment 
that there was no such thing as the "xhtml" type. Now the question is what 
is the correct way for an aggregator to display an "application/xhtml+xml" 
document. There's nothing in the spec that says an aggregator can't display 
that document inline. That's not second-guessing, that's an implementation 
choice. The fact that displaying such a document inline turns out to involve 
the same process as the "xhtml" type is irrelevant.

> type in the first place, instead of saying it’s `xhtml`.

That's not an entirely unreasonable interpretation of what the spec might 
mean, but it's not the only interpretation. My interpretation of the spec is 
that MIME types are there to allow for content not covered by the three 
basic types. So if I wanted my content to be a pdf document or a jpg (as 
*inline* content), I could do so by specifying the necessary MIME type and 
encoding them in whatever format the spec required for that type.

A really smart aggregator would probably have built-in support for several 
common types, perhaps have plugins for less common types, and worst case 
fall back to saving to disk when it knows nothing about the type at all 
(much like web browsers and email clients). If I specifically wanted to 
include content that should be rendered out-of-line or displayed as an 
attachment, I would use an enclosure link.

Following my reasoning, application/xhtml+xml documents are just another 
extension type for which I happen to have built-in support and can be 
rendered inline.

> MUST.

My apologies then. I've obviously misunderstood what you're saying. The 
example feed shown below is using what I believe to be 0.3 customs (i.e. the 
type set to "application/xhtml+xml", but the content is not a complete xhtml 
document):

<?xml version="1.0" encoding="utf-8"?>
   <feed xmlns="http://www.w3.org/2005/Atom">

     <title>Example Feed</title>
     <link href="http://example.org/"/>
     <updated>2003-12-13T18:30:02Z</updated>
     <author>
       <name>John Doe</name>
     </author>
     <id>urn:uuid:60a76c80-d399-11d9-b93C-0003939e0af6</id>

     <entry>
       <title>Atom-Powered Robots Run Amok</title>
       <link href="http://example.org/2003/12/13/atom03"/>
       <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
       <updated>2003-12-13T18:30:02Z</updated>
       <content type="application/xhtml+xml">
       <div xmlns="http://www.w3.org/1999/xhtml">Some text.</div>
       </content>
     </entry>

   </feed>

Now your words were that a content producer following 0.3 customs would be 
in error. From that I concluded that you believed the above document would 
be in error. And since the only thing I can see wrong with this document is 
a SHOULD violation, I assumed you were claiming a SHOULD violation was an 
error. Where did my line of reasoning go wrong?

Regards
James

---

From: "James Holderness" <[email-masked]>
Date: Tue, 17 Jan 2006 20:55:48 -0000

Antone Roundy wrote:
> I'm all for consuming applications that want to be really smart  checking 
> whether the content of <content type="application/xhtml +xml"> is a 
> fragment or a complete document and handling either, but  if your content 
> is an xhtml document fragment, is there any reason at  all to publish 
> type="application/xhtml+xml" rather than  type="xhtml"?

First off I wouldn't recommend using XML content at all because I don't 
think the aggregator support is very widespread yet. But if you were 
determined to use an XML type you may decide to choose application/xhtml+xml 
for backwards compatibility with aggregators that don't yet implement Atom 
1.0. I'm not sure whether there are any aggregators that fall into this 
catagory but it's not an unreasonable hypothesis.

Regards
James

---

From: David Powell <[email-masked]>
Date: Tue, 17 Jan 2006 21:51:44 +0000

Tuesday, January 17, 2006, 9:48:22 PM, I wrote:

> Eg: perhaps the publisher is attempting to send a HTML document that
> they saved in Word, full of CSS styles, that is intended for printing. [*]

[*] Off-topic rant:

Let's hope that the user doesn't attempt to publish their document
as .mhtml, which is bizarrely banned from appearing in Atom. I
suppose it could be base64 encoded and typed as application/octet-stream.

Hmm.

-- 
Dave

---

From: David Powell <[email-masked]>
Date: Tue, 17 Jan 2006 21:48:22 +0000

> the same process as the "xhtml" type is irrelevant.

Assuming that the document's /html/head section is irrelevant and
discarding it, even when the publisher has specifically used non-core
types to send the full document, is second-guessing the user though.

I agree that how you display such content is just an implementation
choice, but if the publisher has specifically used a non-core type to
label content, I think it is a better choice to just treat the content
identically to any other non-core type, and probably display a
download link.

-- 
Dave

---

From: James M Snell <[email-masked]>
Date: Tue, 17 Jan 2006 15:47:32 -0800

James Holderness wrote:
>> all to publish type="application/xhtml+xml" rather than  type="xhtml"?
> 

-1, bad recommendation.  Applications should feel free to use the full 
capabilities of Atom content model regardless of current feed reader 
capabilities.  If a particular use case catches on, the feed readers 
will either implement support or be replaced by ones that do.

- James

---

From: "James Holderness" <[email-masked]>
Date: Wed, 18 Jan 2006 02:35:26 -0000

> types to send the full document, is second-guessing the user though.

Fair enough, but you could also say that anyone filtering out javascript and 
potentially harmful css is also second-guessing the publisher. Maybe they 
specifically wanted to annoy the user with a platypus attack or some such 
thing. If that's second-guessing then I'm all for it. What you're suggesting 
is kind of like having virus checking in your email that can be disabled by 
sending a message with a particular header.

It's always possible to add a save menu that would allow the user to save 
the full unfiltered document (along with a security warning) or an option to 
turn off security filtering for those that like living on the edge, but IMO 
the users' security should comes first. I appear to be in the minority here, 
but nobody has yet persuaded me that what I'm doing is a bad idea.

> they saved in Word, full of CSS styles, that is intended for printing. [*]

If you need to include a document in full and unfiltered, the most sensible 
thing to do is add an enclosure or just a simple <a href>. That's the most 
direct route you can get from yourself to the user and you don't have to 
worry about gateways that might be converting from Atom to RSS, transcoders 
converting from UTF8 to GB2312, or security-concious aggregators stripping 
your css.

> identically to any other non-core type, and probably display a

Well I would display all non-core types inline if I could. I certainly 
display text/html and text/plain inline. When I get a chance I'll do the 
image/* types inline too. If possible I'd like to handle things like pdf, 
word and excel if there are plugins available. So technically I *am* 
treating the content identically to other non-core types. Displaying a 
download link would be the exception rather than the rule.

Regards
James

---

From: "James Holderness" <[email-masked]>
Date: Wed, 18 Jan 2006 05:19:44 -0000

James M Snell wrote:
> capabilities.

Ok. Let me put it another way. Let's say my neighbour came to me and asked 
whether he should be using xhtml for his content type or html. Here's what I 
would tell him:

If he uses xhtml I can guarantee at least one aggregator that will not be 
able to see his content at all. If he needs to use a less than sign or 
ampersand at any point I can guarantee a further 4-13 [1] aggregators will 
have problems displaying his content.

If he uses html I can guarantee at least one aggregator will have problems 
displaying his markup, but the content will still be basically readable. If 
he needs to use a less than sign or ampersand at any point I know of only 
one other aggregator that might [1] have problems displaying that content.

He is of course welcome to choose whichever format he wants, but my 
recommendation, based on the limited testing that I've done, would be to go 
with the format least likely to cause problems. In other words I wouldn't 
reommend xhtml.

You are of course free to recommend the exact opposite to your neighbour.

Regards
James

[1] Depends on what kind of escaping his authoring tool uses.

---

From: "A. Pagaltzis" <[email-masked]>
Date: Tue, 24 Jan 2006 23:22:36 +0100

Hi James,

* James Holderness <[email-masked]> [2006-01-17 21:50]:
>>To recommend conflating `xhtml` and `application/xhtml+xml` is
>>to deprive content producers of precise expressibility of
>>intent.
>
>So what is your intent? What do you expect aggregators to do
>with that content?

I had to think about this for a while.

Really, what I expect them to do with that content is to not fail
to display a full document if a full document is what I provide.
That aggregators attempt to render such full documents inline
(and unsurprisingly fail to produce a reasonable facsimile of the
original) is in error.

I would further expect that full document to be rendered as is,
unadorned by any decorations the aggregator applies to its
rendering of inline content and unaffected by any aggregator
stylesheets. In other words, I shipped a full webpage in my feed,
and I expect it to render as a full webpage in a standalone
browser would.

To show a fragmentary document inline, I suppose, cannot be
argued to be in error. It is still in violation of the spirit of
the spec on the producer’s side to provide such, though.

>Show it as some sort of document attachment that the user would
>have to save to disk and then open up in a separate browser
>window?

As I said before, most likely, it would be shown in an IFRAME.
(This would also provide a local scope for any CSS, thereby
greatly reducing the need to cripple any included stylesheet for
anti-spoofing reasons.)

>My interpretation of the spec is that MIME types are there to
>allow for content not covered by the three basic types. So if I
>wanted my content to be a pdf document or a jpg (as *inline*
>content), I could do so by specifying the necessary MIME type
>and encoding them in whatever format the spec required for that
>type.

Yes, sure, but the whole point is moot because no such thing as a
fragmentary PDF or JPEG documemt exists. Either you have a full
PDF or JPEG document or you have corrupt, undecodable data. This
is markedly distinct from the situation we’re dealing with.

>Now your words were that a content producer following 0.3
>customs would be in error. From that I concluded that you
>believed the above document would be in error. And since the
>only thing I can see wrong with this document is a SHOULD
>violation, I assumed you were claiming a SHOULD violation was an

Your line of reasoning did not go wrong. I just did not mean “in
error” formally as in “producing invalid Atom”. What they’re
doing is technically permissible, because it’s not technically
disallowable. Replace “in error” by “abusing/misinterpreting the
format”, if you want.

Regards,

---

From: "James Holderness" <[email-masked]>
Date: Wed, 25 Jan 2006 01:10:44 -0000

> (and unsurprisingly fail to produce a reasonable facsimile of the

I think I provide a reasonable facsimle of the document when I display it 
inline (although no CSS). I know of maybe 4 other aggregators that handle 
inline xhtml documents to some extent. Maybe 8 handle inline html documents. 
That's not exactly great support considering I've tested 15, but nothing 
about Atom content rendering is well supported.

> and I expect it to render as a full webpage in a standalone

Well to be fair your browser probably includes a title, a url bar, 
navigation buttons, a status bar, etc. I don't think it's unreasonable for a 
feed reader to display the title, author, date, related links and whatever 
other metadata you've associated with that document. I agree that applying 
other styles to your document is probably pushing it.

> the spec on the producer’s side to provide such, though.

Agreed.

> greatly reducing the need to cripple any included stylesheet for

The real problem with CSS is javascript, and I don't think an IFRAME is 
going to help that. I'd happily leave all CSS and javascript in the HTML 
(whether you're using a MIME type or not) if I thought it was safe. But 
everything I've read on the subject has lead me to believe that is a bad, 
bad idea.

> is markedly distinct from the situation we’re dealing with.

Well I thought we were disagreeing about whether content with a MIME type 
should be shown inline or not. I was just saying that I'd ideally render all 
MIME type documents inline if I could. If we can agree that that's 
acceptable, then I think our only difference of opinion is whether it's 
acceptable to preprocess a document prior to displaying it.

I can understand why you wouldn't want your beautiful CSS being mucked up by 
a feed reader for no good reason, but security is a fairly good reason IMO. 
Ideally if I had the time to write a full CSS parser that could safely 
remove the bad bits and leave the good bits I would, but I don't think it's 
reasonable to expect all (or any) feed readers to do that.

> disallowable. Replace “in error” by “abusing/misinterpreting the

Ok, sorry for the misunderstanding. I can agree with that.

Regards
James