Subject: Atom ConformanceTests results and feedback

From: Sam Ruby <[email-masked]>
Date: Sun, 23 Apr 2006 21:38:09 -0400

http://planet.intertwingly.net/AtomConformanceTests/

It would be helpful if every entry had a distinct atom:id.  And if the
tests were valid:

http://feedvalidator.org/check.cgi?url=http%3A%2F%2Fplasmasturm.org%2Fattic%2Fatom-tests%2Fxmlbase.atom

- Sam Ruby

---

From: "A. Pagaltzis" <[email-masked]>
Date: Tue, 25 Apr 2006 16:39:30 +0200

* Sam Ruby <[email-masked]> [2006-04-24 03:50]:
> It would be helpful if every entry had a distinct atom:id.  And
> if the tests were valid:
> 

Yeah, I should fix those. I’ve also been thinking about the
suggestions to use `<img>` tags to make it easy to scan the
results quickly. Likewise I’ve been thinking about Gordon
Weakliem’s comment on the wiki:

> I suggest that the tests be documented with respect to the
> expected results. For example, TitleConformanceTests is
> perfect: viewing the feed tells you what the expected result
> is. LinkConformanceTests, OTOH, gives me no idea of what the
> author expected to see when viewing the entry in a reader. For
> example, what does the author expect to see when viewing the
> second entry? If I display only the second link, do I pass? Do
> I need to display both links to pass?

I’d like to do more, but writing tests is menial work, and I
don’t have a lot of tuits at the time being. That’s why I asked
about being able to host these at the wiki, so that the touch-up
process would be low-friction.

If you lack tuits to take care of that, I could copy everything
to my site, for the time being, for easier editing.

I make no promises as to when any of that will be, though. :-/

Honestly, I’m a little disappointed that not more tests have been
written so far, and that is has been happening in such haphazard
fashion. Is it really because noone cares? (I suppose I don’t
care that much either, judging by my output.) What would it take
to get more people more involved? Would it help if there was a
list of outstanding testable spec aspects? What aspects need to
be tested (this needs more feedback from consumer developers!)?

Hmm, #atom would be an ideal place to get this done within a
short timeframe, provided a mob got together.

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>

---

From: "James Holderness" <[email-masked]>
Date: Tue, 25 Apr 2006 21:03:45 +0100

A. Pagaltzis wrote:
> fashion. Is it really because noone cares?

Well the end users probably couldn't care less. The aggregator developers 
are actively hostile towards such tests. Feed producers probably find 
informational tests more helpful than conformance tests. And for the poor 
slobs creating the tests it's just a lot of hard work with no visible 
benefit.

Regards
James

---

From: "A. Pagaltzis" <[email-masked]>
Date: Tue, 25 Apr 2006 23:04:46 +0200

* James Holderness <[email-masked]> [2006-04-25 22:15]:
> The aggregator developers are actively hostile towards such
> tests.

Really? I can only think of counterexamples, though my sample is
admittedly tiny. Who are the hostile ones?

Personally, as someone who has written patches for an aggregator
and is flirting with the idea of building one, I would be very
glad to have a defined target to aim at instead of just
eyeballing the overlap between the spec and the code. What sort
of motivation would compel a developer to be hostile toward
tests?

> Feed producers probably find informational tests more helpful
> than conformance tests.

But they are the ones who stand to gain from consistent and
complete implementation of the standard, in the long term.

In any case, can’t we even rally four or five people from the WG
who care enough about the spec to want to do something likely to
increase the chance of good implementations? Where are those who
participated in the interminable flamewars brought on by every
rathole that lay on the way to RFC4287? Have they stopped caring
now, or was all that vitriol just bikeshed painting after all?

Regards,

---

From: James M Snell <[email-masked]>
Date: Tue, 25 Apr 2006 14:34:46 -0700

A couple more link rel test cases:
http://www.snellspace.com/public/linkreltest.xml

See the bottom of: http://www.intertwingly.net/wiki/pie/LinkConformanceTests

- James

---

From: "James Holderness" <[email-masked]>
Date: Wed, 26 Apr 2006 00:03:05 +0100

May I suggest you add a third link with rel="ALTERNATE" in your second test. 
As it is you're catching aggregators that choose the first of multiple 
alternate links, but not catching those that choose the last link.

Also can you get your server to return the correct Content-Type with that 
feed. IE7 will not recognise text/xml as a feed (at least not the build that 
I have).

Regards
James

James M Snell wrote:
> See the bottom of: 
> http://www.intertwingly.net/wiki/pie/LinkConformanceTests

---

From: James M Snell <[email-masked]>
Date: Tue, 25 Apr 2006 16:21:31 -0700

Yes, I'll make those changes this evening.  Actually, the feed was
supposed to have already been served with application/atom+xml...
Thanks for pointing it out.

- James

James Holderness wrote:
> 
> May I suggest you add a third link with rel="ALTERNATE" in your second
> test. As it is you're catching aggregators that choose the first of
> multiple alternate links, but not catching those that choose the last link.
> 
> Also can you get your server to return the correct Content-Type with
> that feed. IE7 will not recognise text/xml as a feed (at least not the
> build that I have).
> 
> Regards
> James
> 
> James M Snell wrote:

---

From: "James Holderness" <[email-masked]>
Date: Wed, 26 Apr 2006 00:43:27 +0100

> admittedly tiny. Who are the hostile ones?

I'm certain not all developers are hostile, but I've witnessed enough of 
that kind of behaviour to put me off getting involved in this kind of thing. 
Maybe I'm just being overly sensitive. If you haven't noticed or it doesn't 
bother you that's great.

> What sort of motivation would compel a developer to be
> hostile toward tests?

The fact that you're putting a bunch of little X's next to their product. 
And to make those X's go away they've got to take time off from whatever 
they're currently working on (which is probably quite important to them) so 
they can deal with your complaints (which in many cases will involve obscure 
features of the spec that they'll never encounter in the wild).

> complete implementation of the standard, in the long term.

That's the thing. Conformance tests will never provide complete coverage of 
the standard, because large portions of it are completely optional. They 
need informational tests that can tell them whether the optional bits have 
any degree of support or whether they're better off avoiding those features.

Regards
James