Subject: Re: todo: add language encoding information

From: Henry Story <[email-masked]>
Date: Fri, 30 Dec 2005 15:17:45 +0100

On 24 Dec 2005, at 00:32, Walter Underwood wrote:
> --On December 23, 2005 11:31:22 PM +0100 Henry Story  
> <[email-masked]> wrote:
>>
>> So  you can't have a link pointing from an entry to an id, without  
>> losing some
>> very important information. We need something more  specific. We  
>> need a link
>> pointing from A to C as shown by the blue line.
>
> Some people will need that in the guts of their publishing system.  
> Why do
> we need it in Atom? Is there something essential that subscribers  
> cannot do
> because this isn't represented?

Yes there are many such uses that good translation information make  
possible. Without precise information telling us what an entry is a  
translation of it is difficult for multilingual sites to tell their  
readers what entries they need to read for their linguistic  
preferences. In multilingual countries like Switzerland, Canada, or  
for the European Union this is very important information. As was  
mentioned in a previous post the Swiss have three official languages,  
and they cannot publish something in one language without also  
publishing it in another. We should try to satisfy their needs within  
a well designed framework.

> This sounds like something needed for the publishing/translation  
> workflow, not for the general readership.

Not the general US readership certainly. But in other countries, such  
as Belgium people do speak different languages. If they come across  
an entry in one language it would be politically correct, if not a  
political necessity, to help them find the version in the language  
they feel most at home in.

Translation metadata can also be very useful to in many other  
regards. Google is working on a translation tool that learns by  
finding vast amount of translated documents on the web and using that  
body of work to create machine translation tools that learn from  
examples. This is clearly going to be a very fruitful area in  
improving machine translation tools. Ever since I was involved in  
launching the Babelfish at AltaVista, machine translation has become  
a 'must have' service for a respectable search engine. There are too  
many languages pairs for it to be practical for this type of  
technology to be entirely coded by hand.

> Extended provenance information is sometimes needed, but there is  
> almost
> no limit to that. It certainly does not stop at translation,  
> source, and
> translator. I'm reading a new translation of Andersen's tales where
> "Thumbelina" is "Inchelina" because the translator knew the right  
> dialect
> of Danish. That is significant, but does it need to be in Atom?

Well we don't need to specify all the translation information of  
course: what dictionaries were used, who the translator was. If we  
can it would of course be nice to make sure we don't exclude those  
options.

> The semantics here should be exactly the same as for dates -- the date
> means what the publisher thinks it means. Same for language info.  
> Trying
> to get more exact means that the model will be wrong for some  
> publishers
> that generate completely legal Atom.

I don't think that the models we are looking at are too difficult  
either for publishers to understand or for consumers to consume. At  
least we should look at the options before we come to the conclusion  
that is the case.

We currently have discovered three types of translation relations:
   - idTranslation: relates an Entry (Version) to an id
   - eTranslation: relates an Entry (Version) to an Entry
   - cTranslation: relates a Content to a Content.

They are clearly distinguished in the following example graph:

These three relations are equally real. They are complimentary in  
that we can find rules to lead us from one to the other. They are, I  
believe non contradictory. And I think we can even specify the rules  
for inferring one from the other.

A. if an Entry A has an idTranslation relation to an atom id, then  
one of the atom entries with that
   id has a eTranslation relation to the subject of the first entry.
   So since _:A xxx:idTranslation <tag:eg.com,2005/en/atom03> we know  
that there is some entry X with the same id, such that _:A  
xxx:eTranslation _:X. In the graph above C is our X.
   in N3 this can be written as (I think)

    { ?e xxx:idTranslation ?id } => { ?c :id ?id. ?e xxx:eTanslation ? 
c. }.

B. if an entry is an eTranslation of another entry, then the titles,  
content and summaries of those entries stand in a cTranslation  
relation to each other.
    So since _:A xxx:eTranslation _:C, we know that:
      - "Robots Atomiques se révoltent"@fr is a tranlsation of "Atom  
powered Robots Run Amok"@en
      - "Apres un cours d'integration en Philosophie Française..." is  
a translation of "After a course of integration in French  
Philosophy..."@en
    in N3 this can be written as:

    { ?e1 xxx:eTranslation ?e2. } => { ?e1 :title ?t1;
                                           :summary ?s1;
                                           :content ?c1.
                                       ?e2 :title ?t2;
                                           :summary ?s2;
                                           :content ?c2.
                                       ?t1 xxx:cTranslation ?t1.
                                       ?s1 xxx:cTranslation ?s1.
                                       ?c1 xxx:cTranslation ?c1.
                                     }.

Now that we are clear about the relations we are speaking about  
(though there is certainly some
debate possible about whether better rules could be found), we just  
need to find the best way to
express this in the atom syntax we so dearly love.

Henry Story

> wunder
> --
> Walter Underwood
> Principal Software Architect, Verity

---

From: Henry Story <[email-masked]>
Date: Sun, 1 Jan 2006 23:21:57 +0100

I just noticed that there was a slight error in my previous diagram.  
I had given all three entries (A, B and C) the same id. I have now  
rectified this. The id of A which is the english entry has a  
different id from that of B and C. Since graphs should try to merge  
elements that are the same, I have here also merged the contents of B  
and C.

On 30 Dec 2005, at 15:17, Henry Story wrote:
> [snip]
> <translation-problem-2.pdf>
>
> that we can find rules to lead us from one to the other. They are,  
> I believe non contradictory. And I think we can even specify the  
> rules for inferring one from the other.
>
>   So since _:A xxx:idTranslation <tag:eg.com,2005/en/atom03> we  
> know that there is some entry X with the same id, such that _:A  
>    { ?e xxx:idTranslation ?id } => { ?c :id ?id. ?e  
> xxx:eTanslation ?c. }.
>
> B. if an entry is an eTranslation of another entry, then the  
> titles, content and summaries of those entries stand in a  
> cTranslation relation to each other.
>      - "Apres un cours d'integration en Philosophie Française..."  
> is a translation of "After a course of integration in French  
> Philosophy..."@en