Sentence Ordering in Multidocument Summarization:   Collection of Multiple Orderings

To obtain our collection, we gave to the participants a set of sentences from different news articles related to the same event. The set of sentences to order is the equivalent to the input to our ordering algorithm, that is, a set of sentences that will form a summary produced by MultiGen before the Ordering component.

The participants were asked to order the texts taking into account only the information conveyed by the sentences, and not to pay attention to the surface cues that could help to find an acceptable ordering. We removed obvious references so that the participants would not be tempted to use them when ordering sentences. (for more details, see the instructions and the experiments interface)