Lexicalized Markov Grammars for Sentence Compression

talk by Michel Galley, Nov. 9


In this talk, I will present a sentence condensation system for text and
speech summarization. The system is fully trainable from full and compressed
sentence pairs, and relies on synchronous context-free grammars (SCFG), which
extend context-free grammars by generating two strings instead of one (Aho and
Ullman, 1969). I will discuss issues pertaining to earlier SCFG compression
models (Knight and Marcu, 2000; Turner and Charniak, 2005) -- in particular,
sparsity resulting from their reliance on raw treebank grammars -- and present
a solution directly inspired by work in syntactic parsing: grammar
"markovization" (Collins, 1999; Klein and Manning, 2003). I will introduce one
way of representing each SCFG production as a Markov process, and discuss some
of its benefits for sentence compression, including the ability to incorporate
lexical dependencies. I will also describe automatic evaluations for two
tasks, summarization and complement/adjunct distinction, and show that the
smoothed lexical probabilities of SCFG models have positive impact in both.
A human evaluation of the Markov model against the noisy-channel model of
Knight and Marcu (2000) indicates that the output of the Markov model is
substantially more grammatical.


Finally, I will discuss ongoing work in compression of meeting utterances.
After reviewing some issues specific to the task, i.e., disfluency removal and
speech parsing, I will analyze a few cases of utterance compressions the
system currently cannot handle properly because of restrictions intrinsic to
SCFGs, and discuss several solutions.