Coping with Information Overload

talk by Allen Gorin, Frb. 15


Coping with information overload is a major challenge of the 21st century. Huge volumes and varieties of multilingual data must be selected for relevance and mined for content. The data includes unstructured speech or text in electronic form plus bitmap images of text. Critical information often spans several documents, several sources, and several languages. Information includes both content and context, which humans deal with as a gestalt but computer systems often treat separately Because it is impossible to analyze all of the data manually, revolutionary new solutions are required. We discuss two complementary approaches to coping with information overload and the open research questions that arise in this emerging discipline. First is value estimation, where humans examine only the golden nuggets of information judged valuable by some process. The second approach is knowledge distillation, where the information is digested and compressed, producing salient knowledge for human consumption.


Allen Gorin is Director of Human Language Technology (HLT) Research in the U.S. Department of Defense at Fort Meade, focused on creating HLT technologies for coping with information overload. Before that, he was at AT&T Labs, leading the research team that created AT&T's "How May I Help You?" natural language voice service, which was deployed nationally in 2001. He is a Fellow of the IEEE, was awarded the 2002 AT&T Science and Technology Medal, has published 98 papers, been granted 23 U.S. Patents.

He received the B.S. and M.A. degrees in Mathematics from SUNY at Stony Brook, and the Ph.D. in Mathematics from the CUNY Graduate Center in 1980. He joined AT&T Bell Labs and led the DARPA ASPEN project, developing parallel architectures and algorithms for pattern recognition. In 1987, he was appointed Distinguished Member of the Technical Staff, then joined the Speech Research Department at Bell Labs in 1988. He joined the DoD in 2004. He was a visiting researcher at the ATR Lab in Japan in 1994 and at MIT in 2002.