Findings from these activities

A prototype of the query input interface has been constructed and integrated into a demonstration version of the clinical information system at New York Presbyterian Hospital. The prototype demonstrates the use of clinical data to assist with the identification of the user's information need by matching clinical data to relevant questions in a database of generic queries. The system can be accessed at http://informatics.cpmc.columbia.edu/dli2 and then selecting "Microbiology" (other parts of the demonstration are under development). The resulting displays of microbiology results are identical to those used in hospital's system, except that a "Digital Library" link has been included. This link passes the clinical concepts to the query interface that allows the user to selct concepts of interest. These are then matched against generic questions, the questions are instantiated with the actual concepts, and the resulting questions are presented to the user. The user then selects a question and the corresponding query graph is passed (in XML) to the search engine.

Building on this research we developed and demonstrated a preliminary implementation of the infrastructure and user interface front-end for PERSIVAL at the DLI2 PI meeting in Stratford-upon-Avon in June 2000. We take a hybrid approach between fat and thin-clients in our infrastructure that permits the creation of rich, interactive and multimodal user interfaces that run on lightweight clients with standard software configurations that are remotely controlled by a secure server. It provides the benefits of a thin-client approach, while maintaining the ability to have a rich user interface that could previously only be implemented using a fat-client approach. We are in the process of writing up this architecture for publication, and are applying for patent protection on the ideas that it embodies.

One of the key contributions of our work will be an automated layout component. Over the past three months, we have been surveying previous work on automated user interface layout, and are preparing for publication an overview paper on existing approaches.

Each of these methods will require text processing components to match user input to generic queries. The queries will then be instantiated and used in the same way as the ones built from clinical data. The query graph will be expanded to include additional relevant patient data - we are in the process now of building the knowledge base for determining which data are relevant.

Once a preliminary infrastructure for the query input interface has been provided and other input modalities (e.g., image or video features, direct manipulation of past queries) have been developed, these different modalities must be integrated in a single display. Thus, the next step will be to add automated layout intelligence to the display system. We expect that this will be completed by July 2001. With a preliminary automated layout component in place, we will perform user studies to tune the layout algorithms and gauge their effectiveness.

The query interface is being expanded to include voice and text input. Work during this period has focused on the speech-enabled query interface. The goal is to develop a method for greatly improving recognition accuracy by customizing the recognition based on the clinical context. Initial work has resulted in development of a prototype multi-modal, client-server architecture for context and content based speech control of web applications. Work has resulted in a prototype clinical query interface utilizing dynamically generated speech grammars embedded in HTML. This demonstrates the feasibility of this approach in the current internet environment. The work entails a novel approach to the deployment of multi-modal rule based application speech grammars, a model for embedded grammars and an architecture for implementing this model via a web-browser.