AVT V. Stirbu Internet-Draft Nokia Expires: August 15, 2005 P. Shrubsole Philips Research J. Costa-Requena Nokia February 11, 2005 LRDP: The Lightweight Remote Display Protocol draft-stirbu-avt-lrdp-00 Status of this Memo This document is an Internet-Draft and is subject to all provisions of Section 3 of RFC 3667. By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she become aware will be disclosed, in accordance with RFC 3668. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on August 15, 2005. Copyright Notice Copyright (C) The Internet Society (2005). Abstract This memo describes an application layer protocol for a framework that enables accessing graphical user interface (GUI) applications remotely, so that devices with different form factors and UI Stirbu, et al. Expires August 15, 2005 [Page 1] Internet-Draft LRDP February 2005 capabilities can scale and adapt the exported UI to their local platform UI look and feel (LAF). Specified in the Extensible Markup Language (XML), the protocol defines generic user interface (UI) operations and a mechanism for extending these operations for specific application needs. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 General Concepts . . . . . . . . . . . . . . . . . . . . . 3 1.2 Definitions . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 The UI Remoting Framework . . . . . . . . . . . . . . . . 4 1.3.1 Discovery-Transport Layer . . . . . . . . . . . . . . 5 1.3.2 Common UI Elements Layer . . . . . . . . . . . . . . . 5 1.3.3 Application Specific Layer . . . . . . . . . . . . . . 5 1.3.4 Architecture Paradigm . . . . . . . . . . . . . . . . 5 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6 3. Protocol Identification . . . . . . . . . . . . . . . . . . . 6 4. Exchange Description . . . . . . . . . . . . . . . . . . . . . 6 4.1 Initialisation Format . . . . . . . . . . . . . . . . . . 6 4.2 Output Update Format . . . . . . . . . . . . . . . . . . . 7 4.3 Input Update Format . . . . . . . . . . . . . . . . . . . 7 5. Widget Description . . . . . . . . . . . . . . . . . . . . . . 7 6. LRDP XML Schema . . . . . . . . . . . . . . . . . . . . . . . 7 7. Internationalization Considerations . . . . . . . . . . . . . 7 8. Security Consideration . . . . . . . . . . . . . . . . . . . . 7 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 7 10.1 Normative References . . . . . . . . . . . . . . . . . . . 7 10.2 Informative References . . . . . . . . . . . . . . . . . . 8 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 9 A. LRDP Design Philosophy . . . . . . . . . . . . . . . . . . . . 9 A.1 Frequently Asked Questions . . . . . . . . . . . . . . . . 9 A.2 Server Considerations . . . . . . . . . . . . . . . . . . 10 Intellectual Property and Copyright Statements . . . . . . . . 12 Stirbu, et al. Expires August 15, 2005 [Page 2] Internet-Draft LRDP February 2005 1. Introduction The protocol outlined in this document is based on the functional requirements described in the Internet-Draft Requirements for Scalable Remote Access to Applications [10]. This document presents a framework that enables applications to remote, in a scalable and adaptable way, the user interface (UI) on remote devices using the Lightweight Remote Display Protocol (LRDP) for describing the semantics of a UI. LRDP is a text based protocol based on Extensible Markup Language (XML) 1.0 as described in [1], XML schema notantion as described in [2] and [3], and XML namespaces as described in [4]. 1.1 General Concepts A user interface (UI) may be composed of several widgets, wherein a widget is understood as an element of a user interface that displays information or provides a specific way for a user to interact with an application. Widgets may for instance comprise icons, pull-down menus, buttons, selection buttons, progress indicators, on-off checkmarks, scrollbars, windows, window edges, toggle buttons, forms and any other elements for displaying information, inviting, accepting, and responding to user actions. When accessing graphical user interface (GUI) applications remotely, provisions need to be made for customizing the remoted UI to match the characteristic display style (the so-called look & feel) of a device platform. This is of particular importance for ensuring that the user experience is not compromised from one device to another. The IETF architecture can be implemented on multiple platforms with its own UI framework for controling the rendering of local application(s) UI(s). Each platform has its specific appearance to provide the appropriate user experience. This platform customisation of the UI has to be replicated with the maximum fidelity on the new platform where the UI is remoted. Therefore, the remoting GUI protocol has to ensure a favorable user experience regardless of the underlying viewing platform, so that the user can perceive the GUI as a local application making it intuitively usable. Based on the above criteria, LRDP defines a protocol framework that supports the following features: 1. A means of describing a UI (or parts of a UI) in terms of widgets and styles. 2. A means of communicating (partial) UI updates from a device that hosts an application to a device that is responsible for rendering the UI. Stirbu, et al. Expires August 15, 2005 [Page 3] Internet-Draft LRDP February 2005 3. A means of describing events triggered through changes in widget state made by the user so that they can be sent to the device that hosts the application. Since the protocol is XML based, LRDP describes the XML elements of the UI, the elements for describing UI events and the elements that must be used to update (parts of) a user interface. The architectural framework that supports this protocol is defined in Section 1.3. Note that LRDP and the XML schema that formally describes LRDP do not specify user interfaces for any particular application or set of applications. This means that it can be applied to many different domain contexts. 1.2 Definitions For clarity, the following definitions are supplied: o UI Servers - The devices that host applications and have the capabilities to export the UI on other devices; they receive UI input from remote users and transmit UI output. o UI Clients - The devices that receive the UI output from a UI server and provide UI input. o UI Remoting Session - The session initiated between UI Servers and UI clients for exchanging UI information to control applications remotely. o UI Remoting Protocol - The types of UI objects, UI events and notifications that are interchanged between a UI client and UI server in order to keep them in sync with each other during a remote UI session. o UI Remoting Framework - The architecture that defines how the protocol should be supported on a UI client and server.. 1.3 The UI Remoting Framework The UI Remoting framework includes the following components grouped on three layers: Discovery-Transport, Common UI Elements and Application Specific. Stirbu, et al. Expires August 15, 2005 [Page 4] Internet-Draft LRDP February 2005 +---------------------------+ Application Specific | UI model | +---------------------------+ Common UI Elements | LRDP | +---------------+-----------+ Discovery-Transport | session setup | transport | +---------------+-----------+ 1.3.1 Discovery-Transport Layer The Discovery-Transport layer includes the components that will implement the remote UI client and server discovery, the remote UI session set up, authentication and connection management, capability negotiation and the transport of UI related information. The Discovery-Transport layer include two components: "session setup" and "transport". o The "session setup" refers to the components that provide discovery features (i.e. for finding remote UI server and clients) and session set up (i.e. negotiating the platform capabilities for initiating the UI remoting). The "session setup" can use either existing IETF session model, encompassing session descriptions using Session Description Protocol (SDP) [6] or successors and the Session Initiation Protocol (SIP) [7], or UPnP Remote UI [8]. o The "transport" refers to the transport protocol used to exchange UI messages between the UI server and the UI client. The "transport" will use existing IETF transport protocols that accommodate the remoting session requirements (e.g. multiplexing several channels, secure transaction, etc). 1.3.2 Common UI Elements Layer The Common UI Elements layer define the basic widgets sets and their properties, which are common to all applications. This layer includes the LRDP component. 1.3.3 Application Specific Layer The Application Specific layer defines widgets, widget properties and styling properties that are specific to an application. This layer includes the UI Model component. 1.3.4 Architecture Paradigm Stirbu, et al. Expires August 15, 2005 [Page 5] Internet-Draft LRDP February 2005 +-----------------------+ +-----------------------+ | UI Client | | UI Server | | +------------------+ | | | | | Host | | | +------------------+ | | | Windowing System | | | | Application | | | +------------------+ | | | +------------+ | | | +------------------+ | | | | App Code | | | | | Viewer | | | | +------------+ | | | | +------------+ | | | | +------------+ | | | | | UI Model | | | | | | UI Model | | | | | +------------+ | | | | +------------+ | | | +------------------+ | | +------------------+ | | +---------+ +------+ | UI update | +------+ +---------+ | | | Session | | LRDP |<----------------| LRDP | | Session | | | | Setup | | |---------------->| | | Setup | | | +---------+ +------+ | UI events | +------+ +---------+ | | | | | +-----------------------+ +-----------------------+ The UI model represents a tree of widgets that make up the user interface plus additional meta-info about the UI. At the top level of abstraction, the UI Model is seen as the Viewport in the MVC model [11]; the Model (application states) and Controller (application specific event handlers) are defined within the "App Code". The UI Remoting Framework maintains two copies of the UI Model: the mastercopy in the Application and a replica in the Viewer. The UI Models are synchronised using the LRDP "UI update" and "UI events" channels. 2. Terminology In this document, the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" are to be interpreted as described in RFC 2119 [5] and indicate requirement levels for compliant implementations. 3. Protocol Identification TBD. 4. Exchange Description 4.1 Initialisation Format This message describes a UI (or parts of a UI) in terms of widgets and styles. (TBD) Stirbu, et al. Expires August 15, 2005 [Page 6] Internet-Draft LRDP February 2005 4.2 Output Update Format This message contains UI updates from a device that hosts an application to a device that is responsible for rendering the UI. (TBD) 4.3 Input Update Format This message contains events triggered through changes in widget state made by the user so that they can be sent to the device that hosts the application. (TBD) 5. Widget Description This section contains description of widgets, their properties, layout and styling information. TBD. 6. LRDP XML Schema TBD. 7. Internationalization Considerations LRDP is represented in XML. The discovery-transport layer MUST negociate a common set of character set encodings to be understood by both UI client an UI server. 8. Security Consideration Input and output data may be highly sensitive. For example input data may contain a user's personal data, such as passwords. At the same time, the user needs to be sure that the UI he wants to interact with was not tampered with. Other applications may not require confidentiality in their UI output. By taking into account the wide range of possible applications, the remoting protocol MUST support end-to-end confidentiality and integrity protection mechanism. 9. IANA Considerations TBD. 10. References 10.1 Normative References [1] World Wide Web Consortium, "Extensible Markup Language (XML) 1.0 Stirbu, et al. Expires August 15, 2005 [Page 7] Internet-Draft LRDP February 2005 (Third Edition)", W3C Recommendation XML, February 2004, . [2] World Wide Web Consortium, "XML Schema Part 1: Structures Second Edition", W3C Recommendation XML Schema, October 2004, < http://www.w3.org/TR/2004/REC-xmlschema-1-20041028/ >. [3] World Wide Web Consortium, "XML Schema Part 2: Datatypes Second Edition", W3C Recommendation XML Schema, October 2004, . [4] World Wide Web Consortium, "Namespaces in XML", W3C Recommendation XML Namespaces, January 1999, . [5] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. 10.2 Informative References [6] Handley, M. and V. Jacobson, "SDP: Session Description Protocol", RFC 2327, April 1998, . [7] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M. and E. Schooler, "SIP: Session Initiation Protocol", RFC 3261, June 2002, . [8] UPnP Forum, "Remote UI Client and Server V 1.0", August 2004, . [9] Scheifler, R., "X Window System Protocol", X Consortium Standard X Version 11, Release 6.7, 2004, . [10] Stirbu, V., Costa-Requena, J. and P. Shrubsole, "Requirements for Scalable Remote Access to Applications", February 2005. [11] Buschmann, F., Meunier, R., Rohnert, H., Sommerlad, P. and M. Stal, "Pattern-Oriented Software Architecture, Volume 1: A System of Patterns", John Wiley & Sons , August 1996, . [12] Lennox, J., Schulzrinne, H., Nieh, J. and R. Barrato, "Protocols for Application and Desktop Sharing", Internet-Draft draft-lennox-avt-app-sharing-00 (work in progress), December 2004, Stirbu, et al. Expires August 15, 2005 [Page 8] Internet-Draft LRDP February 2005 . Authors' Addresses Vlad Stirbu Nokia Tieteenkatu 1 Tampere 33720 Finland Phone: +358 7180 60572 Email: vlad.stirbu@nokia.com Paul Shrubsole Philips Research Prof. Holstlaan 4 Eindhoven 5656 AA The Netherlands Phone: +31 40 27 42102 Email: paul.shrubsole@philips.com Jose Costa-Requena Nokia P.O. Box 321 NOKIA GROUP FIN-00045 Finland Phone: +358 7180 08000 Email: jose.costa-requena@nokia.com Appendix A. LRDP Design Philosophy A.1 Frequently Asked Questions This section provides some arguments for supporting the design of XML based protocol for remoting UI applications in scalable manner. 1. Why "lightweight"? The bandwidth is reduced because UIs are described in a more abstract way when compared with graphics or framebuffer solutions. The latency is reduced because the protocol supports both fine grain interaction between a UI client and UI server as Stirbu, et al. Expires August 15, 2005 [Page 9] Internet-Draft LRDP February 2005 well as course grain interaction. 2. Why "remote display"? This allows the user to setup, control and view applications of a wide variety of devices irrespective of their location relative to the user. 3. Why abstract UI semantics? Devices are intended for a diverse range of purposes and their capabilities can vary considerably; screen ratio, color depth, windowing system with various widget sets, input methods that make the environment highly heterogenous. By having a UI semantics language enhanced with metadata for layout and styling, applications UIs can be scaled and adapted to a wide range of viewing platforms. 4. How LRDP improves the user experience? The look and feel of a user interface can be kept in line with UI environment offered by the interaction device (UI client). This means that users do not have to learn different interaction styles for the different applications that they access on a given UI client device. A.2 Server Considerations Stirbu, et al. Expires August 15, 2005 [Page 10] Internet-Draft LRDP February 2005 +-----------------------------------+ | UI Server | | | | +------------------+ +----------+ | | | Application | | | | | | +------------+ | | | | | | | App Code | | | UIModel- | | | | +------------+ | | Graphics | | | | +------------+ | | Adapter | | | | | UI Model | | | | | | | +------------+ | | | | | +------------------+ +----------+ | | +---------+ +------+ +----------+ | | | | | LRDP | | Graphics | | | | Session | | | | Driver | | | | Setup | +------+ +----------+ | | | | +-------------------+ | | | | | Transport | | | +---------+ +-------------------+ | +-----------------------------------+ The UI Server may have an optional element "UIModel-Graphics Adapter", which converts the UI Model into a format that is suitable for a Graphics Driver. A framebuffer protocol, such as the one described in I-D Protocols for Application and Desktop Sharing [12], can intercept graphics driver APIs and remote the UI to a compatible viewer. Stirbu, et al. Expires August 15, 2005 [Page 11] Internet-Draft LRDP February 2005 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The Internet Society (2005). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Stirbu, et al. Expires August 15, 2005 [Page 12]