\documentclass[11pt]{article} 
%\documentclass[twocolumn,11pt]{article}
\usepackage{graphicx}
\usepackage{times}



\author{\begin{tabular}{c}
Omer Boyaci and Henning Schulzrinne \\
Department of Computer Science  \\
Columbia University  \\
{ \{boyaci,hgs\}@cs.columbia.edu} \\
\end{tabular}
}
\title{Application and Desktop Sharing}
\date{}
\begin{document} 

\maketitle

\begin{abstract}
Application and desktop sharing allows sharing any application with numerous people over the Internet. The participants receive the screen-view of the shared application from the server. Their mouse and keyboard events are delivered and regenerated at the server. Application and desktop sharing enables collaborative work, software tutoring and e-learning over the Internet. We have developed an application and desktop sharing platform called ADS which is efficient, reliable, operating system independent, scales well, supports all applications and has true application sharing. With ADS the users can share any application from their desktop and the participants do not need to have the application installed on their systems.

\end{abstract}
 
\section{Introduction}

Application and desktop sharing allows two or more people to collaboratively work on a single document, drawing or project in real-time. They do not need to email the file back and forth for modifications and corrections. Similarly instructors can give a lecture over the Internet using ADS. Students or instructor can record the session and this recording can be played back locally or from a streaming server to multiple users. Application sharing can be enriched with audio and video for online lectures. Moreover ADS can be used for software tutoring where an instructor teaches a particular software like photoshop, dreamweaver, office suites, autocad and visual studio. Students can see what the instructor does and they can participate to the sharing session by remotely using mouse and keyboard. Instructors can request from students to repeat their actions after showing a feature. 

Section 2 gives some background information on different application sharing system models. Section 3 discusses the related work and why they are inadequate. System architecture is discussed in Section 4, whereas the details of the ADS protocol is discusses in Section 5. Client and server architectures are explained in Sections 6 and 7 respectively. Section 8 gives information about the performance of system in terms of bandwidth and processing power requirements. Section 9 talks about future work and finally Section 10 summarizes our work.

\section{Background}
Application and desktop sharing (\emph{ADS}) allows sharing an application with remote users. All participants see the same screen-view and use the same application. There is only one copy of the shared application and it runs on the server. The main challenges of application and desktop sharing are scalability, reliability, true application sharing, operating system independence and performance. We believe that an application and desktop sharing system should be operating system independent because participants can use different operating systems. Also, the system should scale well because some sharing scenarios such as e-learning and software tutoring may consist of several simultaneous participants. Systems supporting multicasting scales well but participants will end up with corrupted screen-views if the system is not designed reliability in mind. The sharing system should be efficient in the sense that it should transmit only the changed parts of the screen and it should not consume all the resources while doing this. 

\begin{figure}[htpb]
\centering
\includegraphics[width=0.8\textwidth]{real_screen_view.png}
\caption{A screenshot of the desktop}
\label{fig:real}
\end{figure}

Application sharing is different than desktop sharing in which there is only one shared application for privacy and security. The key challenge is that some other application can sit on top of the shared application and the shared application can open new child windows like options or fonts. A \emph{true} application sharing system should blank all the other applications if they are on top of the shared one and should transfer all the child windows of the shared application. For example, if the user wants to share only the Windows Internet Explorer application which has the title ``Windows Live Hotmail - Windows Internet Explorer" from the desktop seen in (Figure~\ref{fig:real}) then the participants should only see the main and the ``Internet Options" windows. ADS (Figure~\ref{fig:ads_view}) displays only these two windows with a correct size while blocking the desktop background and the non-shared application. Whereas another application sharing client Mast, which will be discussed later, could not display the shared application in correct size and also could not block the non-shared application and desktop background (Figure~\ref{fig:mast}).

\begin{figure}[htpb]
\centering
\includegraphics[width=0.8\textwidth]{ads_client.png}
\caption{ADS Client View}
\label{fig:ads_view}
\end{figure}

\begin{figure}[htpb]
\centering
\includegraphics[width=0.8\textwidth]{mast_client.png}
\caption{Mast Client View}
\label{fig:mast}
\end{figure}

There are two models for application sharing: application-specific and generic. The application-specific model requires the developers to add this feature to their applications; for example, the latest version of Microsoft Office has this feature. Also, in order to have a sharing session all participants must have a copy of the shared application. In the generic model, the application can be anything such as word processor, browser, CAD/CAM, Power point or movie editor. Also, the participants need not have the application installed on their systems. The only disadvantage of generic application sharing is that its generic nature makes it a bit inefficient as compared to the application-specific model in certain scenarios.  We have developed ADS based on the generic model therefore, users can share any application without requiring the participants to have the application.


\section{Related Work}

As we have mentioned in the previous section, application and desktop sharing is a very useful feature therefore, operating system providers, commercial companies and several open source teams try to come up with a solution. Microsoft has Windows Meeting Space for Windows Vista and Netmeeting for Windows XP and older versions. Netmeeting was released in 1999 for Windows 98 and in our tests it fails to display pop-ups and menus. Windows Vista brings application sharing feature with the Windows Meeting Space but all the attendees should use Windows Vista. Apple will bring desktop sharing (no application sharing) with Mac OS X Leopard but again both parties should use Mac OS. These two solutions are operating system dependent. VNC is a cross-platform open source desktop sharing system but it does not have application sharing feature too. UltraVNC~\cite{pa} has the application sharing feature but it has failed in our tests due to following problems: the cursor position did not match, there is no privacy, new windows belong to same application are not recognized and the long menus are not shown properly. 


\begin{table}[!htbp]
\caption{Comparison of Related Work}
\label{table:relatedwork}
\begin{tabular}{|l|c|c|c|c|c|}
\hline
& OS & S & AS & RC & REC \\ \hline
VNC & + & - & - & + & + \\ \hline
Windows Meeting Space & - & - & + & + & - \\  \hline
Leopard Screen Sharing & - & - & - & + & - \\ \hline
TeleTeachingTool & + & + & - & - & + \\ \hline
Mast & + & + & - & + & - \\ \hline
\emph{ADS} & + & + & + & + & + \\ 
\hline
\end{tabular}
\\ \\ Legend\\
\begin{tabular}{|l|l|}\hline
OS    & Is it operating system independent?\\
S       &Is it scalable?\\
AS    & Does it support application sharing?\\ 
RC   & Does it support remote control?\\
REC& Does it support recording?\\\hline
\end{tabular}
\end{table}

VNC, Netmeeting, Windows Meeting Space and Leopard Screen Sharing are all unicast based and because of that they do not scale well. Sharing an application via unicast increases the bandwidth usage linearly. For example, Microsoft suggests Windows Meeting Space for a group of 10 users or less. TeleTeachingTool~\cite{ttt} and Multicast Application Sharing Tool (MAST)~\cite{mast} use multicasting to overcome this limitation. TeleTeachingTool adds multicast support to VNC servers however, it is developed just for online teaching so it does not allow participants to control the shared desktop. Also, it does not support true application sharing due to its underlying VNC system. MAST allows remote users to participate via their keyboard and mouse but its screen capture model is based on polling which is very primitive and not comparable to current state of art the capturing methods like mirror drivers which will be discussed in section 3.1. Also MAST does not support true application sharing due to problems which we have mentioned in the previous section (Figure~\ref{fig:mast}). Although both TeleTeachingTool and MAST use multicasting for scalability, they do not provide a real solution to the unreliable nature of UDP transmissions. UDP does not guarantee delivery of packets and if delivered packets can be out of order. In order to compensate the packet losses the TeleTeachingTool and MAST periodically transmit the whole screen which increases the bandwidth and CPU usage. Table~\ref{table:relatedwork} compares the mentioned sharing systems. 





\section{System Architecture}
This proposed system is based on a client-server architecture (Figure~\ref{fig:arch}). The server is the machine which runs the shared application. The participants use a lightweight Java client application for connecting to server and they do not need the shared application. They receive screen updates from the server and send keyboard and mouse events to the server. 

\begin{figure}[htpb]
\centering
\includegraphics[width=0.5\textwidth]{arch.png}
\caption{System Architecture of ADS}
\label{fig:arch}
\end{figure}

We have developed a Java client which naturally works in almost every operating system. The server could not be written with Java due to lack of Java's low level support. Therefore, there should be a server for each operating system and we have developed a windows XP server and mirror driver. Mirror driver is the best known technique for capturing screen update events and will be discussed in detail at section 3.1. Windows XP calls the same drawing functions on both real graphic driver and mirror driver. This mechanism allows the server to learn the updated screen regions without polling. Therefore, the overhead of application sharing is minimal on the server. Without the mirror driver sharing server should poll the screen state in order to detect the changed regions. The mirror driver runs in the kernel mode and notifies the user mode server when it detects some changes in the GUI of the shared application. Server then prepares a packet which consists of a RTP~\cite{bfcp} header and a PNG~\cite{png} image of the updated region. RTP allows the clients to re-order the packets, recognize missing packets and synchronize application sharing with other media types like audio and video. The screen updates are distributed as PNG images. PNG is an open image format which uses a lossless compression algorithm zlib which is necessary to have a clear and undistorted image of the shared application on the clients. 

The server supports both multicast and unicast transmissions. For unicast connections, either UDP or TCP can be used. TCP provides reliable communication and flow control therefore, it is more suitable for unicast sessions. The bandwidths of TCP clients can be different so we have developed an algorithm which sends the updates at the link speed of each client. For UDP clients server controls the transmission rate because UDP does not provide flow control. Several simultaneous multicast sessions with different transmission rates can be created at the server. ADS has a NACK based retransmission mechanism for UDP clients. The UDP based clients request the missing packets from the server while storing the received out-of-order packets in a receive buffer. Multicast clients listen the NACK messages from other clients in order not to send multiple NACK requests for the same lost packet. Briefly, the server can share an application to TCP clients, UDP clients and to several multicast addresses in the same sharing session.

Late-joiners should be handled carefully otherwise they can degrade the systems performance. ADS generates a full-screen update for the late-joiners. But if more than one participant join lately within a minute, ADS generates only one full-screen update for the first one and starts all the others from this full-screen update. They will receive the full-screen update first and then all the other partial updates up to current status of the screen.   

Although multiple users could receive the screen updates simultaneously, clearly only one of them can manipulate the application via keyboard and mouse events. ADS uses the Binary Floor Control Protocol (BFCP)~\cite{bfcp} to restrict the control of the application to a single user. BFCP receives floor request and floor release messages from clients and grants the floor to the appropriate client for a period of time while keeping the requests from other clients in a FIFO queue. All BFCP messages, keyboard and mouse events are transmitted directly to the server using TCP. For mouse and keyboard events Java's own key-codes are used because these events are captured from a Java client and regenerated by a Java component at the server.

We have added recording feature to the ADS. Participants can record the sharing session to a file. This file can be used to watch the session locally or to stream it to multiple receivers simultaneously. This feature is very useful for preparing lectures or software tutorials for future use.

\section{ADS Protocol}

ADS Clients and servers communicates using the ADS protocol which defines the packet structures and message types. The protocol defines five messages from server to client and two messages in the reverse direction (Table~\ref{table:adsmessages}). Each new client needs one ``open new window'' message for each shared window. For TCP clients server transmits this message right after the connection establishment. UDP based clients send a ``full intra-frame request'' to the server when they join the session. Receiving this FIR message server first transmits ``open new window'' messages and then the whole screen image via ``region update'' messages. 

\begin{table}[!htbp]
\caption{ADS Message Types}
\label{table:adsmessages}
\begin{tabular}{l}
Server Messages\\ \hline
Open New Window\\
Close Window\\
Window Size Update\\
Region Update\\
Move Rectangle\\
\\
Client Messages\\ \hline
Full Intra-frame Request (FIR)\\
NACK request\\
\end{tabular}
\end{table}

Each protocol message consists of a RTP header, message type identifier, window identifier and message specific payload (Table~\ref{table:adsmessage}). Message specific payload structure and length are determined by the message type. ``Close window'' message does not have a message specific payload whereas ``window size update'' message carries new size information in this part. These messages are carried on UDP packets for UDP clients. Neither TCP nor RTP declares the length of the RTP packet therefore, RTP framing~\cite{rtpf} is used to split RTP packets for TCP clients. Server and clients should ignore messages with an unknown message type. Also new message types can be added easily to the protocol. 

\begin{table}[!htbp]
\caption{ADS Message Structure}
\label{table:adsmessage}
\begin{tabular}{|l|l|l|l|}
\hline
RTP Header&Message Type&Window ID&Type Specific Payload\\ 
\hline
\end{tabular}
\end{table}
         

\section{Client Architecture}
The ADS Client is very simple compare to the server. Basically it receives screen updates from the server and displays them to the user. It is completely stateless in the sense that it can disconnect and reconnect to the server without losing anything. Due to its simplicity, client for different platforms can be developed easily. A Java ADS Client has been developed in our lab and can run on any Java supported system. 

ADS clients can be passive or active. In active mode, they connect to ADS servers whereas in passive mode they wait for incoming connections. The passive mode can be used as a RGB cable replacement for presentations. Basically, the client becomes a network projector which waits for incoming connections. Presenters can connect to this client and share their applications. A computer can be dedicated as a network projector and only this machine should physically attached to a real projector. The presenters do not need to attach their laptops with an RGB cable to a projector. They just stream their application view to the network projector. 

All participants of a sharing session can be view-only or they can request the control from the server. In order to request the control from server, user should press the ``control'' button from the GUI of the client application (Figure~\ref{fig:ads_view}). The client will send a floor request message and then the server will reply with a granted or request queued message. The server grants the floor immediately if nobody has the control of the floor. Otherwise, the request will be queued in a FIFO queue and floor will be granted to the requesters one-by-one automatically. Users can release the floor whenever they want by pressing the ``control'' button again. After server grants the floor the client captures all keyboard and mouse events locally and transmits them to the server for regeneration. Javas' standard key-codes are used for keyboard events.  

\section{Windows Server Architecture}
TBD later...

\section{Performance Results}
TBD later...

\section{Future Work}
TBD later...

\section{Conclusion}
We have developed an operating system independent, scalable and efficient application and desktop sharing platform. ADS supports all applications due to its generic model and transmits only the shared application and its child windows. We have used industry standards like RTP, BFCP and PNG. The CPU usage of ADS is very low thanks to the mirror driver. The bandwidth usage of ADS is similar to other solutions like VNC. ADS allows collaborative working on a single document or project. It can also be used for e-learning and software tutoring. People can develop clients and servers compatible with ADS by implementing its open protocol.   



\begin{thebibliography}{99} 

\bibitem{pa} UltraVNC Win32Server 1.0.2 Release Build Jun 18 2006 14:56:09 
\bibitem{png} Portable Network Graphics, http://www.libpng.org/pub/png/
\bibitem{java} Java Technology, http://java.sun.com/
\bibitem{ttt} Peter Ziewer and Prof. Dr. Helmut Seidl. Transparent TeleTeaching. In Proceedings of ASCILITE 2002, Auckland, NZ, Dec. 2002.
\bibitem{mast} Mehmood Hasan, Gareth Lewis, Vassil Alexandrov, Dove Martin and Tucker Matt. Multicast Application Sharing Tool for the Access Grid Toolkit. In Proceedings of the UK e-Science All Hands Meeting, Nottingham, UK, 2005.
\bibitem{bfcp} G. Camarillo, J. Ott, K. Drage. The Binary Floor Control Protocol (BFCP). RFC 4582, IETF, November 2006.
\bibitem{rtp} H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson. RTP: A Transport Protocol for Real-Time Applications. RFC 3550, IETF, July 2003.
\bibitem{rtpf} J. Lazzaro. Framing Real-time Transport Protocol (RTP) and RTP Control Protocol (RTCP) Packets over Connection-Oriented Transport. RFC 4571, IETF, July 2006.

\end{thebibliography}



\end{document} 