Copyright 1996 IEEE. Published in the Proceedings of the IEEE Fifth Workshops on Enabling Technology: Infrastructure for Collaborative Enterprises (WET ICE '96), June 19-21, 1996, Stanford, California, USA. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works, must be obtained from the IEEE. Contact: Manager, Copyrights and Permissions / IEEE Service Center / 445 Hoes Lane / P.O. Box 1331 / Piscataway, NJ 08855-1331, USA. Telephone: + Intl. 908-562-3966.

The EMSL TeleViewer: A Collaborative Shared Computer Display

Paul E. Keller and James D. Myers
Environmental Molecular Sciences Laboratory, Pacific Northwest National Laboratory
K1-87, P.O. Box 999, Richland, WA 99352, USA
paul.keller@pnl.gov and jim.myers@pnl.gov

This paper has been submitted to the IEEE Fifth Workshops on Enabling Technologies: Infrastructure for Collabative Enterprises to be held at Stanford University, California on 19-21 June 1996.

Pacific Northwest National Laboratory is a multiprogram national laboratory operated by Battelle Memorial Institute for the U.S. Department of Energy under Contract DE-AC06-76RLO 1830.

Abstract
Today's desktop conferencing and groupware software often assume a serial work model in which information (pictures, documents, presentations) are prepared by one person and then disseminated to others for comments, revision, or review. However, many types of collaborative work are much more parallel, with two or more people viewing, modifying, and adding information simultaneously. Effectively supporting parallel collaborative work electronically is still a challenge because of the current serial work paradigm found in most software. One promising and, broadly applicable capability is a collaborative shared computer display (CSCD). A CSCD allows several people to simultaneously view, and possibly annotate or manipulate, a live computer display sent from a remote machine to their local computers. We have been building a prototype CSCD, the Environmental Molecular Sciences Laboratory TeleViewer, which supports cross-platform display sharing between MacOS, Windows, and UNIX operating systems. The current EMSL TeleViewer prototype supports display sharing of application windows, screen regions, and desktops. We are adding enhancements to the EMSL TeleViewer that will provide collaborative annotations over the display, shared mouse cursors/pointer, high performance data compression, and session recording capabilities. When completed, the EMSL TeleViewer will provide researchers, their collaborators, and the scientific community with a powerful tool that can by itself open up many new avenues for collaboration and will fit well with other tools to provide a comprehensive collaborative environment.

I. Introduction

The Environmental Molecular Sciences Laboratory (EMSL) is a U.S. Department of Energy (DOE) collaborative user facility, where collaboration was originally envisioned as relying on visits to the facility by external researchers. The EMSL is funded to perform environmental molecular sciences research to address DOE and national problems related to the environment. Within the EMSL are state-of- the-art computing facilities and instrumentation in nuclear magnetic resonance (NMR) spectroscopy, Fourier transform ion cyclotron resonance mass spectrometry, molecular beams, material science, and chemical physics. One goal of our Collaboratory software development effort is to design and build tools that enable more effective collaboration at a distance, thereby optimizing the valuable time that researchers spend visiting EMSL to conduct experiments. At the same time, the instruments will be made more effective tools for both local and remote researchers through the integration made possible with a Collaboratory approach. The ability to share expertise, instruments, computations, data, and analyses via electronic means will facilitate all phases of the experimental process.

Collaboration involves the exchange of many types of information over many timescales. The mechanisms used to pass information between people can be categorized according to how static or dynamic the shared information is, and how parallel (synchronous) the processes of information creation and consumption are. The first tools for collaboration via the Internet, such as email, FTP, etc., allowed information to be passed back and forth between collaborators, with changes made in a serial manner. The World Wide Web (WWW) made sharing of relatively static information with large audiences simple. The WWW also allowed dynamic information to be shared (by using common gateway interface scripting), and allowed the development of tools for synchronized browsing (through the common client interface) such as the EMSL WebTour [1]. Electronic notebooks are pushing the frontier by allowing users, or other data sources, (e.g. a scientific instrument) to dynamically store information for later use by distributed workgroups [2]. Other tools such as video-conferencing and shared whiteboards allow completely synchronized interaction and dynamic creation of some new information through conversation and shared drawing.

Support for parallel activities where all participants are generating information is still quite limited. Single platform solutions for sharing generic applications between a remote and local user, (i.e. PC-Remote, Co-Session, etc.), provide some support for parallel work. The development of group editors will help solve this problem for text. We are pursuing a more general solution, extending slide sharing, whiteboard technologies, and remote control technologies to create a collaborative shared computer display (CSCD). The ultimate goal of this work is to allow multiple remote users to view, annotate, and interact with any application on any computer.

We have developed a prototype cross-platform CSCD, the EMSL TeleViewer. This prototype provides a real-time view of a display rectangle, window, or the entire computer display to multiple participants via a TCP/IP network (e.g., the Internet). The EMSL TeleViewer currently permits computer display sharing between clients running Windows (3.1, Windows for Workgroups, Windows 95), MacOS (System 7) and several UNIX platforms (SunOS, Solaris, SGI/IRIX), and supports a variety of bit depths and color formats.

Integrated into the EMSL Collaborative Research Environment (CORE) software, the EMSL TeleViewer has already been demonstrated in a variety of situations including remote participation in a scientific experiment (using one the EMSL NMR spectrometers), remote theoretical chemistry analysis and molecular geometry visualization, joint analysis of financial data in a spreadsheet, collaborative document editing, and distance learning. The TeleViewer allows collaborators to work with each other directly, deciding on improved parameters for an experiment and monitoring the results, choosing basis sets for calculations, understanding the molecular structure by rotating and zooming in on the structure and highlighting specific regions of the molecule, discussing and updating spreadsheet formulas and the phrasing of documents, and allowing a mentor to assist a student in running unfamiliar simulations. The TeleViewer allows remote collaborators to be active participants on work-in-progress rather than passive viewers of prepared material.

II. Technical Description

The EMSL TeleViewer currently functions in a client-server mode as illustrated in Figure 1. The client-server approach permits the receiving clients to request image updates at rates they can handle, with the server providing buffering of the image stream. Thus, the slowest computer in the collaboration will not be the bottleneck for everyone's reception speed. In this mode of operation all communication is routed through the server. Support for direct client-to-client communication is anticipated later this year.

[Figure of MacOS, Windows, and UNIX clients communicating with the server.]

Fig. 1. TeleViewer's client-server model.

A. TeleViewer Client

The basic TeleViewer client performs several key functions including window, screen, and region capture; image transmission; image reception; image display; file storage and retrieval; and annotation. The TeleViewer client captures a window, desktop screen, or region of the screen as a raster image. This capture is performed at all standard video bit depths for both color and monochrome video displays. Repeated capture in real-time is supported. The shared image is transmitted to the server (or other client when in client-to-client mode) in the TeleViewer Interchange Format (TVIF). The shared image is also compressed with a frame-to-frame differencing method followed by compression with a run length encoding method to reduce the network bandwidth required to transmit the live image stream.

Images are sent to a TeleViewer server, which redistributes the shared image to the other clients within the collaboration. The communication between the server and clients is via a TCP/IP network, using Berkeley standard sockets and our own sockets based messaging library.

The TeleViewer Interchange Format (TVIF) is the common data format for the shared application window. It combines features of the Windows DIB/BMP file format [3] with the Tagged Image File Format (TIFF) [4] and supports both big and little endian byte formats. TVIF is composed of individual tagged data structures. These structures may be transmitted independently via the sockets messaging system. These data structures include the raster image data, the image colormap, descriptions of the image format, data format information, platform information, etc.

The TeleViewer client redisplays transmitted images from other clients in real-time and supports reception of binary images; palette based color and gray-scale images at 1,2,4, and 8 bit depths; and true color images at 16, 24, and 32 bit depths.

A static image of the display shared in the TeleViewer can be saved as a native TVIF file, a TIFF file, or a Graphics Interchange Format (GIF) file [5]. Depending on its acceptance as a standard, the clients will potentially export to the Portable Network Graphic (PNG) format [6]. In addition, platform specific formats are supported by individual clients. The Windows clients support file exportation to the Windows BMP/DIB format. The MacOS clients support file exportation to the PICT format [7]. The UNIX clients support file exportation to the Postscript format. In addition, the TeleViewer clients permit printing of the shared computer display.

Currently, static images can be read by the TeleViewer from native TVIF files. Future versions of the TeleViewer may permit file importation from TIFF, GIF, and PNG files. In addition, importation of platform specific formats (i.e., Windows BMP/DIB on the Windows platform, PICT on the MacOS platform) will likely be supported by individual clients.

The MacOS and Windows TeleViewer clients interact with their respective clipboards by permitting users to copy an image of the shared computer display from the client to any other application that interacts with the clipboard. These clients also permit images to be pasted onto the shared computer display for viewing by all collaborators. Under MacOS, images are in the PICT format. Under Windows, images are in the Windows bitmap format. In addition, text can be pasted to the TeleViewer client for sharing among the collaborators. This text is transmitted in the ISO 8859-1 character set [8].

The Windows TeleViewer clients run on Intel i80386, i486 and Pentium PCs and PC compatibles with network capabilities. The basic system requirements for the Windows client are Microsoft Windows (3.1, WFWG, NT, 95); WinSock; VGA or better video; and at least four megabytes of free memory. The MacOS TeleViewer clients run on Motorola mc68k and Power PC based Macintoshes and Macintosh compatible computers with network capabilities. The basic system requirements for the Mac client are MacOS System 7, MacTCP or OpenTransport, at least three megabytes of free memory, and network capabilities. The UNIX TeleViewer clients utilize X-Windows and currently run on SPARC based Sun workstations running SunOS 4.x or Solaris 2.4, and MIPS based Silicon Graphics workstations running IRIX 5.3. Additional platforms such as RS/6000 based IBM workstations running AIX, HP 9000 workstations running HP/UX, Alpha based DECstations running OSF1, and MIPS based DECstations running Ultrix are being considered.

Fig. 2. View from a Windows EMSL TeleViewer client during a collaboration session in which a window from a Macintosh is being shared with other collaborators who are providing annotated feedback.

The TeleViewer client provides familiar drawing tools for basic annotation of the shared application window. Annotation includes tools for adding text, erasing objects, drawing lines, drawing arrows, and drawing geometric objects rectangles, ellipses/ovals, arcs) in a variety of colors and line widths. Figure 3 illustrates the annotation tool bar with these tool buttons. All members of a collaborative session are able to annotate the shared window. These capabilities are similar to a real-time fax where markups are instantaneously available to all collaborators. When operated in a client-server mode, annotation of the shared computer display is coordinated by the server.

Fig. 3. TeleViewer annotation tool bar. The top half illustrates the buttons for the text, eraser, line, arrow, rectangle, ellipse, and arc tools. The bottom half illustrates the buttons for line width and color palette selection.

B. TeleViewer Server

The basic TeleViewer server routes incoming images from a TeleViewer client to the other clients in the collaboration session. The server supports late arrival of additional collaborators and provides individual data streams to remote users to simultaneously support users with different data transfer and hardware speeds and guarantees valid displays at all times. This design easily supports recording and storage of the shared view at a selectable frame rate. Use of a generic cross-platform messaging library also makes it easy to add new features and message types (annotations, cursor positions, key presses, etc.) to the TeleViewer. The power of the server approach is that each user can view the shared computer display at the rate their computer and network connection permits.

TeleViewer servers currently exist for SPARC based Sun workstations running SunOS 4.x or Solaris 2.4, and MIPS based Silicon Graphics workstations running IRIX 5.3. Support for additional UNIX platforms is anticipated by the end of 1996.

III. Operation

In its normal mode of operation, the EMSL TeleViewer is launched from the EMSL CORE Session Manager [1]. The TeleViewer can also be run independently. When used with CORE, the TeleViewer client will automatically be started from the Session Manager, along with other requested tools. The session manager will also launch a Televiewer server for the session. During startup, the client displays a message box, providing status information to the users as communications are established between the client and server.

To run the TeleViewer client as a sender, a user selects either rectangle, window, client window, active window, or desktop from the appropriate Capture Menu item. If the user selects rectangle, the user moves the mouse pointer to a corner of the of the desired rectangle, the mouse button is clicked, and the mouse pointer is dragged to the other corner of the desired rectangle. If the user selects window or client window, then the user moves the mouse pointer to the window of interest and then clicks the mouse button. When the user selects active window or desktop, nothing else is required. The user then selects an appropriate update rate for the transmission from several options in the Mode Menu (single image, repeated as-soon-as-possible, or repeated at timed intervals).

To run the TeleViewer as a receiver, the user selects one of the update modes from the Mode Menu. If properly connected and another collaborator is sending, the user will see an image of the shared window appear on the local TeleViewer client.

To switch from between receiving and sending, a user simply initiates a capture operation. The Televiewer system reverses the direction of communications and the new image stream is sent to the other Televiewer clients, including the original sender. To stop sending or receiving, the user selects Standby from the Mode Menu

IV. Future Directions

As the current EMSL TeleViewer prototype is developed into a production level tool, additional features are envisioned including remote application control capability, data security, and time-sequence file storage and playback. Through the DOE's Distributed Computing Coordinating Committee (DCCC) Applications Working Group (AWG), we've developed and distributed a CSCD requirements survey that will be used to prioritize the additional of new TeleViewer features. Survey results will be available from the DCCC AWG and from the authors. In addition to the extensions mentioned above, support for additional platforms, display depths, and file import and export formats, as well as image printing and clipboard interaction are likely candidates for inclusion in the EMSL TeleViewer.

The addition of the mouse cursor/pointer sharing will permit collaborators to highlight areas of interest on the shared application. In this mode, a user moves the mouse cursor/pointer over a region of interest. and then this position is highlighted on all TeleViewer clients.

The addition of remote control and manipulation capabilities would allow users to fully share applications across platforms and distance. Implementation will involve capturing mouse and key events on one users computer, sending these events to the remote computer, translating the events (e.g., Mac to Windows, X to Mac, etc.), and posting these events in the remote computer's event queue.

Because of concerns about proprietary, secure, and even unpublished data, the TeleViewer clients and servers will likely be provided with DCE/Kerberos data security.

Another useful feature on future versions of the TeleViewer is time-sequence file storage and retrieval. This would permit the TeleViewer to store and replay TVIF sequences and would configurable as a viewer for WWW clients, so that TVIF sequences could be embedded in HTML documents as MPEG movies are now.

V. Conclusions

The EMSL Televiewer allows groups to collaborate directly on work in progress via the Internet. Such a capability can provide a catalyst for strengthening collaborative professional relationships, allowing user's to overcome differences in vocabulary and the limitations of static presentation materials and develop an in-depth understanding of their collaborators' contributions to joint projects. Coupled with the EMSL's CORE environment, the TeleViewer provides basic support for the diverse scientific collaborations expected between EMSL researchers and their remote colleagues.

The EMSL TeleViewer prototype, integrated into the EMSL Collaborative Research Environment, is ready for beta testing on the Windows, SunOS, Solaris, and IRIX platforms. We anticipate the MacOS version to be ready shortly for beta testing. For more details contact the authors or visit the EMSL TeleViewer WWW page at:
http://www.emsl.pnl.gov:2080/docs/collab/userdoc/tv/

VI. References

[1] D.A. Payne, J.D. Myers, "The EMSL Collaborative Research Environment (CORE) - Collaboration via the World Wide Web," submitted to the IEEE Fifth Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises, Stanford University, California June 1996.

[2] J.D. Myers, C. Fox-Dobbs, J. Laird, D. Le, D. Reich, T. Curtz, "Electronic Laboratory Notebooks for Collaborative Research", " submitted to the IEEE Fifth Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises, Stanford University, California June 1996.

[3] C. Petzold, Programming Windows 3.1, Redmond, WA: Microsoft Press, 1992, chapter 13.

[4] TIFF Revision 6.0, Aldus Corporation, 411 First Avenue South, Seattle, WA 98104-2871.

[5] Graphics Interchange Format, CompuServe Incorporated, 5000 Arlington Centre Blvd., Columbus, OH 43220.

[6] T. Boutell, PNG (Portable Network Graphics) Specification, ftp://ftp.uu.net/graphics/png/.

[7] Inside Macintosh, Menlo Park, CA: Addison-Wesley Publishing Company, 1988-1993, vol. 1, chapter 6; vol. 5, chapter 4; vol. 6, chapter 16.

[8] M.K. Gschwind, ISO 8859-1 National Character Set FAQ, e-mail: mike@vlsivie.tuwien.ac.at, http://www.cis.ohio-state.edu/hypertext/faq/usenet/character-sets/iso-8859-1-faq/faq.html

This work was supported by the U. S. Department of Energy through the Distributed Collaboratory Experiment Environments (DCEE) program sponsored by the Mathematical, Information and Computational Sciences Division of the Office of Energy Research, and through the Laboratory Directed Research and Development program at Pacific Northwest National Laboratory (PNNL). PNNL is a multiprogram national laboratory operated by Battelle Memorial Institute for the U.S. Department of Energy under Contract DE-AC06-76RLO 1830.