Anne Schur, Kelly A. Keating, Deborah A. Payne, Tom Valdez, Kenneth R. Yates, James D. Myers

Pacific Northwest National Laboratory

P.O. Box 999

Richland, WA 99352

Collaborations are the essence of science, yet in our information age most scientific collaborations rely heavily on face-to-face interactions, individual actions, hands-on experimentation, and delayed communications (such as e-mail).

For remote collaborations to be as fruitful as sessions with our colleagues down the hall, we need rich communication environments that enable us to focus on knowledge tasks. We also need shared access to primary scientific resources such as instruments, analysis tools, and information sources. Electronic collaborative environments can meet these needs by providing real-time access to collaborators and shared resources.

The Pacific Northwest National Laboratory (PNNL) is designing and using an integrated suite of collaborative tools to support both research and training for more than 200 researchers at the emerging Environmental Molecular Sciences Collaboratory (EMSC) [5,8,9]. We conducted interviews and discussions with the researchers as input to the design of the collaborative suite. When the prototype suite was in place, we observed its use by scientists and intelligence analysts who were geographically dispersed in more than three locations. This article describes how these two groups used the suite notes the impacts of dispersed collaboration. We propose a taxonomy of collaboration in four broad categories that place different emphases on tool use. We also present a general model of the usersí analytical activities. Finally, we highlight some impacts of these findings on collaborative software development and deployment.

From a scientistís or intelligence analystís perspective, electronic collaboration can streamline the scientific process and change the way scientists think about working alone and with others. Our goal was to design a system that sustained work activities of small groups (2-5 people) working for a full eight hours. User and social centered participatory design approaches were used in our development process [2,7]. We identified and defined the distribution of work activities (Figure 1) and noted that scientists and analysts wanted to spend more time interpreting and analyzing data with options to confer as needed.

Figure-1. Distribution of Activities in the Experimental Research and Information Analysis

Process

Kelly, in Washington, receives from Jeff, in California, a protein sample for an NMR instrument experiment. Kelly loads the sample into the NMR, goes to her office, and starts a Collaborative Research Environment (CORE) session. Jeff remotely logs into the NMR and joins the session. Jeff shares the NMR console display in a TeleViewer window. He and Kelly discuss the experimental setup, and Jeff types in the final parameters. When both are satisfied, Jeff types the command to begin the experiment, which will run overnight.

That evening, Kelly from home remotely logs into the NMR instrument and checks the experiment's status. She also accesses the NMR project electronic notebook to read a note entered an hour ago by Jeff. She screen-captures a picture of the NMR console with the data displayed onto a notebook page and then logs out of the NMR instrument. The electronic notebook automatically sends e-mail, alerting Jeff that Kelly has added something to the notebook.

When Jeff remotely logs into the NMR in the morning, the experiment has completed. Jeff saves the data to a file on the NMR disk. Before beginning the next experiment, Jeff wants comments from his boss, Dave, so Jeff initiates a CORE session. Jeff also sends e-mail to Kelly, letting her know so she can join if she wishes. Jeff telephones Dave in his office and asks him to join the CORE session. With the NMR display shared in the TeleViewer window, Jeff discusses the setup with Dave. Dave offers suggestions, and Jeff types the appropriate parameters for the next experiment.

Jeff and Dave see text come up in their "chat" windows. Kelly has joined the session and seen the TeleViewer images from the NMR instrument. She suggests another change and asks them to look at the TeleViewer where she loads a protein molecule image. Kelly rotates the molecule while explaining her point. What she says about lengthening the delay time in the NMR experiment is plausible to Jeff and Dave based on the molecule's configuration. Jeff types in the new parameter and starts the experiment. Kelly leaves the session.

Jeff and Dave remain to discuss the data collected the previous night. Jeff uses the whiteboard, pasting a picture of the recently saved data set and draws annotations on it. Dave circles some faint but important peaks in the data that Jeff had missed. Jeff records their discussion by capturing the whiteboard image into the NMR notebook. An automatic alert goes to Kelly.

Four types of collaborations occurred during the use of the collaborative suite. These types of collaborations can form a taxonomy that can be used to classify the roles that participants play within a collaboration:

1. peer-to-peer -- Researchers employ common training and vocabulary and work closely together through prolonged visits to sites with necessary resource(s), such as a scientific instrument. The researchers wanted collaborative tools that would enable them to share instrument control, sketches, and raw data files without site visits.
2. mentor-student -- Mentors use prepared materials and perform live demonstrations in a lecture style to teach topics such as data acquisition and analysis techniques to students ranging from undergraduates to postdoctoral fellows. Mentors observe students and provide direction as needed. Direction can be highly interactive: talking, showing what needs to be done, and sharing the activity.
3. interdisciplinary -- Researchers do not share a common background and often must translate results into terms that each can understand. For instance, a theorist may calculate molecular structures, while an experimentalist uses laser spectroscopy to make experimental structural measurements. These exchanges require summaries and results rather then raw data.
4. producer-consumer -- Researchers provide data as input to others of different disciplines who use the data to achieve very different goals. Researchers frequently know little about what others do with the data. Virtual collaboration offers opportunities for communications between groups that traditionally have minimal contact, and for new ideas to be fostered between them.

The Collaborative Research Environment (CORE) provides a loosely integrated suite of Internet collaboration tools that appear as web-browser extensions that can be run cross-platform. The goal established when designing the CORE was to develop a system that would support the identified workflows, activities, and different collaboration types. Traditional tools of chat, audio/video conferencing, whiteboard, and file transfer tools were provided, along with a set of newer tools:

• Shared Computer Display (TeleViewer) -- Participants can view any program displayed on any individual's monitor, thus turning noncollaborative applications into collaborative ones.
• Notebook -- An electronic version of a paper laboratory notebook has extra capabilities: distributed access to data, automated data entry, searching, and other information processing not possible in a traditional notebook.
• Web browser synchronization -- When one user displays a new web page, all linked browsers automatically follow.
• Shared instruments -- Remote instrument control.

Although we followed the use of CORE by many groups, we focused on two groups: 1) experimental researchers determining the structures of proteins by using an NMR spectrometer and 2) intelligence analysts working in the area of non-proliferation to detect nuclear material. We obtained feedback on the use of CORE and the changes it provided in work practices through follow-up interviews, demonstrations followed by discussion, and observation of groups using CORE with scenario walkthroughs and unscripted work. Users could also forward their comments to developers via e:mail or telephone.

Although a session primarily exhibited one type of collaboration, the character of a collaboration could shift from one type to another during the session. A group could move from a producer-consumer collaboration to a more interactive interdisciplinary one. Such role shifting was not intentional but occurred naturally as participants used the collaborative tools.

Some researchers were concerned about how their roles on research projects would change. For example, researchers local to instruments voiced concerns about becoming technicians for remote users and no longer sharing physical maintenance tasks. In fact, as they used the environment, they found they had more time available to pursue their own science interests.

As the scientists and analysts used CORE two to three times a week over a one-month period, they reported a shift in their workload distribution (Figure 1). They reported more time spent on analysis for themselves and with others, and less time on collecting and transmitting data (by facsimile machine or FTP) to others. They could also confer with others for as long as necessary with fewer constraints by place, time, and planning. The phone was usually chosen for audio because the Internet did not reliably offer clear speech.

The focus on analysis accompanied a shift in the tools used. Many users initially used the collaborative tools in accustomed ways: teleconferencing and telelectures. Gradually users elected to forego face-to-face video and used the TeleViewer intensively to share data. We did not attribute this shift to the ease of tool use. Rather the users realized that they could be more productive by using the tools in ways that were different from their current practices and culture [11]. In practice they were familiar with teleconferencing, but with respect to culture they were initially reticent about using real-time tools to share data, even if they knew how. Traditionally sharing is seldom done and represented a change in group process.

Workflow between knowledge activities did not occur linearly but rather opportunistically (Figure 2). Conferring among individuals occurred at any time in the process. These professionals required that many traditionally disparate technological tools and resources be brought together at a single moment in time to ìdo science" or "do analysisî in sustained conferral with others. At those times several different activities occurred, in parallel and with a variety of feedback loops between different tasks, so the discrete activities became parts of a single socially distributed cognitive system.

Figure-2. Socially Distributed Cognitive System for the Experiment Scientific Process

(Collaboration can occur at any time)

When working alone, users typically divide their attention between the computer and the task. Collaboration adds a third element, the participants, that competes for the userís limited attention. To reduce the need for the user to attend to the technology, we sought design solutions that moved technology management to the computer. Based on users' actions, the system inferred users' intentions and made context-sensitive responses to the users' conscious high-level actions. For example, a one-click method to start or join CORE from a web page is provided (Figure 3). A default automatically provides all tools unless the user excludes particular tools by selecting checkboxes. Thus users pick capabilities appropriate for their work unaware of technology such as the connection syntax of individual tools, port numbers, firewalls, or internet addresses.

Figure-3. Single Action Method to Start or Join a Session

Provide a Sense of Place

Session participants found information about the physical location of each session member unimportant. This may be because our collaborations were primarily knowledge work rather than meetings [3]. Information items critical to providing a sense of place included:

• the session name, to serve as the anchor of place
• the collaborative tools used in the session (participants adjusted their communication styles according to tools used)
• the presence of people entering and leaving a collaborative space, and
• the designation of session leaders, if applicable.

CORE provided the first three items on the start/join screen. Activity flow would have improved if this information were accessible on demand, such as on a pop-up status panel.

Rhythm/Pace

Times between the initiation of events and their experienced results were used as cues to establish team rhythm. Any inability to make time predictions made it difficult for team members to establish a rhythm to work smoothly together. Frequent indicators of pace breakdown were participants "trampling over each other" or repeating themselves. When images were part of a CORE collaboration, the status of images sent and their arrival was important. This was probably because the image was central to the message. When the TeleViewer and WebTour were used, the team worked hard to set and maintain an efficient pace. For the WebTour, sending and receiving web-page cues were provided.

Leadership

Cues indicating when the leader had exclusive control of the collaborative tools were important to group members.

Summary Views of Multiple Instances of a Single Tool

Users often created multiple instances of the same tool yet found it difficult to keep track of the resulting windows. Many times users wanted to put them out of the way, view a summary of all the instances of a tool, and control their occurrence from a single point. Then as appropriate, the scientists could immediately switch contexts without diverting their attention from the knowledge task(s). Using the whiteboard as a test case, we built a working prototype called the ìMultiWhiteboard (Figure 4) to provide the capabilities they requested.

Figure-4. MultiWhiteboard Control and Status Information Used for Management of

Whiteboards.

Users performed more than two activities in parallel rather than in linear sequences. Traditionally, scientists and analysts talk on the phone and work at their computers or perform activities at their desks. During a collaboration, especially when the video was turned off, a few intelligence analysts talked on the phone, worked within a session, worked alone at their computer, and performed a task on their physical desktop. Observers had the impression that if the analysts had had the capability to join multiple sessions, then they would have become active members of more than one session and continued to perform all the other activities in parallel.

All groups go through a process to form a mature team. CORE users were no exception. In the physical world, group members usually know when to say or show something and how to indicate that a communication event is completed. In CORE's electronic environment, the first stage of team development, ìTesting/Forming" behavior, focused on forming similar skills for basic communication. Once these skills were established, workflow between group members moved smoothly. This was manifested by pictures and words "not trampling over another," as one scientist described it. Verbal cueing was the primary means of controlling workflow pace. During the "forming" phase, face-to-face video was important in facilitating the socializing behavior [6]. The group swiftly "formed" and moved to "performing." As they did, the video became less important and was ultimately reserved to show examples during discussions or to introduce new people.

Vision is a primary sense through which people interpret and experience the world, but as the group matured, face-to-face interaction (video use) declined. When the Internet audio degenerated, or the phone was not available, activity slowed significantly and collaborators started to work alone. We believe verbal exchanges were important because they provided the instantaneous feedback traditionally provided visually in the physical world. The analysts developed distinct voice protocols to control the workflow. The dialogue, for the analysts resembled that of airline pilots. When an exchange was complete, the speaker said "Over." Those listening indicated to other team members they had received the message by saying "Check." Users did not like using vocal cues to provide feedback about whether an image had arrived or not because it felt "unnatural."

In the physical world, visual cues are typically used to indicate changes of activity, topic, or pace. In our observations and listening, we noted that speech and cursor movement were used to signal such transitions. Outside observers may perceive that nothing is happening, yet these "white spaces" are important to smooth shifts in the work focus and flow.

From our experience we extrapolate five findings that could be broadly applied.

• Establishing and sustaining a social dialogue is critical for any collaboration. Without it, participants, in the worse case, cannot convey and receive, and the collaboration fails. Five guiding principles influenced our approach to addressing social interactions:
  1) Support knowledge work activities, including cognitive aspects.
  2) Enable transparent transitions between knowledge activities.

• Scientific and intelligence analysis work is performed in a framework This framework represents a process in which numerous knowledge activities can be associated with any component in the process, thus forming a socially distributed cognitive system. This system can change how professionals perform work. The scientist and analyst can move away from a linear approach to doing work and can perform more iterations of the process before drawing conclusions. These iterations may encompass all components of the process, but the cycles may be shorter and collectively may have the potential to enhance the analytical process. This framework could serve as a baseline to systematically gain insights about knowledge work in the context of electronic collaborative environments.

Design decisions should consider the context of social situations. Attending only to human-computer interface (HCI) issues for individual tool operation creates usability deficiencies in collaborative environments. For example, users often switched between tools as they worked. Attention to the task was sustained when they could move between tools without thinking. Providing features that were consistent and compatible with each other between tools assisted in this goal. HCI issues resolved in the context of workflow gave benefits. For example, the HCI had to be flexible for any possible knowledge activity to be performed in any tool at any time (Figure 5). Then users could unconsciously switch activities (from authoring to information sharing) and the purpose for which they were performing the activities (collecting, preparing for analysis).

Figure-5. Three Team Members Working Between Parallel Coordinated Workloops

ïCulture and trust play significant roles in the acceptance and success of a technology. Usability of a system is not enough. When introducing new technology, addressing cultural issues and the impact of change on a personís role in an existing social structure is important.

ïTimeliness (from real-time, just-in-time exchange and discussion of data and information) can increase productivity.

We plan objective evaluations using ethnographic [12] techniques with appropriate performance measures to determine impacts of this new method of performing science and analysis. Ideally we would use longitudinal studies to assess if problems are solved faster, costs and time are saved, quality is improved, work practices change, and the content of research papers changes. For example, are the results presented differently?

We will investigate the mechanisms involved in creating and maintaining rhythm and enabling self-regulation between team members to understand how interactions within media spaces can be perceived as natural.

Many individuals are contributing to the Environmental Molecular Science collaboration effort, [10] including students.

This work was supported by the U.S. Department of Energy through the Distributed Collaboratory Experiments (DCEE) program [4] sponsored by the Mathematical, Information and Computational Sciences Division of the Office of Energy Research, and through the Laboratory of Directed Research and Development Program at Pacific Northwest National Laboratory (PNNL). PNNL is a multiprogram national laboratory operated by Battelle for the U.S. Department of Energy under contract DE-AC06-76RLO 1830.

1. Abel, M.J. Experiences in and Exploratory Distributed Organization Intellectual Teamwork. Galegher, J., Kraut, R.E., and Egido, C., Eds. Intellectual Teamwork: Social and Technological Foundations of Cooperative Work. Lawrence Erlbaum Associates Hillsdale, NJ, 1990 pp. 489-509

2. Blomberg,J., Giacomi, J., Mosher, A., and Senton-Hall, P. Ethnographic field methods and the relation to design. Participatory Design: Principals and Practices. D. Schluler and A. Namioka, Eds. Lawrence Erlbaum, London, 1993 pp.123-155.

3. Bly.S.A., Harrison, S.R., and Irwin, S. Media Spaces: Bringing People Together in a Video Audio, and Computing Environment Communications of the ACM, 36, 1(1993), pp.28-46.

4. DOE Distributed Collaboratory Experiment Environments. (see http://www-itg.lbl.gov/~jtchew/DCEE_Overview.html).

5. Emerging Environmental Molecular Sciences Collaboratory. (see http://www.emsl.pnl.gov:2080/docs/collab/

6. Goffman, E., Interaction Ritual Essays on Face-to-Face Behavior. Penguin Books 1967

7. Greenbaum, J. and Kying M. eds Design at Work: Cooperative Design of Computer Systems. Lawrence Erlbaum Associates Hillsdale, NJ,1991.

8. Kouzes, R.T. The Collaboratory: Creating R&D laboratories Without Walls. in Electronic Laboratory Notebooks and Collaborative Computing in R&D: Social Legal, Regulatory, and Technology issues, R. Lysakowski, Eds. Team Science Publishing, Sudbury, MA to appear 1997.

9. Kouzes, R.T. Myers, J.D., and Wulf, W.A. Collaboratories: Doing Science on the Internet. In IEEE Computer, 29, August 1996, pp. 40-46.

10. PNNL Collaboratory Development Team. (see http://www.emsl.pnl.gov:2080/docs/collab/CollabTeam.html)

11. Schatz, B.R. Building an Electronic Community System. Journal of Management & Information Systems, 8, 3 (Winter 1991-1992), 1992, pp. 87-107.

12. Simonsen, J. and Kensing, F. Using Ethnography in Contextual Design. In Communications of the ACM, 40, 7(July 1997), pp. 82-88.