Grounding in Multi-modal Task-Oriented Collaboration

3 Experimental setting

• The task. Two subjects play a mystery solving game: Mona-Lisa Vesuvio has been killed and they have to find the killer.The subjects play the parts of the detectives Sherlock and Hercule. They walk in a text-based virtual world (a MOO environment) where they meet suspects, ask questions about relations with the victim, regarding what they have done before the murder, and so forth. Suspects are programmed robots. The two detectives explore rooms and find various objects which help them to find the murderer. More precisely, they are told that they have to find the single suspect who (1) as a motive to kill, (2) had access to the murder weapon and (3) had the opportunity to kill the victim when she was alone.

• The MOO environment.[1] This is a standard MOO called TECFAMOO, developed in our research team.[2] The subjects move in different rooms (see figure 1), where they find suspects and objects. They talk to each other via two commands: "say..." to communicate with anybody in the same room, and "page <Player>..." to communicate with this player wherever he is. We have also augmented the standard MOO commands with others implemented specially for these experiments. This area of TECFAMOO is protected against the entry of other visitors. All actions and communications are recorded. The subjects carry a notebook which automatically records the answer to all the questions that they ask, organizing it by suspect and subject, to allow easier retrieval. In these experiments the subjects used a MOO client called Mudweller which runs on Macintosh. This client displays about 60 lines of text without scrolling back (any interaction uses several lines) and, provides a 3 line text entry pane.

• The Whiteboard. The subjects were invited to draw any representation that they felt would be useful to solve the problem. Drawings were displayed on the screens of both subjects, using a shared whiteboard (a component of the BeingThere 2.0 groupware system). This program only supports elementary drawing: boxes, lines (with or without arrows), and text frames of different colours and thicknesses. It does not include free drawing and does not enable users to edit displayed text. Users can move, remove, resize or change colour of objects, regardless of who created them. They cannot see each other's cursor. They can copy and paste in the whiteboard and also from the MOO (e.g. pasting the list of suspects). Both users see the same fixed area, there is no scrolling of the whiteboard window.The MOO and the whiteboard are side by side, splitting the Macintosh screen into two 14 X 19 cm areas. The two detectives are provided with a map of their virtual environment (see Figure 1), so that the drawn schema focus on the inquiry itself instead of on a (trivial) spatial representation of their environment. We recorded the interaction on the whiteboard (1 image per second)

• Subjects were familiarized with the MOO and the whiteboard through a training task, in which they explored a MOO area of 7 rooms, and draw a map of these rooms and their contents on the whiteboard. The subjects received 30 Swiss Francs to do both tasks.