Figure 1. Multimedia information about lunar habitat design in PHIDIAS.
| Artificial Intelligence in Engineering Design, Analysis, and Manufacturing (AIEDAM), 11, 2 | April, 1997 | pp. 141-154. |
Frank M. Shipman III
Department of Computer Science
Texas A&M University
College Station, TX 77843-3112
E-mail: shipman@cs.tamu.edu
Raymond J. McCall
College of Environmental Design & Institute of Cognitive Science
University of Colorado, Boulder
Boulder, CO 80309
E-mail: mccall@spot.colorado.edu
Abstract: Design rationale is a topic which implies different things to different people. To some it implies argumentation and frameworks for argumentation. To others it implies the documentation of design, like that required for many types of industrial or government work. Still others describe design rationale as the capture and potential reuse of normal communication about design. These perspectives of design rationale use different representations which influence their ability to capture and to retrieve and use information. We propose an integrated approach to design rationale where design communication is captured and, over time, incrementally structured into argumentation and other formalisms to enable improved retrieval and use of this information. Two systems, PHIDIAS and the Hyper-Object Substrate, are used to demonstrate: (1) how to capture and integrate a variety of design information, (2) how to support the structuring of unstructured information, and (3) how to use design information to actively support design.
The term design rationale (DR) means different things to different people. There are, in particular, at least three distinct perspectives on DR, which we will call argumentation, communication, and documentation. These perspectives differ but are not contradictory, for each aims at achieving a different goal and therefore uses different--albeit overlapping--approaches.
To some people, DR means the argumentation--i.e., reasoning--that designers use in framing and solving problems. It includes both the reasoning of the individual designer and the discourse among participants in a design project. This perspective is characteristic of designers and methodologists who seek to improve the quality of design by improving the reasoning of designers--in particular, by improving the arguments that they use. From the argumentation perspective, the purpose of recording design rationale is to call attention to deficiencies in that rationale and thus lead to the correction of these deficiencies. This approach was originated by Rittel [Rittel 1972], and the best-known examples of it are based on his Issue-Based Information System (IBIS) framework for argumentation [Kunz, Rittel 1970]. In the last decade a number of other frameworks have also been pursued. Some of these are described below.
Typically, those who use the argumentative approach to DR attempt to get designers to think and discuss design within a given argumentative framework. Thus, for example, many attempts have been made to use some variation on IBIS to structure the work of designers both individually and collaboratively. In the case of IBIS, this means organizing the design process around the treatment of questions, which in the method are called issues. This works as follows: Project participants propose issues. For each issue, they propose various answers--also called positions--to each issue; and then state arguments about the pros and cons of the proposed answers. Finally, a decision is made about which answer or answers to accept and reject for the given issue. Design then becomes a process of raising and deciding a set of related issues. Such structured approaches to design generally also result from use of the argumentative frameworks other than IBIS.
For some people, DR means capturing and retrieving naturally occurring communication--e.g., design discourse--among members of a project team. The goal here is recording thought that has occurred rather than shaping the thought that occurs. The recording of design communication is not meant to have any effect on the design process. It is merely meant as a way of retrospectively tracking down various information that might have been the subject of communications among participants in the design process.
In part because it does not have its own single and separate objective, the content of design communication tends to overlap the content of both design argumentation and design documentation. It lacks the clarity of the documentation approach to DR, in the sense that there is no motivation by those communicating or by recording communications to describe things for those outside the project--or for anyone other that than the ones engaging in the discourse. Recorded communication therefore typically includes all sorts of information irrelevant to the argumentation or documentation perspectives--e.g., discussions about scheduling meetings or even about the weather.
Recorded communication also tends to lack the discipline and structure of the other two types of DR. Both argumentation and documentation attempt to impose some sort of order on the record of things discussed. Thus, documentation attempts to make the write-up of results orderly even if the thinking that led to them was not. Argumentation typically attempts to make the thinking itself more orderly than in it otherwise would be, so that the design process can be improved. In recorded communication, however, people say whatever they say however they say it. Discourse among project participants might be, and often is, quite free-form, disorderly, and full of digressions. In fact, the adjective discursive, has come to mean "rambling from one subject to another" [Oxford American Dictionary 1980]. Errors of fact, logic, grammar, and, of course, spelling are bound to occur. Often the only apparent structure is the turn-taking that is normal in conversation--and even here people often interrupt each other or talk simultaneously.
To some people DR means the documentation of information about design decisions: what decisions are made, when they are made, who made them, and why. There are two things that motivate the creation of such documentation. One is to enable people outside the project group to understand, supervise, and regulate, if necessary, what is done by the project group. The other motivation is to identify and secure intellectual property generated in a project--e.g., for purposes of obtaining and defending patents.
It is characteristic of the documentation approach to DR that far less information is recorded than in either the argumentation or communication approaches. For both supervisory and intellectual property purposes it is not important to record all design reasoning. It is only necessary to document the results of that reasoning plus the immediate explanation of and rationale for these results. Possibilities considered, dead-ends pursued, basic philosophical discussions, the long trains of thought leading to results: none of these is relevant.
While it does not necessarily aim to influence decisions, design documentation does most definitely influence what is written about those decisions. In particular, the documentation is of little value if decisions are not described, explained and justified clearly. In addition, those responsible for decisions must be clearly identified. For intellectual property purposes there is also a legal requirement that the novelty and utility of ideas be explained at some point. The descriptions, explanations, and justifications given must be comprehensible to those outside the project. Thus, for example, this documentation must be self-explanatory in the sense that its explanations do not rely on undocumented background knowledge possessed only by members of the project group.
In the next section we describe these three perspectives of DR in practice with an emphasis on the representations, capture, and use of information. Section 3 argues for an integrated approach to DR to improve the support for design and the acceptability of systems to designers. Section 4 describes our experiences with two systems containing features important to such an integration.
Each of the three perspectives uses a particular type of representation for design rationale. The nature of these representations together with the ways in which they are created and used decisively influence the success of DR in real-world design projects. In this section we look at these representations and evaluate their use in practice. In particular, we describe both the successes of DR to date and the continuing failures that have severely limited its practical utility.
The representations associated with the argumentation perspective are semi-structured, using typed nodes and typed links to interconnect the nodes. For example, Rittel's IBIS includes three node types: issue, position, and argument. Nodes may be interconnected by eight link types (supports, objects-to, replaces, responds-to, generalizes, specializes, questions, and suggested-by) [Rittel 1984]. The other argumentation schemes, such as QOC [MacLean et al. 1989], DRL [Lee 1990], and Toulmin schemas [Toulmin 1958], also use this same sort of node-and-link representation. The differences among the various approaches are primarily differences in the types, names and numbers of nodes and links.
Experiences with capture of argumentative design rationale provide common results. There is widespread agreement that most design tasks produce rationale that can be adequately represented in argumentation schemas using post-generation analysis. Unfortunately, there also appears to be substantial agreement that there are significant difficulties in getting designers to use such schemas to structure their thinking while they are performing real design tasks [Buckingham-Shum, Hammond 1994]. Paradoxically, designers seem to spontaneously produce relatively well-structured arguments in natural discussion, yet have considerable difficulty in expressing their arguments when forced to use an argumentation schema. Furthermore, empirical studies seem to refute the notion that this problem results from a misfit between schema structure and the structure of the arguments [Fischer et al. 1991]. Conklin and Burgess Yakemovic have reported success in practical use of a simple system based on the IBIS representation, but in this case the capture of argumentation was facilitated by a "champion" of the method who transcribed the arguments of other group members into the system [Conklin, Burgess Yakemovic 1991]. There are few if any cases where argumentation has succeeded without such a champion.
While capture of argumentative rationale remains problematic, retrieval of relevant rationale is an area where the argumentation approach has excelled. Over the years advocates of this approach have developed a number of techniques and technologies that enable effective and timely retrieval of relevant rationale.
The most basic reason for this success is that the structured representations used in the argumentative approach provide much of the indexing needed for retrieval of rationale. Most basically, the relationships between elements of argumentative rationale--e.g., an issue's links to its answers, an answer's links to its arguments--provide a powerful means for associative indexing. This in turn provides a basis for the use of hypertext to retrieve relevant rationale. And, in fact, a number of hypertext systems were developed purely out of the urge to support frameworks for argumentation. Such systems included MIKROPLIS [McCall et al. 1981] and gIBIS [Conklin, Begeman 1988] for the IBIS framework as well as EUCLID [Smolensky et al. 1987] and SIBYL [Lee 1990] for other argumentative frameworks.
One highly effective, hypertext-based retrieval strategy for argumentation is issue-based indexing of rationale using IBIS. In this approach the conventional IBIS links are augmented by links based on "serve" relationships [McCall 1979]. These relationships connect a given issue to the other issues that need to be answered in order to answer the given issue. Issue-based indexing thus enables direct retrieval of all the recorded rationale useful for resolving any given issue. Since this approach of using "serve" relationships generally produces a tree-like structure of issue discussions, it is called Procedural Hierarchy of Issues (PHI) [McCall 1991].
The retrieval strategy called task-based indexing, extends notions of issue-based indexing beyond the IBIS method--and can thus be used for any argumentation schema. Task-based indexing has been incorporated into a number of software systems that combine argumentative hypertext with CAD-style representations of designed artifacts. These systems include JANUS [Fischer et al. 1989], PHIDIAS [McCall et al. 1994], INDY [Reeves 1993], and HOS [Shipman 1993].
There are three variants of task-based indexing in these systems. One is artifact-based indexing, which uses the graphical representations of the artifacts being designed to retrieve the rationale relevant to their design [Reeves 1993]. A second variant is critique-based indexing, which uses knowledge-based critics to "look over the shoulders" of designers as they work and detect possible violations of principles of design [Fischer et al. 1992]. A third variant is operation-based indexing, which enables designers to retrieve the design rationale relevant to specific editing operations on a given graphical object--such as moving, resizing, or replacing it [McCall et al. 1994].
In summary, the argumentation approach to DR, while having difficulty capturing information, has developed a rich collection of techniques and technologies for retrieval of relevant rationale. This success is above all due to the effort invested in structuring argumentative rationale in various ways. These ways include describing the rationale itself as a network of components of specified types--e.g., issues, answers, arguments. They also include relating rationale to the objects and task situations to which they are relevant. This structuring provides the basis for the indexing that enables effective retrieval.
The capture of communication puts few restrictions on the representation of design information, although it is necessary for the representation to fit into the natural process of design. Typically, the media and representations are chosen to match communication channels already used by designers. The variety of media includes memos, notes, drawings, electronic mail, phone conversions, and design meetings. The representations associated with such capture vary from ascii text for memos and electronic mail to audio and video for phone conversations and design meetings.
Precisely because it imposes little or no restrictions on the actual process of design communications--in particular, requiring little or no structuring or other change of representation--capture of design communication is generally not problematic. Traditionally, all communication took place orally--e.g., on the phone or in meetings--or in writing and drawing on paper or black/whiteboards. These days email is playing an increasingly important role in communication among designers. Phone-calls can be easily recorded, while voice mail already captures much oral communication. Meetings can easily be videotaped to capture oral discussion as well as writing and drawing on black/whiteboards.
For a computer system to retrieve design communication, this communication must be in--or be transferred into--an electronic form. Electronic mail messages are, of course, already in this form. Video and audio tapes, however, must be converted to digital video and audio on hard disks--a technically straightforward, inexpensive and practical process, albeit one not yet finding much use in practice. Paper-based communication, both in the form of written documents and drawings can be converted to digital form using scanners--a somewhat cumbersome, but nevertheless practicable process. Recently Gross has done work on directly capturing sketches [Gross 1996]; and Moran's group at Xerox PARC has done work on directly capturing handwritten documents [Moran et al. 1995]. This process works equally well for printed and hand-done documents. Most such documents could also be faxed directly into computers using current day desk-top technology.
Having design communication in electronic form is a necessary but not sufficient condition for its retrieval. For effective retrieval, communication must be well indexed. This indexing is often difficult and in practice it is seldom done adequately if at all. Machine-readable text, such as email, lends itself most easily to indexing. Printed documents--such as memos and reports--must, however first be put into machine-readable form. Optical character recognition still falls far short of perfection, but it is increasingly successful in converting printed text to machine-readable text. In principle, "hard-line" drawings can be scanned and then "vectorized" using special software, but this process is expensive, error prone, and generally not capable of reconstructing the semantics of the drawings. Hand-written and hand-drawn documents are fundamentally more difficult to deal with. Handwriting recognition has, to date, had only modest success--far too little for large scale practical use with handwritten documents. Sketch recognition is in an even more primitive state. In practice, all handwritten documents and all drawings, whether hard-lined or hand done, must be indexed by hand--as must photographs and other graphical documents. Voice- based communication must also for current practical purposes be indexed manually as must videos.
Overall, the communication approach succeeds in capturing design information but fails in effectively retrieving this information and putting it to use. The lack of task-based structure, like that found in argumentation structures, undermines both the ability to retrieve information when needed and the system's ability to provide design support based on the information.
The documentation perspective of design rationale differs from the previous two perspectives in that it has widespread acceptance in design practice. Designers in many large companies and defense contractors are required, either by policy or by law, to create a record detailing their role and actions in the design process.
One prevalent form of documentation is the periodic report--e.g., the quarterly report. Such reports are, of course, highly retrospective in nature. Additional kinds of reports require filling out special forms where particular subsets of project information are to be described briefly and in more or less constrained formats.
The most common instances of design documentation are paper-based. Even when computer support for documentation is available, it is generally limited to static data-entry forms and database management of the information. Recent work on dynamic forms which hide fields not applicable to the current situation and identify fields still needing attention may make the documentation task less onerous but do not substantially change the representations of the paper-based practices [Girgensohn et al. 1995].
Another prevalent, but very different, form of documentation is the laboratory notebook. These are kept by engineers and scientists in a wide range of high-technology industries. The goal in using such a notebook is to record any ideas that might constitute intellectual property together with any results that might confirm the novelty and value of these ideas. As with the form-based documentation, the computerized lab notebooks have tried to retain much of the representation of the paper-based notebooks while facilitating the use of new media, like audio and video [Gorry et al. 1991; Shipman et al. 1989].
Unlike the recording of argumentation or communication, the creation of documentation tends to occur after--though sometimes immediately after--the decision processes being documented. This is in part because the primary goal of documentation is merely to record decisions, not to influence them or the thinking processes leading up to them. Of course, the knowledge that design decisions will be made known to people outside the project group has the potential to influence those decisions. Documentation also takes place retrospectively because, unlike the recording of communication, it does not attempt to preserve the language or form in which the decisions were made. Instead, people are encouraged to write up their reasoning in a form that will be clear to people outside the project group.
While legal requirements--such as liability and intellectual property--often mandate documentation, its use can be problematic. For one thing, it is costly in money, time and effort. There are inevitably "lost opportunity costs" resulting from the spending of design resources on documentation when they could have gone into other design activities. In addition, there is the danger that what is documented is not exactly what happened. Special effort must be devoted to making sure that documentation is accurate; and there is no way for an outside observer to determine how accurate a given piece of documentation is.
As mentioned above, neither the argumentation nor the communication approach to DR has been generally successful in practice. Each has failed to solve a fundamental problem. For argumentation the unsolved problem has been effective capture of rationale. For communication the problem has been effective retrieval. Interestingly enough, the weakness of each of these approaches corresponds to the strength of the other. Thus, for communication, capture is generally not problematic, given the technologies of e-mail, digital scanning, and video and audio recording. For argumentation, retrieval is generally not problematic. Software systems that offer issue- and task-based indexing in conjunction with conventional content-based, global indexing--e.g., using keywords and freetext search--provide highly effective means of finding relevant, argumentative rationale.
It would be ideal if we could somehow combine the communication and argumentation approaches in such a way that we could use the strength of each to overcome the weakness of the other. Unfortunately, there appears to be a fundamental dilemma in the relationships between the capture and retrieval problems. That dilemma arises from the role of structured representations in these problems.
We saw that requiring designers provide structure during the capture of rationale--as is the case in the argumentation approach--typically prevents its effective capture. On the other hand, effective retrieval requires structure--particularly in the form of associative indexing--e.g., for issue-based and task-based retrieval. So it would seem at first that there is an inevitable trade-off between effectiveness of capture and effectiveness of retrieval: increasing structure to aid retrieval diminishes the effectiveness of capture--and vice versa. Such an inevitable trade-off would be deeply problematic for the future hopes of DR, since success of DR ultimately requires both effective capture and retrieval.
Fortunately, when we consider the documentation approach to DR, we can see that there is something wrong with this notion of an inevitable trade-off between effectiveness of capture and retrieval. For documentation effectively captures rationale in structured form and then uses this form for effective retrieval--precisely what argumentation aims but fails to do. Of course, documentation does not work as well as it might: it is costly, time consuming, and deals with less rationale than argumentation aims to. Nevertheless, documentation remains the one type of DR that can justly claim to be effective and widely used in real-world design projects.
It is easy to see why documentation has effective retrieval: it captures rationale in a structured format specially devised to facilitate retrieval of relevant information. The crucial question is therefore this: Why is documentation effective using structured capture when argumentation is not? We believe the most basic reason is simply this: documentation, unlike argumentation, makes no attempt to force designers to be explicitly aware of their thought processes while they are designing. Furthermore, it does not interrupt the design process or burden designers with the task of categorizing and explicitly interrelating their thoughts as they are designing. Instead, documentation of design decision making always takes place after rather than during decision making--although sometimes immediately after.
Two other factors also contribute, though in a more minor way, to the success of documentation and the failure of argumentation in practice. One is that documentation, by its very nature, tends to be motivated by legal constraints--such as contractual obligations and intellectual property concerns--while argumentation seldom has such constraints. The other is that documentation attempts to deal with smaller quantities of rationale than argumentation attempts to deal with.
The first and most basic lesson of this analysis is that since there is no inevitable trade-off between the effectiveness of retrieval and capture, there must in principle be a way to have both be effective. The second lesson is that for capture to be effective, the structuring of design rationale must happen after, not during capture. The problems we are then left to solve are those of how to capture and subsequently structure large quantities of argumentative rationale and to do so in the absence of the sort of legal constraints that motivate documentation.
Our basic approach in attempting to solve the problems of design rationale is threefold:
In using the communication perspective for capture our approach is to create software that integrates all the major kinds of digital media. Thus, we seek to incorporate e-mail text, word processor documents, video of meetings, voice mail conversations, scanned documents, photographs, and CAD drawings into a single, hypermedia database. By integrating these various media, we can integrate and interrelate all major forms of human communication. Our goal is thus to create an integrative information environment that serves as a repository for all communication within a design team. In addition, we aim to route as much team communication as possible through the system, so as to avoid the extra step of importing communication into the system. This integrated repository for information guarantees the retrievability of all captured design communication.
Our goal in retrieval is to use the various forms of task-based and content-based indexing that have proven useful in our prior systems that support retrieval of argumentation. We have also sought to augment these with new kinds of agent-based retrieval.
For structuring communication for retrieval we use an approach that we call "incremental formalization" Shipman, McCall 1994]. In this approach structure emerges gradually during the use of the system on a demand driven basis. Partial formalization--i.e., structuring--of data tends to suggest additional possibilities for formalizing data; thus the formalization of the data tends to "snowball" during use of the system.
Our central goal in integrating communication and argumentation perspectives is to make argumentation workable in practical design. A natural--and in fact inevitable--side-effect of using communication capture to make the argumentation approach work is that argumentative retrieval makes the communication approach to DR work as well. Communication is captured, structured and then retrieved as argumentation. Thus, the capture problem is solved for argumentation and the retrieval problem is solved for communication. Furthermore, in the process, much of the information needed for the documentation approach to DR is automatically collected and made retrievable. While this is unlikely to eliminate all the work of documentation, it could substantially reduce the designers' effort while increasing the accuracy and quantity of design documentation.
There are a number of issues that arise in designing systems to support an integrated approach to design rationale based on the three-fold approach outlined above. Here we consider three: 1) how to capture and integrate a variety of types of design communication in a single system, 2) how to support the emergence of structure in design communication, in particular, the kind of indexing structure useful for retrieval, and 3) how to retrieve and use this rationale to actively support design.
There is a large variety of design communication to capture and integrate. As mentioned above, text, audio, video, sketches, and CAD diagrams are just some of the media of design communication. In addition to the problem of multiple media, a system's representation for this design information must be flexible enough to allow both the unstructured information associated with design communication and the structured content of argumentation and documentation.
The flexibility in media and structure required is well matched to the capabilities of hypermedia -- frequently implemented as "chunks" of information in various media interconnected through navigable links. We will discuss our experiences capturing and integrating design information in two hypermedia systems we have worked on over the last seven years: PHIDIAS and the Hyper-Object Substrate.
PHIDIAS combines computer-aided design, argumentation, and multimedia capabilities [McCall et al. 1994]. PHIDIAS, like its predecessor MIKROPLIS [McCall et al. 1981], supports the authoring, indexing, and retrieval of hierarchically-structured discussions about design issues using the PHI variation of the IBIS methodology.
In addition to supporting the use of text, PHIDIAS information can include sketches, CAD designs, and audio and video clips. The interconnection of this information takes the form of hypertextual links between chunks of information. Argumentation is represented by assigning types to the links within this hypermedia structure. Designers can define new link types to modify the argumentation model provided or to create alternative representations. The CAD definition of the design artifact is represented in the same hypermedia structure so as to be interlinked with the argumentation about its design.
Figure 1 shows information on the design of a lunar habitat in PHIDIAS. The main window contains subwindows displaying a manipulable three-dimensional CAD design (upper left), issue-based argumentation about the design (center), and photographs and video clips related to the design.
Figure 1. Multimedia information about lunar habitat design in PHIDIAS.
To locate and make use of information in the linked structure, PHIDIAS includes a query language, called PHIQL. PHIQL is the basis for much of the interaction with the information space. Designers may use the query mechanism directly, but frequently execute stored queries attached to particular designer actions. For example, in Figure 1 a mouse selection in the indented view of the argumentation was used to expand the issue "How has the table been placed in previous designs?" This expansion executes a PHIQL query locating the answers to this issue, which are then used to update the window's display. In this case the query locates CAD designs, graphical renderings, and videos of mockups. PHIDIAS uses a hypermedia structure and a query language to support the integration and retrieval of multimedia and multistructured design information.
A domain-independent prototype, the Hyper-Object Substrate (HOS), extends a basic hypermedia representation with features more common to knowledge-based systems. In HOS, all information is kept in a single object store supporting a number of object types for representing text, line drawings, composites and spatial arrangements, and for dynamically computing displays and taking actions based on information currently in the object store and available through the Unix shell.
Because design artifacts and design communication are represented within a single, conherent object store, annotations need not be separate from the design. Annotation of the design facilitates artifact-centered discussion among designers not possible when artifact and communication are represented and interacted with separately [Reeves, Shipman 1992].
Figure 2: Communication and a piece of the design shown in the construction kit (back window) is used as an example in an issue base (front window).
Figure 2 shows an example from XNetwork, an environment supporting computer network design, using a combination of natural communication and design in an argumentation structure [Fischer et al. 1994]. The rear window in the figure shows a HOS construction kit displaying a piece of a network design along with textual annotations discussing that design. The front window shows a generic view window containing an argumentation structure which uses a piece of the design and discussion as an example in this structure.
Even with the integration of communication and artifact, it is unreasonable to expect designers to cease using current communication channels. To take advantage of existing communication practices, HOS includes facilities for importing electronic mail and USENET news files. Thus, network designers can continue their existing practice of using email to inform one another on the progress of tasks and later include this information in their HOS design space.
As the examples from both PHIDIAS and HOS show, the integration of media and structure creates the potential for a more integrated design environment. The combination of the various communication media with the representation of design artifacts reduces the fragmentation and duplication of information among multiple systems, and is more likely to fit into a designer's existing work practices. Both systems also show how, once captured in the system, information has the potential to be reused to meet argumentation and documentation goals.
As the problems associated with the communication perspective of design rationale indicate (discussed in Section 2.2), capturing information alone is insufficient to provide useful design rationale. Some structure is crucial for adequate retrieval and use. To gain this structure we have investigated methods to enable and support the gradual emergence of formalized information, a process we call "incremental formalization" [Shipman 1993]. This process allows for formalization to be demand driven, thereby not requiring effort of designers prior to their being ready to put information in a more structured representation, and to be actively supported by the system, thereby reducing the difficulty of formalization for designers [Shipman, McCall 1994].
In describing this work we will first discuss HOS's characteristics which enable this process. Second, we will discuss the potential for systems to reduce the burden on designers by actively supporting this formalization process.
The particular combination of hypermedia and knowledge-based system functionality in HOS is meant to facilitate the incremental addition of formalism to any piece of information in the system. Every HOS object may have a navigational link to a view object. Also, each object includes an extensible set of attributes and relations. Attributes and relations may be inherited between any pair of objects using a variation of prototype inheritance [Lieberman 1986] which allows inheritance networks to form generic graphs.
This flexible representation can support communication, argumentation, and even more formally represented domain knowledge, avoiding the need for designers to move between systems in order to work with different types of information. Over time, communication between network designers can be migrated from informal textual annotations to either structured rationale about the design or to modifications to the underlying formal representation of the domain.
Experiences from the development of XNetwork and the use of HOS for knowledge-based system class projects in the domains of archeology and neurosciences show that the flexibility for incrementally adding and formalizing information is useful for the rapid prototyping and modification of knowledge-based applications. Our users could add textual information to the system without difficulty. Once information was in the system, they would add navigable links and other structure to it based on their current goals. When it served their current goals, users would formalize domain knowledge in attributes and relations. Not only was the resulting formalization demand-driven, but the domain representation also could change with the evolving goals of the system.
The use of a single underlying representation allows HOS to support the integration design communication and design artifacts without designers needing to consider information format and initial representation. For task-based structure to be added to the design communication it is critical that the representation enable an incremental approach to formalization.
HOS actively supports incremental formalization through the suggestion by the system of formalizations for addition to the information space. These suggestions are based on simple text analysis and the existing formalized knowledge in the information space. Because the inferences are used for suggestions, they need not be completely accurate to aid the designer. Suggestions provide a starting point which can be edited, thus changing part of the process of formalization from the difficult act of initial creation to reaction and modification.
Figure 3: Email about workstation along with suggested attributes and relations in HOS.
An example of the type of active support provided in HOS is shown in Figure 3. Here an email message from a network user to a network designer has been imported near the part of the design discussed. The property sheet shows a number of attributes and values taken from the header of the imported email message along with some additional attributes suggested by the system (those shown in bold). These suggestions are generated by a string search for domain terms already known by the system. This simple method for generating suggestions was chosen to focus the investigation on the tools supporting formalization rather than text analysis algorithms.
Our experiences with the use of HOS indicate that sensible suggestions for formalization can be provided in small domain-oriented information spaces. Even when not entirely accurate, suggestions can provide short-cuts to adding intended formal information. In addition, because designers develop expectations regarding the suggestions, the suggestion mechanisms has the added benefit of informing designers of new information and pointing out when designers' assumptions no longer match the information in the system.
While not solving the inherent problems with accessing informal information, HOS does reduce the burden of initial entry of information so that it may be used for communication between designers. Once in the system, HOS enables the development of demand-driven structure and uses suggestions to aid designers in the formalization of domain information.
We have begun considering how similar suggestion mechanisms can be applied to the other media of design communication. Our work on VIKI looks at the use of recognized implicit structure from the spatial layouts of objects to generate suggestions for formal structure [Shipman et al. 1995]. Each medium poses unique problems for recognizing potential structure to suggest for formalization, but the success with suggestions in these two media demonstrates an initial feasibility for future design environments to build on.
So far we have discussed system functionality associated with acquiring a variety of design information. We will now turn our attention to ways in which systems can make use of this design information to better support design.
One method of making use of acquired information in the context of design are critics [Fischer et al. 1992]. Critics "look over the shoulder" of the designer and attempt to recognize when particular aspects of design are suboptimal or the designer might otherwise benefit from being pointed at particular design rationale.
As discussed in Section 2.1, the specific nature of the critics and their associated rationale allows systems to provide the designer with contextualized support. Critics can either be left active, and thus providing information, throughout the design process, or execute only when requested by the designer. Figure 4 shows a critic in PHIDIAS informing the designer of the need for an inventory management station in the galley of the lunar habitat.
Figure 4. A critic warning in PHIDIAS points out needed functionality in the galley.
Besides providing the designer with initial information, such as the need for the inventory management station, the critic provides access to rationale associated with this design issue. The designer may also decide to argue with the critic, providing rationale about why the critic suggestion is inaccurate in this case.
PHIDIAS uses the direct connections between design artifacts and design rationale, along with features of the current design situation, to determine when to provide information and what information to provide. In PHIDIAS, critics not only aid the act of design, but also promote the capture of design information and its use as argumentation.
In HOS, we have generalized on our concept of critic to aid in the problem of communication and coordination between designers. Similar to the original notion of a critic, this work focuses on allowing members of a development team to work separately but to be alerted to the existence of potential overlaps or conflicts between their work and the work of others.
Agents are represented in HOS as a combination of a trigger, a query, and an action, similar to agents in OVAL [Malone et al. 1992]. The trigger specifies when the agent is active. The query determines what information will be sent on to the agents action. Actions can vary from collecting the located information in a particular HOS page to adding a bookmark or presenting the designer with a moded dialog.
Figure 5. Designers create agents in HOS by selecting triggers, queries, and actions.
Designers can create or modify agents in the "agent editor", shown in Figure 5. The agent editor provides popup menus containing a set of triggers and a set of actions from which the designer may choose. After selecting a particular trigger or action, the designer is asked to specify extra information needed for that trigger or action. For example, when a designer selects the action to "present a message" or "add a suggestion to the bookmark list" the system asks what message should be displayed or what view should be added to the bookmark window.
The query definition area within the agent editor is similar to the property sheet used to attach attributes to objects and to create relations between objects in HOS. An implicit conjunction is used when multiple attributes or values are defined in the query section. This interface limits the expressiveness of the query to the location of objects matching attribute and relation patterns but allows the transfer of skills acquired in using the property sheets.
Our experiences with HOS have shown that agents have the potential to play an important role in supporting communication. In XNetwork and the class projects agents were used to deliver information based upon recent changes to the information space made by the users. Besides alerting designers to potential problems, these agents can act as reminders for unfinished tasks, such as the need to document a recent change, or act as an intermediaries when discussing a particular design policy or strategy with other members of the design team.
Traditionally, communication, argumentation and documentation are frequently discussed and represented as artifacts external to the design. To make this information useful during design, both critics in PHIDIAS and agents in HOS are meant to provide designers with information when they need it but do not know it. By providing information to designers in the context of design, these mechanisms make the information serve the design, and ultimately the original goals of the three perspectives discussed.
We have discussed three perspectives on design rationale and their relative advantages and disadvantages. The argumentation perspective represents design information as structured arguments. Experience indicates it is difficult to acquire information in this representation but the resulting rationale may be retrieved through a combination of issue-based, task-based, and critic-based indexing.
The communication perspective of design rationale uses a variety of representations to capture the natural communication that takes place as part of the design process. This approach can easily acquire large quantities of design information, but the variety of media and unstructured nature of the content makes retrieval and use of this information difficult.
The notebook and form-based representations of the documentation perspective are an example of the successful use of structured representations. The success is due in part to the specific goal of documentation -- information recorded is meant to answer specific pre-specified questions for people not part of the design team.
Looking at the relative strengths of these perspectives it appears beneficial to build systems which take an integrated approach to design rationale. In particular, combining the capture of design communication with the support of more structured argumentation has the potential to solve the acquisition problems associated with argumentation and the retrieval and usage problems associated with communication.
To support such an integrated approach, systems need to facilitate the capture of design information, support the incremental structuring of this unstructured information, and promote the use of this information. PHIDIAS and HOS are two prototype systems instantiating these characteristics.
PHIDIAS combines multiple media of communication with argumentation and CAD structures. Argumentation structures can include video, graphical, and audio information in addition to text and the structures may be directly connected to features of the design artifact and its components. PHIDIAS's query language and critics support the retrieval and use of the argumentation and other information.
HOS adds knowledge-based system features to a hypertext representation to enable and support a process of incremental formalization. Designers already communicating on-line can import electronic mail, USENET news, and plain text into their HOS information space. Once in the system, textual information is analyzed and suggestions for possible formalizations are provided. Similar to critics in PHIDIAS, agents in HOS support communication between designers and may act as reminders of progress on design tasks.
While neither PHIDIAS or HOS contains all the features identified with the integrated approach to design rationale, they do provide examples of how this integration may occur. Future work is required to combine these features into a single system.
[Buckingham-Shum, Hammond 1994] Buckingham-Shum, S. and Hammond, N. Argumentation-Based Design Rationale: What Use at What Cost?, International Journal of Human-Computer Studies, 40, 4, 1994, pp. 603-652.
[Conklin, Begeman 1988] Conklin, J., and Begeman, M. "gIBIS: A Hypertext Tool for Exploratory Policy Discussion." In Proceedings of the Conference on Computer-Supported Cooperative Work (CSCW'88) (Portland, Oregon, Sept. 26-28). ACM, New York, 1988, pp. 140-152.
[Conklin, Burgess Yakemovic 1991] Conklin, E.J. and Burgess Yakemovic, K.C. A Process-Oriented Approach to Design Rationale, Human Computer Interaction, 6, 3-4, 1991, pp. 357-391.
[Fischer et al. 1989] Fischer, G., McCall, R., and Morch, A. "JANUS: Integrating Hypertext with a Knowledge-Based Design Environment." In Proceedings of Hypertext `89 (Pittsburgh, Penn., Nov. 5-8). ACM, New York, 1989, pp. 105-117.
[Fischer et al. 1991] Fischer, G., Lemke, A., McCall, R., and Morch, A. Making Argumentation Serve Design, Human Computer Interaction, 6, 3-4, 1991, pp. 393-419.
[Fischer et al. 1992] Fischer, G., Lemke, A.C., Mastaglio, T., Morch A. The Role of Critiquing in Cooperative Problem Solving, ACM Transaction on Information Systems, 9, 2, pp. 123-151.
[Fischer et al. 1994] Fischer, G., McCall, R., Ostwald, J., Reeves, B., and Shipman, F. "Seeding, Evolutionary Growth, and Reseeding: Supporting the Incremental Development of Design Environments." In Proceedings of Human Factors in Computing Systems (CHI `94), 1994, pp. 292-298.
[Girgensohn et al. 1995] Girgensohn, A., Zimmermann, B., Lee, A., Burns, B., and Atwood, M. "Dynamic Forms: An Enhanced Interaction Abstraction Based on Forms." In Proceedings of INTERACT '95, 1995, pp. 362-367.
[Gorry et al. 1991] Gorry, G.A., Long, K.B., Burger, A.M., Jung, C.P., and Meyer, B.D. The Virtual Notebook System: An Architecture for Collaborative Work, Journal of Organizational Computing 1, 3 (1991), pp. 233-250.
[Gross 1996] Gross, M.D. The Electronic Cocktail Napkin - computer support for working with diagrams, Design Studies, 17, 1, 1996.
[Kunz, Rittel 1970] Kunz, W. and Rittel, W. Issues as Elements of Information Systems, Working paper 131, Center for Planning and Development Research, University of California, Berkeley, 1970.
[Lee 1990] Lee, J. "SIBYL: A Qualitative Decision Management System." In Artificial Intelligence at MIT: Expanding Frontiers, P. Winston, S. Shellard, Eds. The MIT Press, Cambridge, Mass., 1990, pp. 104-133.
[Lieberman 1986] Lieberman, H. "Using Prototypical Objects to Implement Shared Behavior in Object-Oriented Systems." In OOPSLA 1986 Conference Proceedings (Portland, Oregon, Sept. 29-Oct. 2). 1986, pp. 214-223.
[MacLean et al. 1989] MacLean, A., Young, R., and Moran, T. "Design Rationale: The Argument behind the Artifact." In Human Factors in Computing Systems, CHI '89 Conference Proceedings (Austin, TX). ACM, New York, 1989, pp. 247-252.
[Malone et al. 1992] Malone, T.W., Lai, K.Y., and Fry, C. "Experiments with Oval: A Radically Tailorable Tool for Cooperative Work." In Proceedings of the Conference on Computer Supported Cooperative Work (CSCW `92) (Toronto, Canada, Oct. 31-Nov. 4). ACM, New York, 1992, pp. 289-297.
[McCall 1979] McCall, R. On the Structure and Use of Issue Systems in Design, Ph.D. Dissertation, University of California, Berkeley, University Microfilms, 1979.
[McCall 1991] McCall, R. PHI: A Conceptual Foundation for Design Hypermedia, Design Studies, 12, 1 (January 1991), pp.30-41.
[McCall et al. 1981] McCall, R., Mistrik, I., and Schuler, W. "An Integrated Information and Communication System for Problem Solving." In Proceedings of the Seventh International CODATA Conference. Pergamon, London, 1981.
[McCall et al. 1994] McCall, R., Bennett, P., and Johnson, E. "An Overview of the PHIDIAS II HyperCAD System." In Reconnecting: ACADIA '94, A. Harfmann and M. Fraser (eds.) Association for Computer Aided Design in Architecture, 1994, pp. 63-74.
[Moran et al. 1995] Moran, T., Chiu, P., van Melle, W., and Kurtenbach, G. "Implicit Structures for Pen-Based Systems Within a Freeform Interaction Paradigm." In Proceedings of CHI'95, 1995, pp. 487-494.
[Oxford American Dictionary 1980] Oxford American Dictionary, 1980.
[Reeves 1993] Reeves, B. Supporting Collaborative Design by Embedding Communication and History in Design Artifacts, Ph.D. Dissertation, Department of Computer Science, University of Colorado, Boulder, Colorado, 1993.
[Reeves, Shipman 1992] Reeves, B. and Shipman, F. "Supporting Communication between Designers with Artifact-Centered Evolving Information Spaces." In Proceedings of the Conference on Computer-Supported Cooperative Work, 1992, pp. 394-401.
[Rittel 1972] Rittel, H. "On the Planning Crisis: Systems Analysis of the First and Second Generations," Bedriftsokonomen, Nr.8, 1972, pp. 390-396.
[Rittel 1984] Rittel, H. "Second Generation Design Methods." In Developments in Design Methodology, N. Cross, Ed., John Wiley & Sons, New York, 1984, pp. 317-327.
[Shipman et al. 1989] Shipman, F.M., Chaney, R.J., and Gorry, G.A. "Distributed Hypertext for Collaborative Research: The Virtual Notebook System." In Proceedings of Hypertext `89 (Pittsburgh, Penn., Nov. 5-8). ACM, New York, 1989, pp. 129-135.
[Shipman 93] Shipman, F. Supporting Knowledge-Base Evolution with Incremental Formalization, Ph.D. Dissertation, Technical Report CU-CS-658-93, Department of Computer Science, University of Colorado, Boulder, 1993.
[Shipman, McCall 1994] Shipman, F. and McCall, R. "Supporting Knowledge-Base Evolution with Incremental Formalization." In Proceedings of CHI '94, (Boston, Mass.), 1994, pp. 285-291.
[Shipman et al. 1995] Shipman, F.M., Marshall, C.C., and Moran, T.P. "Finding and Using Implicit Structure in Human-Organized Layouts of Information." In Human Factors in Computing Systems, CHI `95 Conference Proceedings, 1995, pp. 346-353.
[Smolensky el al. 1987] Smolensky, P., Bell, B., Fox, B., King, R., and Lewis, C. "Constraint-Based Hypertext for Argumentation." In Proceedings of Hypertext '87, 1987, pp. 215-245.
[Toulmin 1958] Toulmin, S. (Ed.) The Uses of Argument. Cambridge University Press, UK, 1958.