Computerising Affective Design Cognition

Dr Terence Love
We-B Centre, School of Management Information Systems, Edith Cowan University, Perth, Western Australia
t.love@ecu.edu.au

 

Introduction

This paper is intended to support the expansion of design computing into those areas that are currently the exclusive provinces of human designers, and which lie outside the scope of existing computer models of design. The paper focuses on the affective aspects of human design cognition so as to lay the groundwork for the development of computerised representations of affectively-based human designing. Computerising affective design complements those design computing approaches that focus on the quantified physical aspects of the design problem, possible design solutions, and the relationships between them. The intention of this paper is to explore how human affectively-based design might best be theoretically represented in order to facilitate its computerisation without losing its unique attributes. The paper concludes with the suggestion that affectively-based human designing might best be modeled through integrated parallel multiple affective-processing streams - a solution made more accessible by the recent availability of cheap supercomputer processing power.

Affective processing has a central role in human design cognition as shown by the frequent reference to the role of feelings in designing by many designers and design researchers (see, for example, Glegg 1971; Cross 1984; Lyle 1985; Davies and Talbot 1987; Dym 1994; Holt 1997). Affect is also foundational to beliefs, human values, and human judgment and for this reason it might be argued that models of design process that do not include affect are essentially, and unnecessarily, weakened and faulted. Until recently, however, the affective aspects of designing and design cognition have been substantially absent from formal theories of design process. Where the role of affect in cognition has been researched, it has almost exclusively been investigated in terms of emotion, with the assumption that feelings and emotions are synonymous (Susac 1998).

Exploring the affective aspect of design activity and cognition in terms of emotion is problematic (Massachusetts Institute of Technology - Affective Computing Research Area 1999A) There are practical difficulties that result from the lack of consistency between measurable data and individuals’ reported perceptions of emotions. There are epistemological difficulties related to the incommensurability of information gleaned from subjective and objective realms. There are also theoretical difficulties in devising objectively satisfactory formal representations of emotions and nuances of emotion that can be addressed in the same manner as other aspects of design problems and solutions. In the main, this problematic emotion-based approach to researching the role of affect in design cognition has also been allied to a parallel focus on the physical aspects of the designed objects (see, for example, Massachusetts Institute of Technology -Affective Computing Research Area 1999B; Massachusetts Institute of Technology -Affective Computing Research Area 1999D) - an approach that has its own epistemological problems in terms of human designing (Dilnot 1982).

This paper draws attention to an alternative perspective on affect and design cognition that focuses on feelings. This alternative perspective differentiates between feelings and emotions, and views feelings as epistemologically more radical. It assumes that feelings form the basis for definitions of emotions — a position that aligns with the classical James-Lang, Cannon-Bard, and Schachter-Singer theories of emotion (Massachusetts Institute of Technology -Affective Computing Research Area 1999C) — and regards feelings as more epistemologically suited to be the conceptual means of including affect into theories of design cognition. In this approach, feelings are viewed as an individual’s moment to moment experiencing of their physiological states. This non-classificatory physiological definition of feeling avoids many of the difficulties that beset the emotion-based approaches to including affect into theories of design cognition, and offers a new basis for computerising aspects of non-rational design cognition.

There are several ways that feelings might be defined in terms of physiology. In this paper, for economy, the feeling-based approach of Bastick (1982) is used to demonstrate how a physiologically-defined basis for affect points to a new approach to computerising design cognition. The paper starts by defining the concept of feeling in more detail, and describes some of the main elements of Bastick’s theories. Attention is drawn to the importance of closure and the role of affect in closure in human design cognition. Three descriptions are given of how passive and active forms of feeling-based affect and closure might be combined with a traditional model of design cognition. The combination of feeling-based affective process and traditional rational design cognition, points to new directions in the computerisation of human design cognition and decision-making. In particular, it suggests new ways of computerising design cognition in the areas of solution search and solution set optimisation. The paper concludes by outlining how this computerisation of affectively-based processes might proceed in order to provide better assistance for designers.

 

Feelings, emotions and affective processes

The term feelings is used in this paper to refer to all those sensations that an individual can perceive that originate in their body. Feelings include physiologically perceivable changes in, for example, muscle tone/tension, kineaesthetics, body posture, heartrate, skin sensations, temperature of different parts of body, hair erection, hormonal balances, brainwave patterns, breathing pace and depth, brain functioning, organ functioning, blood pressure. Bastick’s approach assumes that feelings in each modality are not necessarily uniform across an individual’s body. For example, different levels of tension, or blood vessel dilation, may occur in different parts of an individual’s body at the same time, and thus be different ‘feeling sets’. This is different from the approach that assumes a uniformity to the physiological basis of affect, and is based on measuring conditions at a single body site (see, for example, (Massachusetts Institute of Technology - Affective Computing Research Area 1999A).

Following on from this definition of feelings, emotions are viewed as a set of abstract concepts, e.g. love, fear, happiness, sadness, each referring to a particular set, or pattern, of feelings. The emotion contentment, for example, refers to a particular ‘set’ of feelings that includes; normal pulse rate and blood pressure, dilated blood vessels, low levels of muscle tension, slow easy and deep breathing, etc. Similarly, the emotion anger refers to a different, but also readily identifiable, set of feelings. In other words, feelings refers to the rich complex of self-originated sensation, whereas, emotions are a limited group of abstract cognitive artefacts that refer, more or less accurately, to identifiable sets of feelings.

There are four main reasons why feelings, rather than emotions, are more suitable for use as the basis for defining affect. First, it removes the epistemological problems caused by the circularity of defining affect in terms of cognitive artefacts, whilst simultaneously regarding affect as one of the foundations of models of cognition. Second, it allows an easier theoretical relationship between those aspects of affective processes that are involved in both cognition and biology. Third, it allows the easy integration of recent insights that are emerging from brain research. Finally, it improves the ‘granularity’ of the analysis of design cognition because feelings are epistemologically, theoretically and practically more elementary than emotions.

Following on from the above, the terms, affect and affective are used in this paper to refer to all processes involving feelings. From this perspective, what is normally referred to as rational thought is viewed as a subset of affective cognition in which the essential role of affect in cognition is temporarily ignored (see, for example, Rosen 1980; Hamlyn 1990).

 

Bastick, Feelings, and the Physiological Basis of Affect

Bastick (1982) explored the extensive literature that relates to intuition and insight, and concluded that the affective aspects of cognition, the drivers of intuition, were best described in terms of them being intimately connected with the biological aspects of feelings. Bastick described cognition as parallel processes of thoughts, body physiological states, and an individual’s perceptions of those states. He drew attention to the importance of the relationship between ‘thinking about something’, and the consequent many small physiological changes that occur in the body - referred to above as feelings. Thus, in Bastick's model of cognition, thoughts ‘map’ onto and create feelings. More importantly, however, Bastick concluded that this is a bi-directional relationship, in which sets of feelings also map onto thoughts. That is, if an individual creates particular sets of biological circumstances in their body, then associated thoughts are brought into attention. This is clearly not a singular mapping process. Bastick's evidence indicates that individuals are rarely conscious of all the relevant physiological states that map onto their thoughts. In other words, many thoughts might make a person feel angry, happy or some other emotion, and, vice versa, sets of feelings, as emotions, can bring to mind a number of associated, but different, thoughts.

Bastick argued that the underlying mechanism by which people identified whether a solution to a problem was satisfactory, or, in more formal terms, whether an individual decided to close a particular problem-solving process, was the combination of the feeling sets associated with the problem and the feeling sets of the proposed solution. When the feeling set of the solution matches the feeling set of the problem, then there is a relaxation of the individual’s body. This was most obviously apparent as a reduction in muscular tension. In general, Bastick’s findings and theories are supported by recent research literature relating to brain functioning (see, for example, Gould, Tanapat et al. 1999; Spinney 1999).

 

The Importance of Closure and the Role of Feeling-based Affect in Closure

The previous section raised the issue of closure. Closure is one of the two main ways that affect impacts on human design cognition - the other is the way that affect influences which new thoughts are initiated in response to a situation. In this context, closure is that human activity that is involved in deciding, in broad terms, whether to initiate a process, continue a process, or to stop a process.

Closure in designing has been somewhat neglected in the literature of design research. Where attended to, the focus has mainly been on object properties through a 'satisficing' or similar evaluative arrangement that is then used to 'decide' whether a feedback process should be terminated. The importance of closure lies in it being an essential attribute of human cognition that involves feelings. Closure in human design cognition depends on affect because of the ways that feelings (and their ontological cognitive associates, human values) underpin human judgments, conscious and subconscious thinking. The combination of closure and affect reaches deeply into and across most of the internal and external processes involved in designing. Without a satisfactory model of closure, researchers are reduced to attempting to represent internal human cognitive activities in terms of the informatic properties of the ephemeral objects changed by those processes. This is as epistemologically and practically problematic as trying to infer the program code of a word processor from documents edited by it.

In essence, the activity of closure takes a set of multiple inputs and compares this set against particular criteria. This process results in four output possibilities:

In general, these four outputs can be reduced to two, a binary yes/no or 1/0, by rewriting the criteria.

Using Bastick's theories leads to a description of the closure process in terms of the dynamic relationships of conscious and subconscious thoughts to body sensations, or feelings. As a designer follows a train of thought, their thoughts map onto sets of feelings that dynamically change as their thoughts change. The result is a change in the designer's overall perception of whether they are overall more (or less) tense or relaxed. The designer's current thoughts exist against a backdrop of the conscious and unconscious thoughts associated with a their prior experience and knowledge, and the problems currently under consideration. Each has feeling sets associated with them. These feeling sets from the designer's solutions, problems and the backdrop of their past experiences combine together into a dynamic complex overall feeling set that is continuously changing with the flow of their conscious and unconscious cognitive patterns. Bastick's theories suggest that it is the designer's perception of the dynamics of their levels of thought-driven tension and relaxation that guides them on whether it is fruitful to continue with a train of thought, or not. That is, it is the basis of the designer's judgement as to whether to go on or to stop (the basis of a binary yes/no choice).

It is this combined cognitive/affective closure process that enables a designer to make single binary choices from complex multi-variable phenomenologically-represented data by 'feel'.

The above version of closure forms the basis for the affect-based decision-making in the models of design cognition described in the next section. This human affect-based closure mechanism is what lies behind the binary 'yes/no' choices in the following diagrams, and what enables designer's choices to be based on analog complex multi-variable feeling-based representations of their thoughts.

The above model of closure also provides an explanation of the role of affect in a designer's optimisation of design solutions. Feeling sets occur in a designer's body as, consciously and unconsciously, partially-completed design specifications are mentally situated against backdrops consisting of the designer's understanding of how the world 'works' and the design problems that they are addressing. The designer's decision-making is shaped by their almost unconscious awareness of these multi-dimensional combined feeling representations of the relationships between aspects of the mental representations of partial designs, the designing contexts and the designer's prior experiences and knowledge. These moment-to-moment integrated feeling-based representations within the designer's body enables very fast management and feedback to the closure processes that shape and optimise design solutions in the designer's mind's eye. That is, as the designer thinks of new solutions, or solution variants, they are almost simultaneously able to 'feel' (according to their prior experience and understanding of the design problem definitions) whether the changes are beneficial. 

 

Simple Models of Design Cognition that include Affective Processes

In this section, three different ways of including affective processes in theories of design cognition are illustrated via the ‘problem/ generate solution/ test solution’ model of designing shown in Fig. 1 below. This model has been chosen to aid brevity than for any other purpose. In the following sections of the paper, the affective additions to this basic model are marked in italic or are shaded.

Figure 1. Basic ‘problem/solution/test’ model of designing.

A passive means of affective testing of design solutions

The simplest way to include affective processes into the ‘problem/solution/test’ model of design cognition is during the ‘testing’ phase. In this case, a solution is chosen, and the process is ‘closed’ when both of the following happen:

This process is shown diagrammatically in Figure 2.

Figure 2. Affectively-based testing of solutions

A passive means of affective design optimisation

Affective processes also contribute, in a passive manner, to design optimisation as described above. A basic model of optimisation consists of testing if a solution resolves the problem, comparing it to the existing best solution, and, if better, storing it as the new best solution. The process continues with the generation of a new solution. In this basic model, there are two testing processes that affect is a part. First, if a solution resolves the design problem in informatic terms, and the designer feels that it is better than the previous best solution, then it becomes the new best solution. Second, the process continues until the designer feels that it is time to close the process.

In this case, a best solution emerges when:

This process is shown diagrammatically in Figure 3 below.

Figure 3. Affective optimisation of design solutions

Additional aspects of affective optimisation

Faced with a design problem, a designer generates a solution, then compares a mental representation of this solution against mental models of the design context - including the problem specification. Within the designer’s body, misfits between the problem context and the solution result in the associated feeling sets. As the designer mentally makes changes to the design solution (or context), these sets of feelings change, and move in a complex manner away from or towards increased relaxation. In this way, the designer can feel whether each change is improving the solution or not.

In the above model, therefore, the affective system is providing an optimisation mechanism in which the dynamics of the various differences between the feeling sets of the solution, and the feeling sets of the problem, act in a similar manner to a conventional ‘cost function’. Unlike a cost function, the dynamics of the feeling sets also offer an optimisation measure that is simultaneously complex, multi-variable yet singular. The detail of relative movements in the optimisation function that are due to changes in the solution are observed in the differences between the feeling sets, and, at the same time, the overall quality of the solution can be assessed in terms of the overall level of relaxation of the body state. A significant difference between this affectively-based optimisation mechanism and traditional methods is that affectively-based optimisation of design solutions do not require the detailed characteristics of the solution and problem context to be specified in the public domain, or even to exist in detail in the conscious thinking of the designer. This model goes some way towards an explanation of why and how designers depend on style, culture and ideology, and why in design cognition, as Brian Eno (Eno 1996) observes, ‘style is fast’.

Active roles of affect in design cognition and solution generation

In the above models, affect is included in a passive manner via streams of affectively-coded data that result from perception of, and thoughts about, a design problem and its possible solutions. In the above discussions, these passive types of affective data streams do not actively contribute to the solution generation. Affect has an active role in the generation of design solutions, however, because of the bi-directional correspondence between designers thoughts of cognitive artefacts and feelings. Thinking about a design problem results in particular sets of feelings in the designer's body. These feeling sets, in their turn, give rise to thoughts. Bastick's analyses indicate that this reflexive feeling-based process tends towards a resolution of the design problem, and a reduction in the associated physical dissonance. (It is possible, however, that some designers might adopt problem definitions that would effectively reverse the resolution measure from relaxation to tension. That is, the designer might look for extreme innovation via maximal feeling dissonance.) Significantly, it is the feelings that result from thinking about a ‘problem’ that are the stimulus for the process that brings elements of potential solutions into a designer's consciousness, rather than the objective details of the problem. This is not to say that the problem characteristics have no role in creating solutions - the alternative would imply that there was little connection between them, and would leave the concepts of ‘problem’ and ‘solution’ relatively undefined. The above analyses, however, imply that there are benefits in moving the focus of design computing towards a view of design grounded in psycho-physio-neurological processes, and away from the previous emphasis on the theoretical relationships between the physical characteristics of problem and solution.

The bi-directional relationship between feelings and thoughts gives affect an active role in solution generation alongside its role in closure. This active aspect of affect in design cognition can be added to the ‘problem/solution/test’ model via an affective feedback loop between problem and solution generation. In this active model of affectively-based design cognition, the ‘problem’ specification brings together:

The active contribution of this problem-based feeling to the generation of design solutions is shown below in Fig 4.

Figure 4. Active role of affect in design cognition.

 

Parallel Processes and Affective Design Cognition

To this point, the passive and active roles of affective cognitive processes have been explored in relation to a simple feedback model of design cognition. The main aspects of these roles include:

Practical approaches to computerising affective cognition described by Bastick's theories are strongly influenced by several factors. First, the thought-feeling mappings are not deterministic in a general fashion across all individuals and circumstances. Second, even a detailed description of a feeling set that spans all modalities is insufficient to define singular solution ‘thoughts’. Rather, the feeling sets bound and define the topology of an individual’s cognitive access to their prior experiences via which they may generate appropriate cognitive solution sets. It is against this prior experiential, affective and rational milieu that are set models of artificial design processes, proposals for computer assistance for designers, and theories of use in computerising designing.

The above factors combine with typical aspects of design practice (e.g. team settings, expert design practitioners each with their own prior knowledge, multi-parameter design problems, complex multi-domain design solutions, access to computerised knowledge-bases and expert systems), and emerging theoretical representations of affect, cognition and brain function, to suggest that parallel processing is an essential aspect of modelling affective design cognition (see, for example, FNRS 1993; Wilson 1999).

Parallel affective/rational processes are intrinsically, and beneficially, redundant because they combine the possibilities of, on one hand, ‘feeling’ the way to a solution, and, on the other hand, achieving the same solution through purely rational processes of designing. A measure of the number of parallel cognitive processes involved in human design thinking can be gained by reflecting on the processes involved in ‘thinking a new thought’. These processes include:

In addition, are those processes indirectly associated with the main task of thinking a new thought. For example:

Each of these processes has their own cognitive artefacts with their associated mappings onto feelings. Bastick’s findings imply that the generation and choice of each of these cognitive artefacts is guided and shaped by the feeling sets of its own, and other, parallel cognitive processes

 

Computerising Affective Design Cognition

The above exploration suggests that computerising affectively-based human design cognition requires a different approach that takes advantage of the understandings that:

Affective representations (feelings) offer designers the means of simultaneously, and redundantly, comparing and contrasting different aspects of a design situation as seen from different perspectives. A designer, by using feelings, can simultaneously compare the emerging characteristics of different design solutions for (say) a roadbridge in terms of their beauty, cost, strength and utility. By using affective processes, the designer needs very little of the physical detail of the emerging design situation to be explicit, or even conscious. The complex detail of the physical characteristics are simultaneously, and redundantly, represented in a designer’s body by affective means—yet the designer’s conscious assessment of different design solutions is singular because it is measured in terms of whether the designer’s body is more or less relaxed and contented.

The approaches mainly used so far in computerising the human aspects of designing have been through attempts to quantify human qualitative considerations, and address them as if they were physical properties: an approach that presents many practical and epistemological difficulties (Coyne and Snodgrass 1993). The development of computerised models that include the roles of affect in human design cognition, intuition and closure requires a new approach, and the feeling-based model of affect described in this paper implies that this new approach should represent design cognition, and design processes, as a large number of interacting parallel streams of knowledge and information processing. There are three main types of these parallel processing streams. The first sort is the traditional, rational processing of theoretically defined information and data about the design problem, its environment and characteristics of possible solutions. The second sort includes ‘knowledge’ alongside information, and represents the design problem, its environment, characteristics of possible solutions, relevant theories, and the designers thoughts in physiological form as feelings. The third sort of processing is that which manages the affective representation of discrete elements of the first two streams for purposes of comparison and meta-cognition.

To this point, the descriptions of the roles of feelings and feeling sets in design cognition and closure have been essentially biologically-based. There appears to be no particular barrier, however, to replicating these biological dynamics computationally. Applying suitable instrumentation to an individual would enable the dynamic transitions of these indicators to be measured quantitatively. This opens the possibility to using computational methods to establish relationships between affective, cognitive and infomatic representations of design solutions, problems and prior knowledge. This can be approached at several levels. In the early 1970s, at Lancaster University, the author computationally explored how solution sets for complex multi-variable engineering design problems could be optimised. An n-dimensional matrix of values of which each axis represented a key variable in the problem was resolved into an n+1-dimensional solution surface. A combination of linear programming methods and hill-climbing algorithms provided the main methodologies for automatically identifying which areas of the solution set surface were likely to contain suitable and well-optimised solutions. This relatively crude approach might be extended to include surfaces representing affective representation. Many of the limitations of this simple approach, however, are likely to be obviated by the use of neural network approaches such as connectionism. Perhaps the most obvious way forward is through the use of parallel self organising neural network models grounded on unsupervised learning paradigms. In this case, a model consisting of multiple Kohonen layers may offer the appropriate complexity of multiple parallel and reflexive learning processes - a necessary feature if the reflexive mapping from feeling sets to cognition that results in solutions being generated from composite feeling sets is to be implemented as part of the model. Parallel processing methods (and unsupervised learning neural networks) offer the possibility of simultaneously mapping the different modalities of feeling representation to and from solution surfaces and the associated cognitive and affective representations via a multi-layer learning process.

The above discussions suggest several factors are important for developing computerised systems that include the affective aspects of human cognition:

Figure 5. Venn diagram of affective/rational cognitive relationships

 

Summary

Historically, design computing has been based on rational cognition of the physical attributes of design situations in which non-rational human factors are quantified and included as if they are physical quantities. This paper explores a complementary approach that focuses on, and gives primacy to, the non-rational affective aspects of human designing. In this approach, physical considerations are addressed through affective processes.

A model for computerising affective design cognition is sketched out. This model assumes that all data and knowledge relevant to a design situation is redundantly mapped onto 'feeling sets' conditioned by designers’ prior experiences. These feeling sets are used, on one hand, as drivers for the identification of cognitive artefacts that are representations of appropriate partial, or complete, design solutions, and , on the other hand, for internal cross-comparison and optimisation purposes.

Taken together, the above discussions describe an approach that points to how alternative models of design computation might be created that include affect, human values, beliefs, prior understandings, feelings, human judgments, and paradigmic understandings.

 

References

Bastick, T. (1982). Intuition: How we think and act. England, John Wiley and Sons.

Coyne, R. D. and Snodgrass, A. (1993). "Rescuing CAD from Rationalism." Design Studies 14(2): 100-123.

Cross, A. (1984). "Towards an understanding of the intrinsic values of design education." Design Studies 5(1): 31-39.

Davies, R. and Talbot, R. J. (1987). "Experiencing ideas: identity insight and the imago." Design Studies 8(1): 17-25.

Dilnot, C. (1982). "Design as a socially significant activity: an introduction." Design Studies 3(3): 139–146.

Dym, C. L. (1994). Engineering Design: A Synthesis of Views. Cambridge, Cambridge University Press.

Eno, B. (1996). How art frames our world. Daily Telegraph: A5, June 22

FNRS (1993). Experimental test of appraisal predictions for emotional response patterning, http://www.unige.ch/fapse/emotion/projects/appraisal.html

Glegg, G. L. (1971). The Design of Design. Cambridge, UK, The University Press.

Gould, E., Tanapat, P. et al. (1999). "Neurogenesis in adulthood: a possible role in learning." Cognitive Sciences 3(5). http://www.biomednet.com/library/fulltext/TICS.tics0599_13646613_v0003i05_00001310

Hamlyn, D. W. (1990). In and Out of the Black Box: on the philosophy of cognition. Oxford, Basil Blackwell Ltd.

Holt, J. E. (1997). "The designer’s judgement." Design Studies 18(1): 113-124.

Lyle, J. T. (1985). Design for Human Ecosystems. New York, Van Nostrand Reinhold.

Massachusetts Institute of Technology - Affective Computing Research Area (1999A). Research on Affective Pattern Recognition and Modelling. http://www.media.mit.edu/affect/AC_research/recognizing.html

Massachusetts Institute of Technology -Affective Computing Research Area (1999B). Applications for Affective Computing. http://www.media.mit.edu/affect/AC_research/applications.html

Massachusetts Institute of Technology -Affective Computing Research Area (1999C). Research on Human Emotions. http://www.media,mit.edu/affect/AC_research/applications.html

Massachusetts Institute of Technology -Affective Computing Research Area (1999D). Affective Computing Research Areas. http://www.media.mit.edu/affect/AC_research/

Rosen, S. (1980). The Limits of Analysis. New Haven, Yale University Press.

Spinney, L. (1999). New Model of Parkinson’s, HMS Beagle. http://www.biomednet.com/hmsbeagle/59/notes/meeting1.htm

Susac, D. (1998). Emotional Machines: A review of Rosalind Picard’s Affective Computing (1997, MIT Press), About.com Inc. http://ai.miningco.com/library/weekly/aa070598.htm

Wilson, I. (1999). "Artificial Emotion: Simulating Mood and Personality." Gamasutra 3(18). http://www.gamasutra.com/features/19990507/artificial_emotion_02.htm

Return To Top of Page