Portfolios in University-based Design Education
For many years, portfolios have been an important feature of student assessment in Art and Design education. Over the last few years, Design Education has been changing in response to changes in technology, and in response to its incorporation into the university system. At the same time, portfolio-based assessment has been developed further and adopted in many other areas of professional education. It is now time for Design courses to review their use of portfolios, to see whether they are still appropriate as a means of student assessment and to see what can be learnt from the use of portfolio assessment in other disciplinary contexts.
This paper describes new and well-justified portfolio-based approaches to assessment that align with the current context of design education in universities and support students, staff, and institutions in the areas of quality improvement, the resolution of equity issues, assessment moderation, and placement-based learning.
For many years, portfolios have been an important feature of student assessment in design education, particularly those originating in that art, craft and architectural traditions (McCracken, 1997) . Design education has, however, seen considerable change - most recently through its incorporation into the university system, changes in technologies that designers use, and changes in the role of the designer (Bullock & Justice, 1999; Dorsa & Walker, 1999; Giard, 1999) . Over a similar period, portfolio-based assessment has evolved - in part through its adoption by many other areas of professional education. (see, for example, Bolger & Scott, 1984; Masters & McCurry, 1990; Thompson & Farrow, 1999) . It is now time for those managing Design Education courses to review their use of portfolios, (McCracken, 1997) to see what can be learnt from the use of portfolio assessment in other disciplinary contexts, and to identify the forms of portfolio assessment that are most appropriate for their pedagogical contexts.
The paper begins with an overview of issues in portfolio assessment. Changes in design education are discussed, and the implications of these changes for the use of portfolio assessment within Design Education degree courses are explored.
The definition of a portfolio used in this paper is that of Cooper (1999) : 'a collection of evidence that demonstrates skills, achievements, learning, or competencies'.
Portfolios are used for assessment purposes in a wide variety of university courses. The four main roles of portfolio-based assessment are in:
· Competency-based education: where assessment is via evidence of the demonstration of specified skills to specified standards.
· Education based on negotiated learning: where each student negotiates what it is that they wish to learn. Different students have different, individualised, learning contracts.
· Recognition of prior learning.
· Pre-study and course entry assessment.
This paper focuses on portfolios for assessment purposes in undergraduate courses. For brevity, there is no discussion of recognition of prior learning and pre-entry assessment. Further details on these issues may be found in Cooper (1997; 1999) .
Portfolios are used other in educational contexts for both formative and summative purposes. Summative portfolios form the basis for the assessment of learning outcomes rather than the processes of learning. They are collections of students’ work that show the range and extent of their skills by selecting the best examples of different types of work that demonstrate different skills. The original purpose of an art portfolio was a summative assessment. If assessment is concerned with the learning outcomes of students, then summative portfolios are the appropriate choice. This is true even when the learning outcome being assessed is the use of a process over a period of time, as is the case when a student uses a design process to address a design brief.
The main role of a formative portfolio is to make explicit the processes of learning in which a student has engaged, and the progress in learning that the student has achieved. A formative portfolio might include, for example, weekly samples of a student’s work, to show how the student had progressed and acquired new skills over a period of time. A formative portfolio documents the learning process of students rather than their success in achieving specified learning outcomes. Formative portfolios are useful for assessing the effectiveness of different teaching methods and individual students’ learning rates.
To summarise, formal assessment within certificated courses must be summative when it identifies the learning outcomes a student has achieved. Assessment must be formative when it identifies the rate of a student’s progress, the teaching quality within a course, and the learning needs of students (Cooper, 1999) . This position contradicts claims that portfolios containing students’ research and project drafts are formative (e.g. McCracken, 1997) . Portfolios that detail the process by which a student fulfils a design brief are summative because their primary focus is providing evidence that the student has completed appropriate steps in the process of addressing a design problem. This evidence describes the student’s progression through a design process rather than the student’s internal learning processes. A competent designer must be skilled at implementing design processes: it is unimportant from this perspective, how a trainee designer acquires this skill.
In any situation, a satisfactory assessment method should be:
· Equitable
· Fair (moderated)
· Fitting
· Cost effective
Equitable assessment methods avoid unfairly disadvantaging students because of structural biases within the course assessment. All assessment methods and course designs have some biases. Course developers should minimise these by their assessment choices. Fairness requires that the assessment leading to an award is consistent. This means the assessment method should be capable of effective moderation and easily explicable to all students. Each assessment method needs to be fitting to its task and should provide a good measure of what it claims to assess. As courses change, the skills, abilities and knowledge base changes and the assessment methods must change to reflect this. Finally, assessment methods must satisfy the above three criteria, and be cost-effective.
Universities prescribe overall educational purposes for their courses that are different from the priorities and purposes of other types of educational institutions. Key aims and objectives of universities identified by Cooper (1999) are:
· To help students develop critical and analytical skills that they can apply across disciplines
· To ensure that graduates gain suitable employment upon graduation
· To ensure that graduates meet the professional requirements of relevant professional accrediting bodies.
· To prepare graduates for post graduate studies
· To give students a well-rounded foundation in interdisciplinary studies.
· To enable each student to realise their full academic potential
· To broaden students’ perspectives on the modern world and their place in it
· To help students’ develop life long learning skills
· To give students the tools that will enable students to solve problems effectively in novel situations.
Within each university, different courses give different priority to each of the aims and objectives above. In addition, several other factors shape the ‘context’ in which design courses develop their curriculum. University culture and their distribution of resources favour styles of teaching and learning that can be delivered to masses of students in lecture theatres rather than courses that require more individualised teaching to smaller groups of students. Universities prefer to minimise the number of special purpose teaching rooms for reasons of economy. The current emphasis on libraries and the Internet as learning resources for students supports some particular types of curriculum rather than others. Taken together, these factors have the effect of hastening the movement of design curricula away from the arts, craft and architecture traditions towards more analytical and theoretical foundations.
Design education has developed in differently in the various disciplines in which it is taught. Some streams of design education can be traced back thousands of years, others, e.g. education courses for website designers, are relatively recent. In some disciplines, such as engineering, the practice of designing may precede university-based education for designers (French, 1985) . In others design practices post-date academic research and design education.
There has been a general progression from the creation of designs for single objects produced by mainly manual methods (the art and craft tradition) to creating the designs for the production of objects using repetitive and mechanised technologies (industrial design and product design). These changes have been accompanied by a transition from tacit knowledge and craft to explicit knowledge, abstract representation, theoretical analysis and computer-based assistance.
In both the art and craft traditions, and the technical design traditions, the content of design education courses have remained almost completely discipline-centric. This is evidenced by the limited extent to which design education courses include material from other disciplines in their curricula. For example, engineering design curricula typically consist of over 90% engineering theory (see, for example, Larson, Collier, Hatfield, & Howell, 1996) . The proportion of time spent on multidisciplinary issues seems inappropriately small bearing in mind that artefacts have extensive social, ethical, environmental and commercial consequences, and that addressing many of these competently requires the ability to include research analyses from other disciplines (see, for example, Dilnot, 1982; Love, 1998; Pacey, 1983) .
Recent directions in design research and the practices of designing are towards multidisciplinary designing: multidisciplinary, cross-disciplinary and inter-disciplinary design teams have been common place for some time (see, for example, Jones, 1970; Larson et al., 1996; Love, 2000) . These teams have until recently been made up from single-discipline specialist designers (Larson et al., 1996) - an approach that aligns with existing discipline-based forms of design education because each designer has a single disciplinary focus. The criticisms of this approach have been of difficulties of communication, and problems caused by team members using different discipline-based paradigms of practice that limit the effectiveness of co-operation. New directions in multidisciplinary designing are towards individual designers being multidisciplinarily competent across many areas of research, information, communication, and design methodologies - echoing claims that designing is an integrative activity/discipline (Friedman, 2000) . The transition that is currently emerging is from 'multidisciplinary teams of specialists' to 'teams of multidisciplinary individuals'.
The trend towards teams of designers consisting of multi-discipline skilled individuals requires that design educators train designers who are able to understand and contribute at a professional level across a wide variety of disciplinary areas. This is a radical change of direction for design education that breaks the boundaries of design education traditionally provided within single discipline-based contexts because it requires individuals to be educated to a professional level in several disciplines at the same time. Current moves towards this form of education include the sundry arrangements of double and treble degrees, and meta-discipline degrees in, for example, systems designing, social informatics, and cybernetic design.
Putting together the two main themes of this paper, ‘new developments in portfolio assessment’ and ‘trends in design education’, a number of issues emerge in relation to the question, ‘What should be the role of portfolios in the assessment of design education?’. The structures described earlier define some of the general characteristics of an improved use of portfolio-based assessment in design education. The appropriate use of portfolios in specific assessment roles within particular courses depends, however, on the vision adopted of design education.
Equity Issues: In many cases, equity issues for design education assessment arise from the new technologies and materials available to students. For example, some students can afford access to better equipment, such as computers, better tools or more expensive materials; some students will have better opportunities in industrial placements than others; and some students will receive better supervision than others in industrial placements. The assessment methods must judge individual student’s achievements not the size of their wallet. This is a potential weakness of assessment that is based solely on a final exhibition or the portfolio that includes only finished products. Where assessment includes evidence of research and design reasoning processes and final product assessment is generally more equitable.
Moderation: The inclusion of design education into the university system requires that academic standards of design education equate with other forms of university education. Design education must also conform to the same moderation requirements as other university courses. The specification of design education courses must, therefore, define the criteria by which decisions are made about the quality of evidence within a portfolio is judged - that is, the marking criteria.
Portfolio assessment can be used for graded or ungraded assessment. If portfolios are graded, the course designer must decide and prescribe the criteria that will be used to determine relative merit. The four main options for superior grading are:
· The quality of evidence that a student offers is greater
· The quantity of evidence that the student offers is greater
· The student offers evidence of having a more extensive range of skills
· The student offers evidence of having a superior level of skill development in a specific range of skills
The four options are not similar. The first two options offer higher grading relating to the quality of the portfolio, rather than the quality of the skills for which the portfolio is the evidence. The second two options offer higher grading for students who show evidence of higher level of skill, or more extensive skills Cooper (1999) . Where a portfolio is graded, then it is necessary to establish the parameters of quality that are appropriate to each level of grading.
For courses in which students can negotiate their learning, grading portfolios provides a feasible way to give recognition to students who make the best use of this situation. If graded assessment is chosen, then the course documentation should provide appropriate guidance to students and staff tutoring on the course. The course developer must decide whether to penalise students who present irrelevant material. This is especially important in situations in which portfolios are graded, because grading portfolios encourages students to include any material that they think may increase their mark - especially if they believe that their grade is linked to the quantity of evidence.
Where the assessment is graded and the students do self-directed work, then direct comparison of students’ work is difficult because of the dissimilar nature of the items being compared. The assessment criteria should be developed to address this difficulty and enable equitable moderation.
Grading portfolios is incompatible with competency-based assessment. If portfolio assessment is competency-based, then the course designer must define how the assessors are to distinguish between satisfactory and unsatisfactory evidence of skill acquisition.
Fittingness: The assessment method must be appropriate to the type of knowledge, professional or practical skills that are being assessed. The changes in design education have implications for the choice of skills addressed in the curriculum and how students should be assessed. Traditional academic forms of assessment assess knowledge and analytical skills well, but correlate poorly with the assessment of many professional and practical skills. Within universities it is appropriate that academic assessment is used to assess analytical skills. Competency-based portfolios may be more effective for assessing professional and practice skills. The strength of competency-based assessment lies in its ability to guarantee that students have achieved certain minimum standards of performance in nominated skills. This makes competency-based assessment appropriate for assessing communication skills and teamwork skills because judgement need only be made about whether students’ performances are adequate.
Whether graded or ungraded portfolios are more suitable for assessing design projects depends on both the pedagogy of the course and equity issues. If students undertake industrial placements they experience varied levels of opportunities, supervision and support. Ungraded portfolios using competency-based assessment avoid the risk of assessing the quality of the placement rather than the quality of students’ work. If these equity issues can be overcome and the moderation requirements addressed, then grading portfolios remains a possibility.
Cost effectiveness: The norms of academic assessment suggest that about two hours per student per unit is about the maximum length of time that is realistic for academics to commit for assessment. Changes in the nature of design education mean that students are now required to have a broader range of skills in more diverse fields. The assessment of these skills must be built into the course assessment in a cost-effective manner without compromising the equity, moderation or fittingness of assessment.
The table below summarises some of the observations about portfolio assessment and design education.
|
Aspects of designing and design education |
Assessment considerations |
Portfolio assessment issues |
|
Designs for artefacts designed via art and craft traditions. |
Aesthetics and functioning of artefacts. Skills of art and craft. Design skills not involving a high level of theoretical modelling . |
Assessment is of the artefact only. Moderation difficulties: how to fairly compare very different artefacts and skills. Equity issues: access to different resources/ opportunities. |
|
Designs for artefacts to be made by repetitive or mechanical methods. |
Domain-based design skills including manufacturing issues. Art and craft skills. Design skills involve theoretical modelling within a limited range of disciplines – usually technical. Limited inclusion of extra-disciplinary abilities and knowledge eg social, ethical and environmental analyses. |
Assessment is of the artefact and the design process. The portfolio includes both the artefacts and a description of the design process. Equity issues arise if the portfolio is to be graded, if students have differing opportunities and level of support during industrial placements. |
|
Multidisciplinary teams of single-discipline designers. |
Domain-based design skills including manufacturing issues. Art and craft skills. Design skills involve theoretical modelling within a limited range of disciplines – usually technical. Limited inclusion of extra-disciplinary abilities and knowledge eg social, ethical and environmental analysis. Some cross disciplinary communication and team skills. |
Assessment is of the artefact, the design process and the ability to work in a multi-disciplinary team. The portfolio includes artefacts, a description of the design process, and evidence of team work /communication skills. Consider competency based, especially for assessment for professional skills. |
|
Teams of multidisciplinary designers. |
Design skills involving theoretical modelling across a wide range of disciplines. Multidisciplinary abilities and knowledge eg social, ethical and environmental analysis at a professional level. Professional communication and team skills. |
Assessment is of the artefact and the design process and the ability to comprehend and address design problems from several different disciplinary perspectives. The portfolio includes artefacts and descriptions of the research and design processes along with evidence of and team work/ communication skills. The description of the research process demonstrates how knowledge traditionally held in other disciplines has been applied to the design problem. Consider conventional academic assessment methods for assessing analytical skills, and competency-based assessment for professional skills. |
Changes in design education brought about by its incorporation into the university system, changes in technology, and changes in the roles of designers, mean that student assessment must be reviewed. Portfolios still have a useful contribution to make, especially in the assessment of practical and professional skills. Competency-based portfolios should be considered for professional skills. If portfolios are graded, then additional attention needs to be paid to ensuring that the grading process is valid, equitable, capable of moderation and aligns with the intended purpose of the assessment.
Bolger, S., & Scott, D. (1984). Starting from Strengths. Leicester: National Youth Bureau.
Bullock, W., & Justice, L. (1999). A Future Role for Research Universities. Paper presented at the Chicago 99 Design Education Conference Proceedings, Chicago.
Cooper, T. (1997). Portfolio assessment: a guide for students. Perth: Praxis Education.
Cooper, T. (1999). Portfolio assessment: A guide for lecturers teachers and course designers. Perth: Praxis Education.
Dilnot, C. (1982). Design as a socially significant activity: an introduction. Design Studies , (3 3), 139–146.
Dorsa, E., & Walker, S. (1999). Making Design Work: Sustainability, Product Design and Social Equity. Paper presented at the Chicago 99 Design Education Conference Proceedings, Chicago.
French, M. J. (1985). Conceptual Design for Engineers. (2nd ed.). London: Design Council.
Friedman, K. (2000). Design Knowledge: Context, content and continuity. In D. Durling & K. Friedman (Eds.), Doctoral Education in Design: Foundations for the Future (pp. 5-17). Stoke on Trent, UK: Staffordshire University Press.
Giard, J. (1999). Industrial Design Education: incompatibility with education in art and architecture. Paper presented at the Chicago 99 Design Education Conference Proceedings, Chicago.
Jones, J. C. (1970). Design Methods: seeds of human futures. London: Wiley-Interscience.
Larson, D., Collier, K., Hatfield, J., & Howell, S. (1996). Student Assessment and Interdisciplinary Design . Flagstaff, Arizona: College of Engineering and Technology, Northern Arizon University.
Love, T. (1998). Social, Environmental and Ethical Factors in Engineering Design Theory: a Post positivist Approach. Perth, Western Australia: Praxis Education.
Love, T. (2000). Theoretical perspectives in the PhD thesis. How many? In D. Durling & K. Friedman (Eds.), Doctoral Education in Design: Foundations for the Future (pp. 261-270). Stoke-on-Trent, UK: Staffordshire University Press.
Masters, G. N., & McCurry, D. (1990). Competency-Based Assessment in the Professions National Office of Overseas Skills Recognition Research Paper No. 2. Canberra: Department of Employment, Education and Training.
McCracken, W. M. (1997). Portfolio Assessment in Design Education . Atlanta, Ga: EduTech Institute and College of Computing, Georgia Institute of Technology.
Pacey, A. (1983). Culture of Technology. Oxford: Basil Blackwell Ltd.
Thompson, R., & Farrow, T. (1999). The Workbook Portfolio: Facilitating undergraduate student learning in the mental health clinical area. Nursing Praxis in New Zealand , (14 2), 21-30.
(c) T. Cooper & T. Love 2000