LEARNING THROUGH DESIGN-IMPLEMENT EXPERIENCES
– A LITERATURE REVIEW
Jonte Bernhard1, 2, Kristina Edström2, Anette Kolmos2, 3 1Linköping University, Department of Science and Technology,
2KTH Royal Institute of Technology, School of Education and Communication in Engineering Science,
3Aalborg University, Department of Development and Planning
ABSTRACT
In this paper we introduce some literature relevant for design-based learning, in particular for design-implement experiences in line with CDIO Standard 5. The aim is to inform the development of such learning experiences and to indicate some areas where new research would be of relevance to educators.
KEYWORDS
Design-based learning, design-implement experiences, literature review, engineering education research, Standards: 5.
INTRODUCTION
“Scientists study the world as it is; engineers create the world that never has been.”
Theodore von Kármán (NSF, 2013) The CDIO approach to engineering education reform advocates a sequence of design-implement experiences through the curriculum, as expressed in CDIO Standard 5 (CDIO, 2010). The purpose of design-based learning is not that students should design something; it is that they should learn from designing something. Design-implement experiences are the main site for the teaching, learning and assessment of competence in conceiving, designing,
implementing and operating. In addition, the intended learning outcomes can cover a wide
range of underpinning disciplinary knowledge (e.g. applying and integrating knowledge from different disciplines), personal and interpersonal skills (e.g. teamwork, project planning and communication) and attitudes (e.g. responsibility for the use of technology and taking sustainable development into account). Therefore the role of design-implement experiences in the curriculum is not only about learning to design, it is equally about learning through design.
In this paper we set out to introduce some literature on design-based learning relevant to faculty engaged in design-implement experiences. The aim is to support educational development in accordance with CDIO Standard 5. While there is a vast body of literature on design processes per se, our perspective here is exclusively on student design projects as
learning activities. This considerably narrows down the task of surveying the field.
In the next section we will discuss the aims of design-based education as a background before continuing to our review.
AIMS OF DESIGN-BASED EDUCATION Intended Learning Outcomes
Designing is a complex process that involves numerous variables, constraints and conflicting values, as has previously been discussed by Schön (1987, pp. 41-42):
Designing, in its broader sense involves complexity and synthesis. In contrast to analysts or critics, designers put things together and bring new things into being, dealing in the process with many variables and constraints, some initially known and some discovered through designing. Almost always, designers’ moves have consequences other than those intended for them. Designers juggle variables, reconcile conflicting values, and manoeuver around constraints – a process in which, although some design products may be superior to others, there are no unique right answers.
The role of design-implement experiences is to provide early experiences in a sequence of projects increasingly resembling engineering practice (Crawley, Malmqvist, Östlund, Brodeur, & Edström, 2014). Students can potentially learn many different things within this framework, and during the sequence of such learning experiences through the program, there can be progression along several dimensions. For instance, the intended learning outcomes can be in these categories:
• Reinforce disciplinary understanding through repetition, integration and application, and to provide opportunity for specialization and just-in-time acquisition of new knowledge
• Develop skills and abilities, both personal (e.g. initiative and self-reflection) and interpersonal (e.g. collaboration and communication)
• Experience authentic problems, i.e. complex and open-ended problems, embedded in realistic contexts, and containing ambiguity and trade-offs
• Practice product, process and system building skills, i.e. conceiving, designing, implementing and operating
• Experience professional working modes, habits and practices
• Reflect on ethics and responsibility of engineers, the potential and limitations of technology, and its contribution to sustainable development of society
Other possible aims, not directly related to learning particular things, could be to increase student motivation for continuing in the program and seeing the relevance of new theoretical courses. Further, as activities like these can be highly visible inside and outside the institutions, they can serve to demonstrate that engineering is creative and fun.
We note that there can be a number of different aims for this type of learning activity, and different priorities will be reflected in each particular implementation.
Authentic Problems
When setting the project tasks for the students, it is worth considering the nature of the challenges that practicing engineers will encounter. In a highly readable paper, Jonassen, Strobel, and Lee (2006) report on a study in which they interviewed 106 practicing engineers. They identified the following attributes of workplace engineering problems:
A. Workplace problems are ill-structured
B. Ill-structured problems include aggregates of well-structured problems C. Ill-structured problems have multiple, often conflicting goals
D. Ill-structured problems are solved in many different ways E. Success is rarely measured by engineering standards F. Most constraints are non-engineering
G. Problem solving knowledge is distributed among team members H. Most problems require extensive collaboration
I. Engineers primarily rely on experiential knowledge
J. Engineering problems often encounter unanticipated problems K. Engineers use multiple forms of problem representation
L. Engineers recommend more communication skills in engineering curricula
The conclusion by Jonassen et al. (2006) is that students need to encounter problems in their education that require them to:
• Analyze and solve combinations of well-structured problems • Manage multiple sub-problems
• Deconstruct multiple, often conflicting goals from a problem statement and analysis • Reconcile multiple, conflicting constraints and criteria
• Analyze and select from a variety of solutions to various problems and to justify their selected solutions
• Identify and reconcile methods for achieving non-engineering criteria for solving problems • Communicate and collaborate with a variety of professional and paraprofessional team
members on all aspects of the problem-solving process
• Anticipate and reconcile intervening problems and perturbations to the problem-solving process
• Adapt to changing project conditions and unanticipated problems
• Use multiple tools and formalisms (visual, verbal, quantitative) to represent problems We suggest allowing these dimensions to be increasingly present in the problems that students encounter throughout the education, i.e. they can all be seen as dimensions of progression.
REVIEW METHODOLOGY
Our aim in this paper is to present an overview of educational research papers published in scholarly journals investigating topics related engineering students’ design-based learning. This means that we did not search for papers presented at conferences. We searched for papers in education research, engineering education research, science education research and design research journals. As a first search with the intention to get an overview of the field our search was exploratory. To be included a paper should be based on empirical research using a sound methodology, study engineering students at university level and investigate a topic related design-based learning. Thus, we excluded papers in which only primary or secondary school students or practicing engineers were studied, and non-empirical papers such as position papers or papers only describing a new approach.
REVIEW RESULTS
We found relevant papers in Australasian Journal of Engineering Education; CoDesign;
Design Studies; European Journal of Engineering Education; Instructional Science; International Journal of Engineering Education; International Journal of Technology and Design Education; Journal of Engineering Education; Journal of Mechanical Design; Journal of Research in Science Teaching; Mind, Culture, and Activity; and Research in Engineering Design. This list demonstrates the difficulty in reviewing the field; research is published in
journals with quite disparate aims.
Time and space only allow us to present brief reviews in this paper and not all papers we found in our exploratory search are included. What we present here are some of the empirical research papers found in research journals, selected according to what we
consider offer the most valuable insights related to student learning in design-implement experiences. Others would probably have made slightly different priorities.
Students’ design learning
Investigating student design tasks in mechanical engineering Samuel and Lewis (1991, p. 314) argued that “[analytical approaches] where there is heavy emphasis on solving problems with unique answers … will fail in engineering design problems which have no unique answers” However, they also argued that it would be a mistake to conclude that knowledge of science does not matter. Indeed, Samuel and Lewis noted that in a relatively simple design task (by professional standards) used in educational content a total of 96 concepts needed to be utilized by the students’ for successful task completion. Hence, they conjectured that robust understanding of concepts and fluency in their use as well as flexibility in crossing boundaries of knowledge domains are essential for an effective engineering designer.
Accordingly, Cynthia Atman and her co-workers found that skilled designers scoped problems more effectively than less skilled or experienced designers, by considering more objects and making more transitions back-and forth among design steps Generally, they found that professional engineering designers made higher quality designs than students (Atman et al., 2007), senior students made higher quality designs than first year students (Atman, Cardella, Turns, & Adams, 2005; Atman, Chimka, Bursic, & Nachtmann, 1999) and that the design skills of individual students developed during their studies (Cardella, Atman, Turns, & Adams, 2008). Furthermore, students typically solve design problems in a linear sequence, while experts move frequently across domains simultaneously (Atman et al., 2007; Atman et al., 2008).
Thus, it is important to bridge the “design-science gap” (e.g. Vattam & Kolodner, 2008) but in design tasks students often have difficulties in relating their knowledge of basic principles from engineering science or applying mathematics to the task at hand (Carberry & McKenna, 2014; Silva, Fontul, & Henriques, 2014). Accordingly, Kittleson and Southerland (2004) observed that such “concept negotiations” were exceedingly rare even in a senior year design project. Indeed, it has been proposed that design thinking should be seen as a “threshold concept” (Johnson, Bull, & Osmond, 2013; Taboada & Coombs, 2013).
The use of theories, models, representations, concepts, and other tools is also important in designing: as observed by Cardella, Atman, and Adams (2006, p. 18) “most participants allocated the greatest percent of time to … modeling”. Modelling involves analytic as well as synthetic processes, and connecting symbolic content, i.e. theory and models, with real content, i.e. objects and events (Bucciarelli, 2002). However, Carberry and McKenna (2014) found that students generally applied models only “to visualize and test their solutions”, and failed to use predictive modelling as a tool in the design process. In contrast, Juhl and Lindegaard (2013) found that engineering students they monitored during a second semester design project materialized their discussions, thinking and decision making through work with different representations. Representations, especially drawings, transcended the boundaries of the individuals and became elements in the group’s collaborative thinking, thereby fostering connection between individual cognition and collective re-cognition.
The above discrepancy may be understood in terms of contextual factors (Goncher & Johri, 2015) and lack of constructive alignment between the intended object of learning, teaching approach and focus of assessment (Biggs, 1996). Juhl and Lindegaard (2013) focused on students engaged in a project that was part of a fairly recently designed engineering program with (presumably) favourable contextual factors and good alignment, while Goncher and Johri reported that the students they studied saw their design project as yet another academic exercise, and learning as “learning how to pass”. In a similar vein, Newstetter
(1998) and Kittleson and Southerland (2004) found that groups used a “divide and conquer approach” to complete design projects “efficiently”, resulting in individual team members missing valuable learning opportunities in efforts to finish tasks as quickly as possible. Although the instructors in both cases wanted the students to collaborate, the assessment practices drove the students to take an instrumental task-oriented approach. This is hardly surprising as the quality of the finished design is sometimes used as a measure, or even the sole measure, of a student design team’s success (Agogino, Song, & Hey, 2007; Oxman, 2001). Students’ learning in design projects is not only framed by how a student design project is devised by teachers, but also by how students experience assessment. The intended object of learning can be counteracted by grading procedures and other contextual factors.
Empirical research methods in engineering design education research
The dominant empirical method to investigate students’ design process seem until recently to have been variants of “think-aloud” exercises with verbal-protocol-analysis (Craig, 2001) mostly of individuals in simulated settings (e.g. Atman et al., 2007; Atman et al., 1999; Cardella et al., 2008).
More recently an emerging trend with an increasing number of studies using different forms of ethnomethodology investigating students’ process of learning to design in naturalistic settings can be observed. For example Gilbuena, Sherrett, Gummer, Champagne, and Koretsky (2015) used audio-recordings, interviews and collection of artefacts, Goncher and Johri (2015) used video-recordings, field-notes and interviews, Juhl and Lindegaard (2013) photos, field-notes and collection of artefacts and Kittleson and Southerland (2004) used audio-recordings and field-notes together with discourse-analysis. Early examples are Newstetter (1998) using participant observations and Smith and Leong (1998) using video-recordings and interaction-analysis.
Another emerging approach in engineering design education research is the use of intervention studies using some kind of pre- and post-test to evaluate the effect of the intervention1. Examples are Biggio, Vázquez, and García (2015) addressing students’ graphical skills and Booth, Taborda, Ramani, and Reid (2016) addressing sketching skills.
DISCUSSION AND CONCLUSIONS
An important conclusion from reading the literature is that successful designing is not a linear process, rather it is important that students’ learn to scope problems effectively and are able to go back and forth among design steps (Atman et al., 2007; Atman et al., 2008). Thus it is important that CDIO is not (mis-)interpreted as advocating a linear design process with discrete steps: Conceive → Design → Implement → Operate.
Bucciarelli (2001, p. 298) remind us that “the methods of reduction and abstraction… fail to capture the collective nature of designing and design knowledge”. Accordingly, “we can’t teach design like we teach science” (ibid., p. 306). However, Owen (2007, p. 22), reminds us that “a combination of science thinking and design thinking is better than either alone”. Above all, education can fail in forging a meaningful relationship between these different types of knowledge, as when students “given an unstructured design task, often have difficulty relating their analysis knowledge to the task at hand” Eastman (2001, p. 151). In line with previous reasoning, (Steele, 2015) suggests that “systemic thinking, framing and the application of first principles make up the primary abilities of the expert design engineer”. The review results point to the importance that students in designing are skilled in modelling
and in the use of disciplinary knowledge, concepts, models, representations and sketches (e.g. Carberry & McKenna, 2014; Cardella et al., 2006; Juhl & Lindegaard, 2013; Kittleson & Southerland, 2004; Samuel & Lewis, 1991). The challenge is thus how science and design thinking, i.e. how analytic and synthetic skills can be fostered in engineering education.
Figure 1. Conceptual representation of content and process factors in engineering, adapted from Owen (2007).
Noteworthy are the results by, for example, Goncher and Johri (2015), Kittleson and Southerland (2004), and Newstetter (1998) that clearly demonstrates that the intended object of learning can be counteracted by assessment and grading procedures and other contextual factors. This underlines the need for constructive alignment between the intended object of learning, teaching approach and focus of assessment (Biggs, 1996) and the need for development of constructive assessment and grading procedures that support the intended object of learning
.
We hypothesise that design-implement experiences are too often thought of merely as projects in which students achieve a design task. However, just as for any other type of learning activity, engineering faculty need the teaching competence to make a good course design that supports learning (in accordance with Standard 10).As shown above, previous educational research indicates that there are complex interplays between individual and collaborative learning, analysis and synthesis, learning and task completion, as well as learning and assessment, leading to a complex set of tensions. From our reading of the literature we identified several tensions displayed in Figure 2 that need to be interpreted and handled by teachers, for example, when formulating intended learning outcomes, when devising project tasks, and when planning the assessment of student learning.
Figure 2. Tensions present in the learning and assessment of design in student teams. The enactment of these tensions in the learning environment, and in particular in the assessment procedures, will be interpreted by students and frame their courses of actions both as individuals and groups.
NEED FOR FURTHER RESEARCH
As shown above, previous educational research indicates that there are complex interplays between individual and collaborative learning, analysis and synthesis, learning and task completion, as well as learning and assessment, leading to a complex set of tensions. There is a need to understand how students can be encouraged to integrate knowledge and make more constructive use of disciplinary knowledge, representations, models and modelling in designing. Furthermore, there is a need to improve understanding of how instruction, design tasks and assessment frame students’ learning in collaborative design projects, notably the most salient aspects for students and how these aspects affect students’ courses of action. Most importantly there have been few investigations of students’ collaborative processes of learning in design projects in naturalistic educational settings, as earlier research has largely focused on individual students in experimental settings.
REFERENCES
Agogino, A., Song, S., & Hey, J. (2007). Triangulation of indicators of successful student design teams. International Journal of Engineering Education, 22(3), 617-625.
Atman, C. J., Adams, R. S., Cardella, M. E., Turns, J., Mosborg, S., & Saleem, J. (2007). Engineering design processes: A comparison of students and expert practitioners. Journal of
Engineering Education, 96(4), 359-379. doi:10.1002/j.2168-9830.2007.tb00945.x
Atman, C. J., Cardella, M. E., Turns, J., & Adams, R. (2005). Comparing freshman and senior engineering design processes: an in-depth follow-up study. Design Studies, 26(4), 325-357. doi:10.1016/j.destud.2004.09.005
Atman, C. J., Chimka, J. R., Bursic, K. M., & Nachtmann, H. L. (1999). A comparison of freshman and senior engineering design processes. Design Studies, 20(2), 131-152. doi:10.1016/S0142-694X(98)00031-3
Atman, C. J., Yasuhara, K., Adams, R. S., Barker, T. J., Turns, J., & Rhone, E. (2008). Breadth in problem scoping: A comparison of freshman and senior engineering students.
International Journal of Engineering Education, 24(2), 234-245.
Biggio, M. N., Vázquez, S. M., & García, S. M. (2015). From representation to construction: A study of graphic skills in students newly admitted to architecture and design courses.
Australasian Journal of Engineering Education, 20(1), 95-102.
doi:10.7158/D13-013.2015.20.1
Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education,
32(3), 347-364.
Booth, J. W., Taborda, E. A., Ramani, K., & Reid, T. (2016). Interventions for teaching sketching skills and reducing inhibition for novice engineering designers. Design Studies, 43, 1-23. doi:
10.1016/j.destud.2015.11.002
Bucciarelli, L. L. (2001). Design knowing & learning: A socially mediated activity. In C. Eastman, W. Newstetter, & M. McCracken (Eds.), Design Knowing and Learning: Cognition
in Design Education (pp. 297-314). Oxford: Elsevier.
Bucciarelli, L. L. (2002). Between thought and object in engineering design. Design Studies,
23(3), 219-231. doi:10.1016/S0142-694X(01)00035-7
Carberry, A. R., & McKenna, A. F. (2014). Exploring student conceptions of modeling and modeling uses in engineering design. Journal of Engineering Education, 103(1), 77-91. doi:10.1002/jee.20033
Cardella, M. E., Atman, C. J., & Adams, R. S. (2006). Mapping between design activities and external representations for engineering student designers. Design Studies, 27(1), 5-24. doi:10.1016/j.destud.2005.05.001
Cardella, M. E., Atman, C. J., Turns, J., & Adams, R. S. (2008). Students with differing design processes as freshmen: Case studies on change. International Journal of Engineering
Education, 24(2), 246-259.
CDIO. (2010). The CDIO Standards v 2.0 (with customized rubrics). Retrieved from http://www.cdio.org/files/document/file/CDIOStdsRubricsv2.0_2010Dec8.doc
Craig, D. L. (2001). Stalking homo faber: A comparison of research strategies for studying design behavior. In C. Eastman, W. Newstetter, & M. McCracken (Eds.), Design Knowing
and Learning: Cognition in Design Education (pp. 13-36). Oxford: Elsevier.
Crawley, E. F., Malmqvist, J., Östlund, S., Brodeur, D. R., & Edström, K. (2014). Rethinking
Engineering Education: The CDIO Approach. Cham: Springer International Publishing: Cham.
Eastman, C. (2001). New directions in design cognition: Studies of representation and recall. In C. Eastman, W. Newstetter, & M. McCracken (Eds.), Design Knowing and Learning:
Cognition in Design Education (pp. 147-198). Oxford: Elsevier.
Gilbuena, D. M., Sherrett, B. U., Gummer, E. S., Champagne, A. B., & Koretsky, M. D. (2015). Feedback on professional skills as enculturation into communities of practice. Journal
of Engineering Education, 104(1), 7-34. doi:10.1002/jee.20061
Goncher, A., & Johri, A. (2015). Contextual constraining of student design practices. Journal
Johnson, C., Bull, K., & Osmond, J. (2013). Cooperative design and communities of practice. In Y. Luo (Ed.), Cooperative Design, Visualization, and Engineering (pp. 141-152). Berlin: Springer.
Jonassen, D., Strobel, J., & Lee, C. B. (2006). Everyday problem solving in engineering: Lessons for engineering educators. Journal of Engineering Education, 95(2), 139-151. Juhl, J., & Lindegaard, H. (2013). Representations and visual synthesis in engineering design. Journal of Engineering Education, 102(1), 20-50. doi:10.1002/jee.20001
Kittleson, J. M., & Southerland, S. A. (2004). The role of discourse in group knowledge construction: A case study of engineering students. Journal of Research in Science Teaching,
41(3), 267-293. doi:10.1002/tea.20003
Newstetter, W. C. (1998). Of green monkeys and failed affordances: A case study of a mechanical engineering design course. Research in Engineering Design, 10(2), 118-128. Owen, C. (2007). Design thinking: Notes on its nature and use. Design research quarterly,
2(1), 16-27.
Oxman, R. (2001). The mind in design: A conceptual framework for cognition in design education. In C. Eastman, W. Newstetter, & M. McCracken (Eds.), Design Knowing and
Learning: Cognition in Design Education (pp. 269-295). Oxford: Elsevier.
Samuel, A. E., & Lewis, W. P. (1991). Evaluation of students' strategies for concept application in engineering design. Instructional Science, 20(4), 311-329. doi:10.1007/bf00043256
Schön, D. A. (1987). Educating the reflective practitioner: Toward a new design for teaching
and learning in the professions. San Francisco: Jossey-Bass.
Silva, A., Fontul, M., & Henriques, E. (2014). Teaching design in the first years of a traditional mechanical engineering degree: methods, issues and future perspectives. European Journal
of Engineering Education, 40(1), 1-13. doi:10.1080/03043797.2014.903227
Smith, R. P., & Leong, A. (1998). An observational study of design team process: A comparison of student and professional engineers. Journal of Mechanical Design, 120(4), 636-642.
Steele, C. (2015). Developing engineering design expertise – part 1: framing. Australasian
Journal of Engineering Education, 1-10. doi:10.1080/22054952.2015.1092649
Taboada, M. B., & Coombs, G. (2013, May 14-17). Liminal moments: designing, thinking and
learning. Paper presented at the 2nd International Conference for Design Education
Researchers, Oslo.
Vattam, S. S., & Kolodner, J. L. (2008). On foundations of technological support for addressing challenges facing design-based science learning. Pragmatics & Cognition, 16(2), 405-436.
BIOGRAPHICAL INFORMATION
Jonte Bernhard is Professor in Engineering Education at Linköping University and guest
professor at the KTH Royal Institute of Technology, Stockholm. His current research focuses on engineering students’ practical achievement of understanding, the materiality of learning
in labs and design projects, modeling, and the development of learning environments through design-based-research. He is currently member of the governing board of the international Research in Engineering Education Network (REEN). He has published more than 130 papers and book chapters in material science and in engineering education research.
Kristina Edström is an Associate Professor in Engineering Education Development at the
School of Education and Communication in Engineering Sciences at KTH Royal Institute of Technology, Stockholm, Sweden, one of the founding members of the CDIO Initiative. Her current research takes a critical approach to the “why”, “what” and “how” of engineering education reform.
Anette Kolmos is Professor in Engineering Education and PBL and Chairholder for
UNESCO Chair in Problem Based Learning in Engineering Education, Aalborg University, and guest professor at the KTH Royal Institute of Technology, Stockholm. She is active in developing the profile of Engineering Education Research in Europe and internationally. She was President of SEFI 2009–2011. She has published more than 190 articles in various books and journals.
Corresponding author
Jonte Bernhard Linköping University
Department of Science and Technology (ITN) SE-601 74 Norrköping
Sweden
+4611-36 33 18 jonte.bernhard@liu.se
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.
