Lärarlärdom 2018
HÖGSKOLAN KRISTIANSTAD
RED: CLAES DAHLQVIST & STEFAN LARSSON
Kristianstad University Press ISSN: 1404-9066
Kristianstad University Press
© Respektive författare 2018
Förord
Konferensen Lärarlärdom 2018 genomfördes den 16 augusti på Högskolan Kristianstad.
Lärarlärdom är en samarbetskonferens mellan Blekinge Tekniska Högskola och Högskolan Kristianstad och går av stapeln varje år strax innan höstens terminsstart. Konferensen erbjuds som en mötesplats för våra anställda där alla kan träffas för kvalificerade samtal kring kvalitet i undervisning och lärande inom högre utbildning.
Vi ser gärna att alla som på något sätt deltar i utvecklingen av våra utbildningar och studenter tar plats på konferensen; Lärare, forskare, doktorander, studieadministratörer, bibliotekarier och personal i olika stödfunktioner som genom åren byggt upp en gedigen kunskap om och reflekterad erfarenhet kring lärande inom högskola och universitet.
2018-års tema var Hållbar utveckling i högre utbildning. Huvudtalare på konferensen var universitetslektor Sally Windsor, nu verksam vid Göteborgs Universitet, Institutionen för Didaktik och Pedagogisk Profession. Utifrån temat satte Windsor en ram för vad
hållbarhetsfrågan omfattas av för ett lärosäte idag och hur lärare kan tänka kring implementeringen i undervisningen. Mer information om den avslutade konferensen hittar du här.
Torsdagen den 15 augusti 2019 är det Blekinge Tekniska Högskola som är värd och konferensen äger rum i Karlskrona. Denna gång är även Malmö Universitet inbjudna att delta. Håll utkik på
https://www.bth.se/lararlardom-2019 där mer information och aktuellt tema finnas tillgängligt.
Utöver själva konferensbidraget har alla som önskar möjligheten att i efterhand publicera fullständiga Papers kring det ämne som
presenterades på konferensen. Flera valde även denna möjlighet och deras bidrag finns att läsa på de följande sidorna.
Mycket nöje!
Stefan Larsson
Högskolepedagogisk utvecklare Högskolan Kristianstad
2
Innehåll
Innovation projects with work integration social enterprises: evidence of challenge-based learning – Marco Bertoni ... 3 Improvements of students learning through changes in Feedback and examinations in Introduction to Strategic Sustainable Development 7,5 credits – Sven Borén ... 26 Impact of the Transition Between Academic Cultures on the Learner’s Academic Performance – Irina Gertsovich m. fl. ... 39 Kvalitetsarbete i högre utbildning – om tillit, tolkning och legitimitet – Åse Nygren ... 53 Hållbar samhällsutveckling - en utmaning för den högre utbildningen – Christel Persson m. fl... 75
3
Innovation projects with work integration social enterprises: evidence of challenge-based learning – Marco Bertoni
Marco Bertoni
Blekinge Institute of Technology, SE-371 79 Karlskrona, Sweden, marco.bertoni@bth.se
Abstract
The Conceiving — Designing — Implementing — Operating (CDIO) framework is a cornerstone of engineering education. In recent years, Challenge-based learning (CBL) has been proposed as an evolution of the original CDIO concept to enhance students’ use of design as ‘learning through’ rather than ‘learning to’. The aim of CBL is that of fostering, among engineering graduates, the identification, analysis, and design of a solution to a socio-technical problem. This is achieved by involving different stakeholder perspectives and aiming to find a collaboratively developed solution, which is environmentally, socially and economically sustainable.
Hence, CBL emphasizes the critical dimension of social sustainability when designing ‘experiences’ for engineering students. This paper explores the pedagogical benefits of choosing Work Integration Social Enterprises (WISE) as case study providers in engineering education. WISE see economic gains most as a means of achieving other (social) goals, such as rehabilitation and work training. This ‘double business idea’, which synthesizes the unique combination of business and social values, makes them promising testbeds for CBL. The objective of this paper is, therefore, to present evidence of challenge-based learning from the analysis of 7 student projects conducted at Blekinge Institute of Technology (BTH) within the Value Innovation course during the fall of 2016 and 2017. The paper first analyses to what extent WISE represent attractive case studies for engineering graduates to work with, compared to more ‘traditional’
company types. It further presents evidence of CBL with regards to problem formulating and designing, entrepreneurial mindset, value-driven, and social-aware learning, as well as the social construction of knowledge.
Eventually, the paper elaborates on how innovation projects with work
4
integration social enterprises (WISE) can represent a step forward in shaping future ‘Global Engineers’.
Introduction and objectives
Engineering systems today are characterized by increasingly complex and multifaceted values. Hence, engineers must become aware not only of the economic and technical aspects of a design but shall also be able to grasp
‘softer’ and more intangible value-adding dimensions in their work. This trend challenges engineering education and stresses the need of providing engineering students with a more holistic/systems view of the world around them. The tension between these growing needs in contemporary undergraduate engineering education is a major driving factor in the development of the CDIO framework (Crawley et al. 2013). CDIO is an educational initiative centered around 12 principles of effective practice that summarize knowledge, skills, and attitudes that alumni, industry and academia desire in a future generation of young engineers. The main message from CDIO initiatives is that the required learning outcomes of engineering education must evolve in relation to the professional role of the contemporary engineer (Huntzinger et al. 2007; Splitt 2003). From this standpoint, the CDIO syllabus is further used to define expected outcomes in terms of learning objectives of the personal, interpersonal and system building skills necessary for modern engineering practice.
In recent years, the original CDIO concept has evolved to highlight the need for engineering graduates to use design as ‘learning through’ rather than
‘learning to’ experiences (see: Kohn Rådberg et al. 2018). Challenge-based learning (CBL) is then proposed as an evolution of CDIO, as well as of more traditional problem-based learning approaches. This novel educational development is inspired by the idea of ‘grand challenges’ (Kohn Rådberg et al. 2018). These are described as issues that critically need to be addressed by society to improve for humankind during the coming century. Grand challenges have not only been very influential in defining directions and scope for national research and innovation agendas (see: European Commission 2014). They have also driven engineering educators to design learning experiences that foster the identification, analysis, and design of a solution to a sociotechnical problem.
In a CBL perspective, sustainability becomes a critical dimension to be leveraged when designing ‘experiences’ for engineering graduates. Since the
5
publication of the Brundtland report (1987), sustainable development has become prominent in engineering, with several initiatives, such as the Framework for Strategic Sustainable Development (FSSD) (Broman and Robèrt 2017) proposing definitions and principles to guide research and education towards a more sustainable society. Nevertheless, while the introduction of social sustainability aspects in engineering education is of foremost importance for the creation of a ‘Global engineer’ (Bourn and Neal 2008), sustainability is often confined to environmental aspects (Missimer et al. 2017) and suffers from an under-development of the social dimension.
The main objective of this paper is to provide evidence of how innovation projects with work integration social enterprises (WISE) represent a step forward in challenge-based learning. WISE are a particular enterprise form that, besides the commercial imperative of providing product and services (e.g., cafés, laundries, recycling centers, and others), offers employment and training opportunities for individuals considered less able to compete in mainstream labor markets, such as the physically and developmentally disabled (Cooney 2016). Intuitively, WISE are of great interest when it comes to foster competing ‘value systems’ and objectives in problem-based learning experiences. On the one hand, they must obey the commercial logic that emphasizes efficiency, profitability and competitive rivalry. On the other hand, they serve a service or social welfare logic, aiming to maximize a program of supportive intervention to produce results for the beneficiary (Cooney 2016).
This paper argues that the multiple goal structure of WISE makes them an ideal testbed for CBL, because such goals reflect different and opposing logic. Yet, in spite of their intriguing mix of business and social values, little is known and even less is published, about the pedagogical benefits of choosing WISE as case study providers in engineering education. Emerging from 7 innovation projects conducted within the Value Innovation course at Blekinge Institute of Technology (BTH) during 2016 and 2017, the objective of this paper is to describe how real-life design projects with WISE has the potential to take CBL a step forward compared with projects involving more
‘traditional’ enterprise forms. The evidence is gathered along 3 main lines of thought, as presented in the following sections, which are:
• evidence of problem formulating and designing;
• evidence of an entrepreneurial mindset and of value-driven learning;
• evidence of social-constructed and social-aware learning.
6
In the Discussion and Conclusions section, the paper further elaborates on educational challenges and open issues concerning the opportunity of
‘exploiting’ WISE on a larger scale to introduce a social sustainability dimension in engineering education.
What are WISE and why caring about them?
The phenomenon of work integration social enterprises (in Swedish:
arbetsintegrerande sociala företag, or ASF) emerged already during the 1990s, awakening interest at both Swedish and European level due to their unique business orientation. WISE combine rehabilitation and work training with the development of social enterprises, as a way for long‐term unemployed to return to the labor market by creating jobs that are adjusted for them. Due to their unique combination of business and social values, WISE are often described as permeated by the so-called ‘double business idea’ (Peverada 2016). While traditional entrepreneurship finds its main rationale on the goal of creating financial returns to its owners, WISE see an economic gain most as a means of achieving other (social) goals (Tynelius 2011), which is that of supporting people in their journey to employment and self-sufficiency.
In Sweden, the number of WISE has significantly increased in recent years.
In the late 2000s, the Swedish Agency for Growth recognized 150 entities at the national level. Just 2 years later, this number had increased by more than 30%, up to 207 enterprises. Nowadays WISE are even more widespread:
about 340 companies were listed at the end of 2017, and WISE can be found in about 100 of the country's 290 municipalities of Sweden, employing approximately 10200 people. A Danish study (Hulgård and Bisballe 2004) – yet numbers are believed to be similar for Swedish ‘ASF’ - shows that main goods and services being sold by WISE are hotel and restaurants services (25%), public services (17%), education (16%), services to enterprises (15%), services to the public administration (15%), and processing industry (9%). By selling both work rehabilitation to the public sector and goods and services in the open market, WISE combine economic growth with the goal of creating integration in the labor market and giving people an opportunity to work based on their own ability. Importantly, the social dimension is not detached from the business one: at the end of the day, it is the ability to generate (even marginal) monetary returns that allows reaching the social goal (Peverada 2016).
7 Value Innovation
Value Innovation is a 7,5 ECTS Master Programme course at Blekinge Institute of Technology. The expression ‘value innovation’ originates from innovation management literature and refers to the creation of new and uncontested market space. This is obtained through the development of solutions that generate a leap of value for customers and users (e.g., by means of new functionalities and/or reduced usage effort), while reducing cost and negative impact on our planet and society. The main objective of the course is therefore to raise students’ understanding of how to develop innovative products and services with a focus on value creation, going from the analysis of customer and stakeholders need, to the generation of innovative concepts, to the creation and verification of value-adding prototypes.
The course introduces students to the Design Thinking (DT) methodology framework (Leavy 2010). DT is an approach for user-centered innovation that has gained increased popularity in the last few decades, both in the industry and in the public sector. DT represents a paradigm shift from the traditional linear problem-solving approaches, being applied to cope with design situations dominated by ambiguity and lack of knowledge (wicked problems). The four phases of the framework – Initiation, Inspiration, Ideation, and Implementation – helps individuals to unleash their innovation potential, and to organize the engineering toolbox when wicked problems are in focus.
The course features lectures on design and innovation, which include a mix of short theory reviews (of methods, tools, and strategies for design and innovation) and active work in different group constellations. They are complemented by workshops and class exercises that give participants a first-hand experience of the most relevant tools in the Design Thinking toolbox. Importantly, course participants are given the opportunity to apply the acquired theoretical base in a ‘real-life’ development project conducted in collaboration with selected company partners. In line with the CDIO framework, the course is designed with an overreaching project work that kicks-off just after the course introduction and stretches along the entire period of the study (8 weeks). Each project is conducted by small cross- functional design teams (4 to 6 participants), which mix individuals from the Master programme of industrial economy (year 4), mechanical engineering (year 5) and sustainability innovation (year 4). Here, the regular interaction
8
with stakeholders from industry and society is an important mechanism for students to gather focused feedback on their learning, as well as to develop their reflections on the topic of value innovation.
Experience and lessons learned from the project work are shared during presentation events in the classroom, while peer evaluation and group coaching (feed forward) are used to stimulate critical reflection regarding the process and the results. Results are gathered in a written report, which constitutes the basis for grading. Individual self-reflections aim at further stimulating students in learning about methods and tools for value innovation.
Redesigning Value Innovation: connecting with WISE to foster CBL
Recent literature has highlighted the benefit of value creation projects for engineering education (Bosman and Fernhaber 2018). These projects connect the traditional scientific method and the engineering design process to business and marketing through a focus on goals rather than problems.
This iterative process promotes a method of solution-focused thinking, which encourages engineering students to think outside the box and to apply active learning and creative thinking to theoretical concepts. An additional benefit of value creation projects is that of increasing motivation for learning by allowing students to see value by connecting real-world applications to the class topic (Bosman and Fernhaber 2018).
An important aspect of designing value creation projects concerns the Operating stage of the CDIO model, which is acknowledged to be the most difficult phase in an academic setting. As discussed by Biggs and Tang (2011), students need to expect success when engaging in the learning task, because nobody wants to do something they see as worthless. At the end of the 2015 edition of the Value Innovation course, there was a general feeling of meaningless about the proposed project works, in particular with regards to those conducted in collaboration with large enterprises. As expressed by one of the students in the course evaluation form:
“The projects are very relevant, but there was an overwhelming sentiment within numerous groups that given such broad prompt they would not be able to deliver an insightful solution within the window of 8 weeks. This carried the feeling of pointlessness in
pursuing bold projects.” (anonymous student in the Value Innovation course 2015)
9
More students the same year pointed out shortcomings in the way projects and company partners were selected. A purposeful choice was made therefore to progressively move from large-scale systems to simpler products and operational contexts, making possible for students to be exposed to aspects of actual operation as well. Another important choice in redesigning the course is that of focusing on companies that geographically positioned close to the students’ main location of study. This was found to be a critical shortcoming when dealing with the Conceiving and Operating stages. On the one end, not being co-located meant for students to be unable to apply their engineering toolbox to gather the actual needs for a solution.
At the same time, the opportunity of observing and testing the solution is operation was strongly limited by the lack of physical proximity with the industrial partners.
Another aspect of interest in redesigning the Value Innovation course concerns the widespread motivation among students to apply their knowledge for the good of society. This feeling was expressed by several course participants in the course self-reflection reports. Not only, but several students also approached the course coordinator at the end of the course to ask for advice on how to exploit their engineering toolbox for the good of different no-profit organizations they volunteered in: recycling centers, the Red Cross and other community centers.
The above considerations paved the way for the course to connect with - and support the design of - the “Social inkludering och Tillväx i Blekinge”
project (http://www.socialinkluderingblekinge.se). This is a 3-year initiative, financed by the European Commission through the European Social Fund that aims at creating structures to support the creation and sustainable growth of WISE in the Blekinge Region. Collaborating partners include Coompanion Blekinge as project coordinator, the municipalities of Karlskrona, Kalrshamn and Ronneby, the Swedish Employment office (Arbetsförmedlingen), the Swedish Social Insurance Agency (Försäkringskassan) and the coordinating agency Finsam.
Students projects with WISE: an overview
During 2016 and 2017, a total of 7 student projects (involving a total of 33 students) were conducted in collaboration with existing WISE in the frame of the “Social inkludering och Tillväx i Blekinge” project. These projects were conducted with a total of 6 different WISE to explore the opportunity
10
for business growth through the development of new value-adding products and/or services. Table 1 provides information about the challenge addressed in collaboration with WISE, detailing the number of students participating in each project, their background and the extent of the to which the CDIO framework was covered within the course.
Table 1: Innovation projects with WISE (A: Problem formulation, B: Idea or model generation, C:
Concept development, D: Testing/evaluation within an academic setting; E: Testing/evaluation by external stakeholders).
YEAR Project name # of students
Students’ discipline background
a b C D E
2016 The Sustainable agriculture experience
5 Mechanical Engineering, Industrial Economy,
x x x x
Product Service Systems innovation in caretaking
5 Mechanical Engineering, Industrial Economy
x x x x
Theo-practical education for asylum seekers
4 Mechanical Engineering, Industrial Economy
x x x x x
A new value proposition for the textile retail market in Blekinge
3 Mechanical Engineering, Industrial Economy, Sustainable Product Service Systems innovation
x x x x x
2017 Kaffestugan: the ‘all-year- around opening’ challenge
6 Mechanical Engineering, Industrial Economy
x x x x x
Redesigning the car washing experience
6 Mechanical Engineering, Industrial Economy
x x x x x
Shoe-polishing Product Service System design
4 Mechanical Engineering, Industrial Economy
x x x x
The main objective with all projects was for students to apply the acquired theoretical base in a ‘real-life’ setting, and to deepen their reflections on the application of different tools thanks to the frequent interaction with selected company partners. The projects were guided by the Design Thinking framework and target the development of value-added engineering products, processes, and systems in a team based-environment. In the first step, students were asked to categorize and describe target groups and customer types for new products and services. In the second step, they analyzed their experience with existing products/services through the use of need finding methods and tools, such as interviews and observation, in a relevant environment. By further analyzing societal and technological trends, they were asked to design and select an innovative product and/or service concepts using systematic innovation approaches in the Ideation stage. In
11
the later Implementation stage, they were asked to assess the value of a new system by operating it, physically or virtually.
The project's results were documented by means of a written project report, which was complemented by a 15-minutes project presentation. Also, each student was asked to formalize his/her reflections from the work in an individual self-reflection report, which was submitted individually. In this paper, evidence of challenge-based learning is gathered mainly from the analysis of this material. Further reflections were collected by analyzing the feedback received in the course evaluation reports, even though literature has shown that student evaluations of teaching (mostly) do not measure teaching effectiveness (see: Marsh 2007; Boring et al. 2016).
WISE: an attractive testbed?
A first underlying question when working with WISE concerns their
‘attractiveness’ for engineering students to work with, compared to more
‘traditional’ company types. In order to understand the level to which students felt motivated in collaborating with WISE in their innovation projects, each student was individually asked at the beginning of the course to rank project proposals from the most (first choice) to the least preferred (last choice). All the projects were presented together in the same format during a project showdowns event. Each of them was first presented by introducing the company name and characteristic, then describing the challenge by means of a title, an overall objective, and a set of activities related to the design thinking process. Noticeably, a total of 7 project proposals were presented in 2016, of which 4 with WISE. Similarly, also in 2017 a total of 7 projects were presented, of which 3 with WISE. At the end of the presentation, students were invited to reflect on which project they would have liked to work with. They were further given 72 hours to communicate their preferences (by email) to the course responsible. It was then possible to measure the number of students who choose WISE as a first and second choice, both in 2016 and 2017.
12
Figure 1: First- and second-hand project preferences 2016 (left) and 2017 (right)
Figure 1 shows that projects with WISE were slightly preferred against projects in collaboration with more ‘traditional’ enterprises. Under the assumption of an equal distribution of first-hand preferences, each of the 7 projects presented in 2016 and 2017 shall have got 14,3% of preferences. In 2016, this would have rendered a preference distribution of 57,14% (i.e., 4 x 14,3%) for WISE, which is lower than the actual 61,29%. Following a similar mental process, in 2017 it was expected for WISE to achieve a preference distribution of 42,85% (3 x 14,3%), a figure that was outperformed by the actual result of 48.65%. Looking at gender distribution (Figure 2), it can be also observed that there were no significant differences between genders when it comes to select projects with WISE and with more traditional enterprises. Among those who indicated WISE as their first-hand choice, about 70% were men and 30% women. These numbers are similar to those for more ‘regular’ projects (74% man and 26% women).
13
Figure 2: Gender distribution among first-hand choices for WISE and non-WISE design projects
Evidence of challenge-based learning
CBL experiences are described as learning situations that expand and deepen both problem-based learning and CDIO (Kohn Rådberg et al. 2018). While in problem-based learning students are posed with a design, research or diagnostic ‘problem’, and the learning takes place through the process of working out the solution (Hmelo-Silver 2004), CBL takes a step forward and highlights the following unique characteristics (Malmqvist et al. 2015):
• finding the design experience starting point in large open-ended problems,
• stressing a value-driven approach to problem formulation and decision-making,
• training of self- awareness, and self-leadership in combination with teamwork in projects that require addressing engineering problem- solving,
• addressing societal concerns, while fostering an entrepreneurial mindset and working method.
Figure 3 shows that CBL experiences are also argued to expand the scope and depth of the CDIO approach. In CBL, products are still developed through a process of conception, design, implementation, and operation, yet a stronger focus is put on problem identification and formulation, on the dialogues with core stakeholders, on the business model components of
14
engineering solutions, and on the societal context and impact of a product rather than just the corporate benefits (Kohn Rådberg et al. 2018). In addition, CBL experiences also seek to foster the ability of teamwork, and personal awareness, by considering ‘values’ and ethics in addition to customer needs in decision-making (Kohn Rådberg et al. 2018).
Figure 3: Evolution from traditional to problem-based to challenge-based education (adapted from Malmqvist, Kohn Rådberg, and Lundqvist 2015).
Evidence of problem formulating and designing
Engineering is many things to many people. But, on the bottom line, engineering does ‘design’, which in turn means that take decisions: how to configure a device or system? What materials, dimensions, and tolerances?
What manufacturing process? It is through his/her decision making, that the engineer is manipulating nature to benefit, at least, some segment of society (Hazerlrigg, 2012). Yet, there are situations in which it is:
“less apparent where problem centers lie, and less apparent where and how we should intervene even if we do happen to
know what aims we seek” (Rittel and Webber, 1973).
The problems that are encountered when exercising the engineering profession are often not well defined, rather:
“they tend not to present themselves to practitioners as problems at all but as messy, indeterminate situations”
(Schon, 1987, p. 4).
15
Yet, a recent study from Nespoli et al. (2018) shows that most design problems that are currently addressed by engineering students during their academic terms are still broadly-defined (as opposed to not-defined or ill- defined), only requiring the application of coded technical and scientific knowledge. Hence, a main feature of CBL is ‘problem formulation’.
Disciplinary knowledge allows a student to solve the problem right, but an integration of broader skills is necessary to teach students to solve the right problem.
WISE have shown to provide a relevant ground for students to practice their problem reformulation skills. Table 2 shows the level to which the design challenge was reformulated from the initial description provided at the beginning of the course (which is, from the project showdown event and from the initial design brief with the company partner). Noticeably, most of the initial objectives were reformulated emerging from the findings of the Initiation and Inspiration stage of the Design Thinking process, as well as from the continued dialogue with the collaborating company partner.
The main conclusion from Table 2 is that from a CBL viewpoint, WISE have been successful, in most cases, in stimulating students in independently formulating the problem to be addressed.
Table 2: Extent of problem reformulation from the initial design brief
Y E A R
Project name Extent of reformulation
INITIAL DESIGN BRIEF
PROBLEM
REFORMULATION
2 0 1 6
The Sustainable agriculture experience
SIGNIFICANT
Development of a machine for plastic recycling to be used in an existing showroom.
Re-design of the entire customer experience for the showroom within the sustainable agriculture theme.
Product Service Systems innovation in caretaking
MODERATE
Development of subscription packages for caretaking services.
Development of a Product Service Solutions in the catering business.
Theo-practical education for asylum seekers
MINIMAL
Development of education and training activities for asylum seekers.
Development of a course package and related practical training activities for asylum seekers.
16
A new value proposition for the textile retail market in Blekinge
MODERA TE
Development of innovative product (one-sale model).
Development of a Product Service System solution including communication channels.
2 0 1 7
Kaffestugan:
the ‘all-year- around opening’
challenge
MINIMAL
Redesign of the café’
experience to make it attractive during winter months.
Development of a Product Service System solution for the
´winter café experience.
Redesigning the car washing experience
MODERA TE
Designing a solution to make the washing process more effective.
Designing a user- and employee- centered car washing experience including new layout design and operations management.
Shoe-polishing Product Service System design
SIGNIFIC ANT
Development of a shoe polishing service.
Development of a Product Service System solution to wash and recondition work clothes.
The analysis of the students’ self-reflection reports further shows that collaborating with WISE was acknowledged to be beneficial to leverage the ability to deal with wicked problems. WISE were perceived to be more
‘heterogeneous’ than traditional enterprise forms. Also, students lifted in their reflections the unique context in which these companies operate. Each problem was perceived to be novel and inimitable, and solutions needed to be carefully developed on that basis, reinforcing the wicked dimension.
Interestingly, students working with WISE were observed in their reflection reports to be comparably more aware than the control group that no single design solution is either ‘good’ or ‘bad’, but that there are multiple different ways of addressing the problem that are not always compatible with each other. Also, students were observed to consider more the fact that a solution may be favorable at one point in time, but highly problematic at another, which is one of the driving characteristics of wicked problems with a sustainability orientation (Lönngren 2014).
Evidence of an entrepreneurial mindset and of value-driven learning
Engineering graduates are often acquainted with the scientific method and the engineering design process. However, they are most comfortable — and sometimes quite good at — focusing on the technical feasibility of a solution (Bosman and Fernhaber 2018). Yet CBL aims to foster an entrepreneurial mindset in the engineering graduates. This means that engineers need to design new products and services with the value proposition and user needs in mind, and not simply based on technical and functional concepts taught
17
in the traditional engineering classroom (Bosman and Fernhaber 2018).
There are many innovations that are technically feasible but that do not make any business sense. Also, there might be no physical or technical constraints in introducing new solutions to a market, yet they might not be desired by its customers or stakeholders.
As the design process moves from discovery to evaluation and then to exploitation, engineers continually iterate through the engineering design process with the ultimate goal to answer questions including: Do they want this? Can we do this? Should we do this? – which is to validate, technologically feasibility, customer desirability, and business viability.
This is another major feature that makes CBL differ from traditional problem-based learning: challenge-based experiences shall be conceived to serve a broader purpose than just ‘designing’ a hardware, which is they shall contribute to added values for the society (Kohn Rådberg et al. 2018).
The main question arising from the WISE collaboration is the following. Are students able to take in value perspectives that are more than just functionality and cost? Both the final project reports and the individual student self-reflection at the end of the course were analyzed to find evidence of the effectiveness of WISE projects to enhance students’ understanding of the meaning of “value” in design.
A major activity in the DT process implemented in the Value Innovation course is the use of concept selection matrixes, to support down selection of innovation ideas generated through brainstorming or other methods. These matrixes (such as Analytical Hierarchical Process, or TOPSIS) are based on the comparison of solution concepts on the basis of a set of value criteria, which are derived from the initial customer needs to be satisfied when designing new products and/or services. One way to measure how much students were able to expand the traditional ‘functional’ and ‘performance’
view typical of Basic level courses, to include softer aspects of value, was then to code and measure how such value drivers were defined by the different groups. The results of this investigation are presented in Figure 4, in form of 2 different data sets.
18
Figure 4: On the definition of value criteria for design concept selection
Figure 4 (left) shows the number of stakeholders considered in the list of value drivers used by students to down select design concepts (converge) after conducting brainstorming activities (diverge). It is noticeable how students collaborating with WISE were able to address, on average, a higher number of stakeholders (3,6) when defining value-related criteria compared with other groups (2,4). Importantly, teams working with WISE were found not only to consider the needs of the customer (a common feature of the control group) in their down selection exercise (right). Rather, customer- related value drivers count only for about half of the criteria used (on average) by the team to measure how much a design solution is good (i.e., is able to create value). Almost half of the criteria selected reflects, instead, value aspects related to the provider organization, to the employees working there and to other stakeholders (e.g., employment agencies, mentors, etc.).
These results show that one important aspect of working with WISE is that allow students to experience the multifaceted meaning of the word ‘value’
in engineering. WISE, due to their ‘dual’ business idea, emphasize the pursuit of different goals and objectives, stressing that competing value systems and objectives exist for each problem – due to the multiple stakeholders who each have different interests. While profit and ROI are often used in engineering education as the main metric for ‘value’, for both tangible and intangible goods, the project activity with WISE gave participants a deeper insight on the meaning and metrics for value.
19
Evidence of social-constructed and social-aware learning
The main reason for bringing in WISE in the engineering curricula discussion is that several key issues and skills which define the global dimension of engineering have a social nature. Design work can be therefore understood as a socio-technical business in “the debates about whether the design is ‘done’, if the specifications have been ‘met’ and if the result is
‘good.” (Minneman 1991, p.63). It is therefore interesting to notice how WISE fostered the social construction of knowledge among the participating student groups. WISE made students interact with a wider number of stakeholders (employees, mentors, supporting organizations) than in a more traditional project setting.
This is because WISE are typically started in the form of projects, which means that stakeholders from regional and local development funds are actively involved in the development of the business, often with the help of a mentor and of external professional support (Peverada 2016). Due to the complexity of the target group (i.e., long-term unemployed individuals), it is not uncommon for a person to have between 10-20 contacts dealing with different authorities and care institutions, from career supporters, unemployment agencies contacts and more (Peverada 2016). This strong multi-stakeholder structure was found to facilitate the social construction of knowledge among the student teams collaborating with WISE. Furthermore, by fostering the necessity of gathering knowledge from a wide range of sources, WISE allows students to see how knowledge is a useful tool for problem-solving. On the other end, they were observed to positively stimulate mutual learning and “kamratåterkoppling” (Elmgren and Henriksson 2010). WISE meant for students to get in touch not only with business people but also with a variety of enthusiasts and volunteers that were seen as models, to admire and identify with (Biggs and Tang 2011, p.36).
Another aspect of interest in CBL is social-aware learning. In the definition given by Bourn and Neal (2008) when describing the vision for the ‘Global Engineer’, social-aware learning refers to the ability of:
“appreciating the social and impacts of a business, recognizing the notion of socially responsible investment and
aligning shareholder and social value” (Bourn and Neal 2008)
20
Design projects are acknowledged as a great opportunity to embed these aspects within the undergraduate curriculum (Bourn and Neal 2008). In order to find evidence of social-aware learning, the students project reports were analyzed from the point of view of how much the different projects include aspects related to social sustainability in concept down-selection activities. The social sustainability principles expressed in the FSSD framework (Missimer et al. 2017) were used to verify whether students embedded a social perspective in their value analysis. The 5 social sustainability principles are defined as the following (Missimer et al. 2017):
- Health: individuals shall not be exposed to social conditions that systematically undermine their possibilities to avoid injury and illness; physically, mentally or emotionally, e.g. dangerous working conditions or insufficient wages.
- Influence: individuals shall not systematically be hindered from participating in shaping the social systems they are part of, e.g. by suppression of free speech or neglect of opinions.
- Competence: individuals shall not systematically be hindered from learning and developing competence individually and together, e.g.
by obstacles for education or insufficient possibilities for personal development.
- Impartiality: individuals shall not systematically be exposed to partial treatment, e.g. by discrimination or unfair selection to job positions.
- Meaning-making: individuals shall not systematically be hindered from creating individual meaning and co-creating common meaning, e.g. by suppression of cultural expression or obstacles to co-creation of purposeful conditions.
Table 3 shows that most of the project teams included social sustainability aspects in the analysis and down-selection of ideas, showing a greater awareness than the control group on the use of ‘social lenses’ to measure the goodness of a proposed solution concept. Noticeably, the below social sustainability aspects are much less leveraged in projects conducted with more traditional enterprise forms, in particular with regards to the
‘influence’, ‘competence’, ‘impartiality’ and ‘meaning making’ dimensions.
21
Table 3: Social sustainability principles coverage (x: covered, P: partially covered) when defining value criteria for concept down selection
Project name HEALTH INFLUENCE COMPETENCE. IMPARTIALITY.
MEANING- MAKING The Sustainable
agriculture
experience x x
Product Service Systems innovation in
caretaking p x
Theo-practical education for
asylum seekers x x x
A new value proposition for the textile retail market in
Blekinge x
Kaffestugan: the
‘all-year-around opening’
challenge Redesigning the car washing
experience x p
Shoe-polishing Product Service
System design p
Discussion and conclusions
The main driving factor in selecting WISE as case study providers was to ensure a quality design-implement (D-I) experience for students. Advanced D-I experience (Crawley et al. 2011) are characterized by tasks of increased complexity and authenticity. These experiences are key features of the CDIO programme: they allow students to design, build and assess an actual product, process or system in a way that the object created is operationally testable by students. Importantly, this design-implement experience is not conducted in isolation. aiming as part of a sequence of other experiences at Basic level (Crawley et al. 2011), i.e. more traditional product development courses. This means that creative design activities include business development aspects, vs. multiple objective design as in previous courses.
Also, the design task requires now the contribution from different disciplines, becoming fundamentally cross-functional.
22
WISE-based design experiences have shown to bring forward certain additional characteristics compared with more ‘traditional’ problem-based and CDIO learning. Students have expanded the scope and depth of their problem identification and formulation activities. Dialogue with core stakeholders was characterized by several iterations. Business model components to engineering solutions have been emphasized and a greater focus on societal context and impact of a product rather than just the corporate benefits permeates concept down selection activities.
WISE have proven to foster a process where students can couple theoretical and practical learning, establishing the link between what they see in the classroom and what they read in the books. Also, working with WISE has shown to be an eye-opening experience for students to recognize that the value generation process is not merely a matter of building the solution (feasibility), but also of addressing hos customer and stakeholders will react (desirability) and of ensuring that the solution is sound in a business sense (viability). The results of the analysis show that the students perceive that they have developed deep skills in problem formulation and sustainable development, as well as working across disciplines and with different stakeholders.
Future work will aim at consolidating the use of WISE as case study providers, strengthening the collaboration with all the different actors involved. A step forward in this perspective for the Value Innovation course was to become part under 2018 of the working model for BLINKEN, which is the regional incubator for WISE in the Blekinge Region (http://www.socialinkluderingblekinge.se/blinken-inkubator). Future work will also be dedicated to strengthening practices with regards to the supervision and tutoring of the project groups. One major factor affecting successful project work and learning process was the guidance provided by teacher acting as tutors. Active involvement and guidance were required especially during the first weeks of the project. Most of the guidance took place in the project work sessions, where noticeable differences were observable between project groups. A major aspect to be addressed in these sessions was that to ensure that students engaged with WISE could reach the required technical depth for design solutions to be implemented and operated, which is finding a trade-off between the time spent for development work and the time spent to interact with the actors in society.
It could also be observed that some groups were more innovative and got
23
started with the project very fast and they were able to decide the role of each group member easily when others required more support. It requires professional skills from the teachers to see where and when additional guidance is required, yet still remaining purely as a tutor and not to influence the problem-solving process by providing solutions.
References
Bernard, J., Edström, K. and Kolmos, A. (2016) Learning Through Design–Implement Experiences: A Literature Review. Proceedings of the 12th International CDIO Conference, June 12–16. Turku: Turku University of Applied Sciences.
Biggs, J. and Tang, C. (2011) Teaching for Quality Learning at University (4th edition).
Open University Press, New York.
Boring, A., Ottoboni, K., and Stark, P. (2016) Student evaluations of teaching (mostly) do not measure teaching effectiveness. ScienceOpen Research.
Bosman, L., and Fernhaber, S. (2018) Teaching the Entrepreneurial Mindset to Engineers. Springer.
Bourn, D., and Neal, I. (2008) The Global Engineer: Incorporating global skills within UK higher education of engineers. Department of International
Development. Available at:
http://discovery.ucl.ac.uk/10000839/1/Bourn2008Engineers.pdf Broman, G.I., and Robèrt, K. H. (2017) A framework for strategic sustainable
development. Journal of Cleaner Production, 140:17-31.
Bruntland, G.H. (1987) Our common future: Report of the World Commission on Environment and Development. World Commission on Environment and Development.
Cooney, K. (2016) Work Integration Social Enterprises in the United States: Operating at the Nexus of Public Policy, Markets, and Community. In Nonprofit Policy Forum, 7(4): 435-460.
Crawley, E.F., Malmqvist, J., Lucas, W.A. and Brodeur, D.R. (2011) The CDIO syllabus v2. 0. An updated statement of goals for engineering education.
In Proceedings of 7th International CDIO Conference, Copenhagen, Denmark. June 20 – 23, 2011
Crawley, E., Malmqvist, J., Östlund, S., Brodeur, D. and Edström, K. (2014) Rethinking Engineering Education: The CDIO Approach. 2nd ed. New York:
Springer Verlag.
Elmgren M and Henriksson A-S (2010) Universitetspedagogik, Norstedts.
European Commission (2014) Climate Action. DG for Communication https://europa.eu/european-union/topics/
24
Hazelrigg, G.A. (2012) Fundamentals of Decision Making for Engineering Design and Systems Engineering, 2012.
Hmelo-Silver, C.E. (2004) Problem-based Learning: What and How do Students Learn? Educational Psychology Review 16(3): 235–266.
Hulgård, L., and Bisballe, T. (2004) Work integration social enterprises in Denmark (No. 04/08). Working papers series.
Huntzinger, D.N., Hutchkins, M.J., Gierke, J.S. and Sutherland, J.W. (2007) Enabling Sustainable Thinking in Undergraduate Engineering Education.
International Journal of Engineering Education 23(2):218–230.
Kohn Rådberg, K., Lundqvist, U., Malmqvist, J. and Hagvall Svensson, O. (2018) From CDIO to challenge-based learning experiences–expanding student learning as well as societal impact? European Journal of Engineering Education, pp. 1-16.
Leavy, B. (2010) Design thinking-a new mental model of value innovation. Strategy and Leadership. 38(3):5-14.
Lönngren, J. (2014) Engineering Students’ Ways of Relating to Wicked Sustainability Problems. Licentiate Thesis, Chalmers University of Technology, Gothenburg, Sweden.
Malmqvist, J., Rådberg, K.K. and Lundqvist, U. (2015) Comparative analysis of challenge-based learning experiences. In Proceedings of the 11th International CDIO Conference, Chengdu University of Information Technology, Chengdu, Sichuan, PR China.
Marsh, H. W. (2007) Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness. In: The scholarship of teaching and learning in higher education: An evidence-based perspective, Springer, Dordrecht. pp. 319-383
Minneman, S.L. (1991) The social construction of a technical reality: empirical studies of group engineering design practice. Doctoral dissertation, Stanford University.
Missimer, M., Robèrt, K. H. and Broman, G. (2017) A strategic approach to social sustainability–Part 1: exploring the social system. Journal of Cleaner Production, 140:32-41.
Nespoli, O., Hurst, A. and Russell, J. (2018) Facilitating need finding and problem formulation during cooperative work terms through virtual instruction- pilot implementation results. Proceedings of the 15th International DESIGN Conference (DESIGN), May 21st-24th, Dubrovnik, HR.
Peverada, E. (2016) Stödstruktur för WISE i Norrbotten: slutrapport. Available at:
https://nord.coompanion.se/wp-
content/uploads/sites/30/2016/06/Slutrapport_Stödstruktur_juni2016.
25
Rittel, H.W. and Webber, M. M. (1973) Dilemmas in a general theory of planning.
Policy sciences, 4(2): 155-169.
Schon, D.A. (1987) Educating the Reflective Practitioner: Toward a New Design for Teaching and Learning in the Profession, Jossey-Bass, San Francisco.
Splitt, F.G. (2003) The Challenge to Change: On Realizing the New Paradigm for Engineering Education. Journal of Engineering Education 92(2):181–
187.
Tynelius, U. (2011) Arbetsintegrerande sociala företag - användning och behov av statligt finansieringsstöd. Tillväxtanalys, p. 19.
26
Improvements of students learning through changes in Feedback and examinations in Introduction to Strategic Sustainable Development 7,5 credits – Sven Borén
AUTHORED BY DR.SVEN BORÉN, DEPARTMENT OF STRATEGIC SUSTAINABLE DEVELOPMENT AT BLEKINGE INSTITUTE OF TECHNOLOGY IN KARLSKRONA,SWEDEN
Abstract
Each year at the Blekinge Institute of Technology, in total 15-30 international master students from the Structural Dynamics and the Erasmus program take a 7.5 credit course in strategic sustainable development. In 2016, the course had rather good scores from the students’ course evaluation, but the students also identified a need for improvements regarding feedback from teachers during the course and also regarding the examination of the course. A study was therefore initiated, aimed at finding out if changes in feedback and examinations can increase students’ learning towards the learning outcomes in the course, and if so, how the teaching and examination in the course could be developed in such direction. The study found through literature review, interviews among colleagues, and surveys among students that instead of a final exam, several knowledge tests and reflective questions during the course could increase the students’ learning. The course design was developed and then implemented in 2017. According to the students’
course evaluation the same year, this lead to even better perceived learning.
Except from providing further evidence that students’ learning increases with individual feedback, the study showed how that can be implemented.
Background
In 2016, the course ‘Introduction to Strategic Sustainable Development 7,5 credits (SL2508/2531) had in general high scores in the course evaluation (Students in SL2508, 2016), but despite of that, 42 % perceived that improvements were needed in the course. In the course evaluation, areas of improvements could be identified as some students perceived that the feedback during the course was not so valuable for their learning, and some believed in the free text boxes that the feedback on the second group project was late. The course evaluations showed that few students perceived that the exam have given them the possibility to show that they have attained the learning objectives.
27
Hattie (2009) recognized that students’ learning can be increased via feedback while the course is running, rather than measuring their learning at the end of the course with a final exam. This concept fits courses with not so many students, as it otherwise increases teachers’ workload heavily, which would probably not be the case for SL2531 as it has less than 30 students. Although the concept probably could address areas of improvements that were identified in students’ course evaluation in 2016, it would be preferable to do a scientific pre-study to investigate if and how students’ learning can be increased with a focus on feedback and examinations in the course.
About the course SL2531
The course is given in study period one and on advanced level and designed for international engineering students on master level in the Structural Dynamics program (MTAMT), the sustainable product innovation program (MSPI), and the Erasmus exchange program at Blekinge Institute of Technology (BTH). The course is based on the corresponding course (SL2533) in the Master program about leadership for strategic sustainable development (MSLS) and have some lectures in common. Staff at the TISU department has since 2011 managed the course, and since 2016 with the author of this report as course responsible and with Henrik Ny as the course examiner.
According to the course syllabus (Borén and Broman, 2017), “The purpose of the course is that the student will develop deepened knowledge about society’s sustainability challenges and how organizations can address these challenges from a systems perspective by means of a structuring and coordinating methodology for strategic sustainable development. The student will also develop skills in applying this methodology and ability to critically reflect upon it in relation to supplementary concepts, methods and tools of relevance to strategic sustainable development.”
On completion of the course, the student will be able to…
(a) …describe society’s sustainability challenges and their underlying causes.
(b) …describe and reflect on the major components of a methodology for strategic sustainable development (FSSD).
(c) …identify and describe different tools, methods and concepts within the field of sustainable development and how these relate to the FSSD.
28
(d) …use the FSSD to structure and coordinate knowledge, concepts, methods and tools for analysis, assessment, and planning within the area of sustainability.
(e) …assess and critically relate to different theories, concepts, methods and tools of relevance to strategic sustainable development,
, where ILO “a-c” are categorized as knowledge and understanding, “d” as abilities and skills, and “e” as judgment and approach.
Regarding teaching/learning activities (TLAs), the course syllabus (Borén and Broman, 2017) describes them as:
“The teaching in the course includes lectures, workshops, supervision, group dialogues, individual assignments, group works and reflection individually and in groups. The lectures introduce theories, concepts, methods and tools.
These are deepened, applied, integrated and reflected upon through the other learning items. Teachers with many different scientific backgrounds, professional experience and perspectives take part in the course. The students’ different educational backgrounds, professional experiences and cultural backgrounds are also taken advantage of in the learning process.”
The examination was in SL2508 divided in two group projects (2 credits each), and a final exam (3,5 credits). The examinations for 2017 were instead planned to consist of one oral exam (1 cr) after three weeks on the students’
learning in the first module, and then in two group projects (4 cr in total) by presentations and a written report. The new assessment in 2017 (3,5 cr) replaced the previous final exam, and was planned to include a reflective journal and knowledge tests.
Aim and scope
The goal of this study was to find out if changes in feedback and examinations can increase students’ learning towards the learning outcomes in the course, and if so, how the teaching and examination in the course could be developed in such direction.
Methods
This study started by investigating how increased individual support via feedback could be applied, and then analyzed the alignment of the Intended Learning Outcomes (ILOs) with the remaining and new Teaching/Learning Activities (TLAs) and Assessment Tasks (TAs). Surveys were then handed
29
out to students to see if they think some solutions for increased learning via increased feedback would have increased their learning. At last, a comparison between course evaluations in 2016 and 2017 were made to see if students perceived that the changes in the course TLAs and ATs were beneficial for their learning.
Literature studies and reflective journal design
Relevant literature about formative assessment was analyzed to see if earlier studies have discussed similar issues regarding feedback and examinations, and if there were any best practices in similar courses. This was achieved by using web-based search tools and scientific databases.
Constructive alignment
The concept of constructive alignment was applied to increase knowledge about how well the learning outcomes are aligned with the new course design, which included changes in feedback and examination. According to Biggs and Tang (2011), alignment of the TLAs and the ATs with the ILOs can be possible by using verbs in the ILOs that specifies what, how and to what standard the student should learn. These verbs should then be addressed in the TLAs, and ATs to show how well students learning have been met.
This study evaluates how well the existing TLAs and ATs meet these requirements, and thereafter discusses if, and possibly how, they could be better met.
Student surveys
Based on the results from the annual course evaluations from previous years, the students that attended the course 2016 where interviewed about if and how feedback during the course would have increased their learning towards the learning outcomes. The survey questions were posed in May 2017 via the learning management system ‘it’s learning’ to the students in the previous year course (SL2508). The survey answers were anonymous.
The annual course evaluation in October 2016 and 2017 included questions to see how well the students believed that TLAs and ATs in the course supported their learning. This study used these responses to analyze if the students believed that changes in the course were increasing their learning.
30 Results
Literature studies on feedback for increased learning
Feedback from teachers to enhance students’ learning can be done by several methods, e.g. via tutorials, comments on results exams/assignments, electronic messages, etc. Another method is reflective learning, which supports life-long learning and can be seen as one of the most effective learning methods as it uses metacognitive skills (Biggs and Tang, 2011).
Moreover, reflective learning is widely used in academia because of the following benefits for students learning; providing data as a starting point for learning; centring students in the learning process; promoting creativity in learning; and encouraging critical reflection, as well as benefits for the instructor regarding; pedagogy; support for instructors as teachers, researchers and administrators; and relationships with students (O Connell and Dyment, 2011). Added to that, O Connell and Dyment state that there might be problems with reflective journals when; no training or structure is provided to the student; ‘writing for the instructor’ is practiced; overuse of journaling; negative perceptions of journaling; journals do not meet the needs of all students; gender differences; ethical concerns; assessment issues; legal considerations; time requirements; and quality of reflection.
This calls for a well-thought-through approach when designing questions for reflective journals, and a couple of models/guidelines, e.g. by Bradbury- Jones et al. (2009), Leung et al. (2010), Morrison (1996), Sen and Ford (2009), and Yang (2009), have been developed to meet challenges mentioned above. Moreover, Hatton and Smith (1995) claim that there are four criteria for the recognition of evidence for different types of reflections;
descriptive writing, descriptive reflection, dialogic reflection, and critical reflection. For novice reflection writers, it might be hard to achieve the most preferable level ‘critical reflection’, but at least the first ‘descriptive writing’
needs to be avoided, as it is not reflective and only describes literature or events.
Framing a reflective journal and knowledge tests in SL2531
Based on the literature review, the reflective journal concept seems to fit this course as it will provide feedback to the students during the course without increasing teachers’ workload as the number of students is rather low (15- 30). Another important factor for choosing this method is that teachers have good experiences of the concept from earlier courses.
