Strengthening the Student’s Role in the Work with Quality Improvement and Innovation in Higher Education
BJARNE BERGQUIST, PhD Assistant Professor in Quality Management
Luleå University of Technology, Sweden Bjarne.Bergquist@ies.luth.se
HÅKAN WIKLUND, PhD Professor in Quality Management
Mid Sweden University and Luleå University of Technology, Sweden Hakan.Wiklund@ter.mh.se
PIA SANDVIK WIKLUND, PhD
Director of Administration, Mid Sweden University, Sweden Pia.Sandvik-Wiklund@forv.mh.se
Abstract
The importance of establishing efficient strategies and tools for quality assessment and improvement in higher education institutions has grown successively during the last decade.
Many good examples demonstrating how students can be involved in this work have been presented in the literature. However, a quite reactive approach to student participation is still predominant among many universities. This means that, for example, student satisfaction is measured after each course, often with poor feedback and follow-up. Student satisfaction and everybody’s commitment in the improvement work are examples of fundamental values incorporated in the quality movement and field works show that universities can benefit from a more proactive approach to student participation in the daily and strategic work with designing and improving courses and programmes.
In this paper, several proactive and systematic strategies for strengthening the student’s role in the work with innovation and quality improvement are presented and discussed. The strategies cover methods aimed at identifying student expectations and evaluating how these can be used as a complement to satisfaction measurements, for example by using focus groups. Studies conducted by the authors also show how students can be actively involved in course design by applying quality function deployment and conjoint analysis. New and informal approaches to involve students in the improvement of higher education can give higher student satisfaction and can result in a more creative improvement climate when compared to traditional and more formal approaches.
Introduction
Student involvement and continuous improvement of higher education have become core values at many universities during the last few years, e.g. (Peterson et al., 1997). A consequence of this is an increased attention to studies of measurements of student satisfaction and measurements of
quality of higher education, see e.g. (Harvey, 1995; Rowley, 1996 and Yanhong and Kaye, 1998). Several arenas for researchers and teachers have slowly begun to bridge the gap between the traditional pedagogical and learning disciplines and the principles of Total Quality Management (TQM1), e.g. (Cleary, 1997).
A central concept of TQM is that processes should be improved continuously and customers, such as students, should be focused, e.g. (Clare, 1995). Various tools and techniques common within the TQM movement have proven useful for improvement of various aspects within the educational system (Sandvik Wiklund and Wiklund, 1999a,b). The service quality perspective is also slowly gaining acceptance in higher education. As examples, two approaches trying to conceptualise and measure the quality of educational processes have been applied; the gap measurement approach, referred to as SERVQUAL, and the performance-based approach, referred to as SERVPREF (Zeithaml et. al., 1990; Yanhong and Kaye, 1998). Today, the student is – as in other parts of the service sector - more seen as an active participant in the learning process rather than as a product or customer.
Participation of students in the educational process is beneficial in several ways, e.g.
(Rockler, 1979), and this participative approach goes well along with the basic cornerstones of TQM. Student participation also facilitates the students’ abilities to co-operate in projects, which is an experience demanded by another “customer” of education, namely the future employers, (Harvey and Green, 1994). Hence, it is reasonable to assume that the active involvement of students in course design and production is of importance for the potential success of a course.
A complex question in this context is: who are the best judges of the quality of higher education?
Is it teachers, students, society, government or who? In Sweden for example, the National Agency of Higher Education assesses the quality of all subjects and programs at all the universities and university colleges that offer them; in this case, the central authority is the judge.
Teachers ask students to conduct course evaluations after each course, probably because teachers are interested in the students’ perceptions of the quality of the courses. We can make a long list of interested parties and ways to measure quality, so it is reasonable to assume that the outcomes of higher education interest many customers. An important measure of quality of courses is how the education satisfies student needs, since students are a customer group with large personal investments into education. (However, one should be aware that factors stimulating student satisfaction do not necessarily have to stimulate learning.)
Student satisfaction surveys are commonly used for the purpose of measuring need fulfilment.
Survey results are used as guidelines for revisions to improve courses and increase student satisfaction. Unfortunately, many student satisfaction questionnaires are reactive and are simply handed out during the last lecture, or worse, during the written exam. This is problematic from several aspects. It is too late to implement changes to improve the situation for current students, who may leave courses unsatisfied. Our experience at the Division of Quality Technology and Statistics is that many students select courses on advice from senior students. Changes
1 According to ISO8402, TQM is a “Management approach of an organisation, centred on quality, based on the participation of all its members and aiming at long-term success through customer satisfaction, and benefits to all members of the organisation and society”.
implemented between courses are too late to improve word-of-moth. Students fail to recognise their input as valuable and motivation to fill in future evaluations decline. A severe bias can also be expected towards heavier weighting of events during the last part of the course. Given during exams, the survey will probably be severely biased by the exam.
Many successful strategies to improve student learning through course design have also been suggested over the last few years, see e.g. (Gibbs and Rust,1997), even though active and systematic applications of TQM methods are still in minority. This paper focuses on the student’s role in the work with quality improvement and innovation in higher education and presents a systematic approach to improve education continuously, by involving students in the work to improve courses using tools and methods common within TQM.
Systematic Measurements of Student Responses
It is the experiences of the authors that a more systematic and proactive approach, such as described in Wiklund (2000), can counteract many of the pitfalls related to thoughtless routine sampling of student satisfaction, see Figure 1. This means that
• student expectations are measured at the beginning of each course and feed-back on these results are given after the answers have been combined;
• the ”temperature” of the course is taken at half-time, a quick feedback is given and if the teaching staff agree on the problems, appropriate action is taken immediately;
• the students prepare and conduct by themselves non-traditional and more effective course evaluations and the students together draw up with recommendations;
• the course is ended with a consensus meeting where the student’s work and recommendations are discussed. Further, decisions are taken on which recommendations should be implemented.
By monitoring expectations before courses have started along with satisfaction measurements midterm and near the end of the courses, adjustments are possible during ongoing courses. After each survey, the students receive feedback from the teaching staff regarding suggested changes.
The sizes of course alterations that have been executed during ongoing course, ranges from faster feedback on lab-reports to changes on the examination form.
Expectation Satisfaction 1 Satisfaction 2
Feedback Feedback Feedback
Figure 1. Survey scheme of courses.
Voice of the Student
For many purposes, surveys are too blunt an instrument as responses lack depth. Symptoms are found but causes or cures more seldom. Complementary methods of gathering feedback from students are necessary to assess a course in greater depth.
One of the courses given by the Division of Quality Technology and Statistics in Luleå, Sweden, is called “Total Quality Management through Customer Based Product Development”. As the name implies, the course focuses on how a dialogue with customers may assist the process to create products with a greater potential to satisfy. Course topics include among other things, techniques to gather the voice of the customer including surveys, deep interviews, and focus group interviews. The curriculum thus presents an excellent opportunity to use focus groups to discuss the ongoing course.
At the start of the course, students are grouped into teams of four or five, with the assignment to monitor the current course. The instruments to monitor the course differ, so that one team is working with surveys, and other with deep interviews, focus group interviews etc. The results of the different teams are presented during a seminar near the end of the course and the differences in information are discussed. More on different ways to gather verbal information has been written by (Dutka 1994). In specific, the work described here involving multiple teams is described in (Wiklund 2000).
Quality Improvement Teams
“Quality improvement teams” is one of the subjects of another of our courses. To practise membership, students are grouped into quality improvement teams already at the introductory lecture. The teacher picks five to seven persons for each team, with the goal to create a large background variation. Each group is then given the assignment to monitor a specific topic during the course.
The foundation of TQM consists of core values such as “Everybody’s commitment”, “Process view”, “Customer focus” etc. see e.g. Bergman and Klefsjö (2001). The assignment has been to monitor if subjects such as teachers, course curriculum, fellow students etc. give proof of these core values. One team will for instance monitor if examples of the core value “Continuous improvement” are found and how aspects of the course live up to them. Aspects could be student groups, teachers, course syllabus, the examination form etc. The students also give suggestions of how the course may be changed so that the core values are better reflected. Since the teams monitor different core values, the collected feedback from the teams is highly informative. One improvement team with the mission to investigate the core value “process view” has for instance investigated how transparent the course processes were, including both the overall course process as well as processes within student groups. A group assigned to monitor “systems view”
has verified if the on-going course fitted within the overall course package given by the department. This type of in-depth feedback would be difficult, if not impossible to receive with other techniques. Quality improvement teams are discussed e.g. by Dale (1999).
Student Participation in Course Design
Course design is an area normally restricted to the producers of the course, and commonly without questioning presumptive students. This does not have to be the case, see e.g. Mintz (1997). In the following two concrete examples are given.
Student Designed Examination Form
After completion of a course given in year 2000, the examiner of the course “Total Quality Management through Customer Based Product Development” decided to expand the practical use of tools within the course by changing one assignment. The design aspects of a university course were selected as the target for the task, since that would be an opportunity to use development tools on a genuine application. After a discussion with involved teachers, the examiner chose to let the students design their own examination form.
The examination form is one of the most important instruments of control for student learning (Snyder, 1968; Trowald and Dahlgren, 1999). The examination form is therefore sensitive and must be transparent to the students, and preferably known before the start of the course. This request could not be met if the students were to create an examination form of their own. Student motivation was on the other hand expected to be high, since all the students following the course have gained experience of and formed opinions about several different types of examination forms. Since spring of 2001, the student-based design of the examination form is a feature of the course.
In practice, the examination form is influencing the whole course, and because of this, the assignment is placed at the beginning of the course. During the first lecture, the students are informed of possible problems related to an unspecified examination form and that the timeline for the assignment is short. The necessary theory including general guidelines and recommendation of how customer based design work can be done, theory of customer information collection and of Quality Function Deployment (QFD)2 is presented the following week, and then the assignment is started.
The goal of the assignment is to deliver suggestions about which topics to evaluate for the examination and also how these topics should be examined, but leave the responsibility to set the levels of grades to the examiner. Before the assignment is started, the students are informed of the demands and wants specified by the examiner regarding the examination form, along with the specific time limits for the assignment. The students are also instructed that they have the opportunity to interview teachers and examiner during the assignment to refine their view of needs. The students are then divided into teams consisting of 5-6 persons and the assignment is started.
During the first phase of the assignment, students gather information about the needs of customers of an examination form, and the choice of which customer groups to investigate is left to the students, see Figure 2. Possible examination concepts are also generated. Customer needs
2 QFD is a Japanese method with a purpose to prioritize the aspects of the product that the customer feels as being most important, and includes tools to transfer verbal customer statements to engineering requirements for the products, see for instance Akao (1990).
and examination form concepts are fed into a “House of Quality-matrix” (HoQ), and the different concepts are analysed. The HoQ normally has basic customer needs presented in the left column and product concepts presented in the top row. A HoQ matrix from one of the student groups is seen in Figure 3. The details of the examination form are then worked out based on the merits of the HoQ evaluation.
Figure 2. Schematic arrangement of assignment “examination form”.
When each student team has come up with a suggestion of an examination form, this suggestion has to pass a stop-go check to see if the demands set by the examiner are met. Proposals having passed this test are then candidates for the final examination form of the ongoing course.
Approved forms are presented during a seminar where students and teachers together select the best form by democratic voting on equal basis. The examiner finally sets the grading levels of the form and the form is used for the course.
So far, nine student groups from two courses have worked out suggestions for examination forms. The concept suggestions worked out by the different groups show many similarities. The suggested forms are all based on credit systems, where credits are given to assignments during the extent of the course. Although one group has proposed that higher grades only should be given to students taking a written or oral exam at the end, no group has suggested that a written exam should constitute a major part of the examination form. All groups also stood united in their belief that the team assignments, which include a large fraction of the student effort during this course, should have the greatest importance for the grades. Different ways to create fruitful classroom discussions, including requirements that students should hand in research questions regarding the topic of the lecture to the teachers in advance, were also common among the suggested forms.
At the focus group interviews held at the end of the course, students were questioned about the examination form assignment, and the respondents were unanimous that the examination form assignment was interesting, instructive and had increased motivation.
Written exam Oral Exam Focus group evaluation Discussion of theory sections Individuall summaries of all theory sections Compulsory attendance at lectures, seminars, laboratory work etc. Group members grade each other Point system Only approved or not approved on one assignment, the rest graded Cross-functional presentations of assignments Presentations of assignments before the class Assignments only presented by reports to teacher Individuall grades for all team members Higher grades optional weight
Flexible versus unintentional absence 5 5 5 5 5 5 5 3 5 1
Fast feedback from teachers with regard to assignments 1 3 3 3 5 3 5 3 5 1 3 1
Differentiated grades 5 3 1 1 5 5 3 1 5 5 1
Clear, simple examination rules 5 3 3 3 5 1 5 1 3 5 5 5 3 5 1
Stimulating creativity 1 1 5 5 5 1 1 5 3 2
Increase variation 3 3 5 5 5 3 5 1 5 3 5 1
No tactical loopholes 1 5 1 5 5 1 5 3 5 5 3 3 3 1
Engaging both teachers and students 1 1 1 5 3 1 3 5 3 5 3 1 1
Give lifelong knowledge 1 1 5 3 5 5 1 3 2
Points 17 27 16 32 50 35 14 14 33 32 45 18 35 18
Rank 12 8 13 6 1 3 14 14 5 6 2 10 3 10
5 = strong connection
3 = connection
1 = weak connection
Figure 3. HoQ matrix from one of the student groups (after Granström et al)
Student Involvement in Early Stages of Course Design
With the goal to create a new course in Quality Management at the Division of Quality Technology and Statistics, that would maximize student satisfaction, the project leaders decided to develop the course with tools common within the TQM movement, such as QFD and Conjoint Analysis3 (CA).
Questionnaires aiming at the basic needs that a course has to fulfil in order to reach high satisfaction levels were given to students in two surveys. The respondents were also instructed to prioritise needs.
The prioritised needs, together with course attributes generated by the project leaders were then fed into a HoQ-matrix. The evaluation of the HoQ resulted in a general outline of the new course design. From the general outline, students also prioritised different course concepts, by acting as respondents of two CA surveys. The approach resulted in several positive consequences, such as a higher degree of student commitment and participation and an increased efficiency of the continual improvement work. It also illustrates an example of an activity where students have been invited as collaborators in the continual improvement work where course theory has been applied in practice, see (Sandvik Wiklund and Wiklund 1999a,b).
Discussion and Conclusion
The feedback received from satisfaction measurements clearly shows that the students following the courses of our division appreciate the work to involve them in the course development process. It is also beyond doubt that the systematic work to act on student responses collected during ongoing courses is welcome. In other words, the student satisfaction has been increased.
As a complement, we have introduced a phase of reflective learning where the students have to seriously analyse the course aim, course theory and laborations and then ask themselves how to evaluate their learning in the “best” way. Many students have not thought of their learning in this way.
The students taking our advanced courses praise the hands-on experience received from using customer satisfaction measurements and the tools to transfer needs into product designs. The high degree of involvement both by students and by teachers creates an atmosphere were students and teachers do not look at each other as adversaries, but as partners. The flip side of the involvement-coin is the workload of the courses, which many students consider being on the high side. Students fortunately state that the rate of learning is high, and that their effort is well spent.
It may be argued that the context described in this paper is extremely benign towards work with methods common with TQM, since students who follow our courses do so voluntarily because of their interest in TQM. It is undoubtedly more difficult to start a group assignment to use QFD to
3 CA is an experimental technique that deals with letting presumed customers value product concepts. The concepts are often realistic, in the sense that no concept is ideal; therefore the customer has to make trade-offs between different aspects of the product. With this technique, unspoken needs can be explored. CA is described for instance by (Green and Wind 1975), or more recently by Gustavsson et al (1999).
reform for instance a course in mathematics, since student time is tied up with other tasks.
Several of the methods presented here, such as systematic customer feedback or conjoint analysis, are however easily adapted to other contexts, teaching or other.
In closing, the authors strongly believe that admittance to the core values of TQM is favourable if one strives to go from course status quo towards course excellence. From the experience of the authors, it is clear that student partnership facilitates, or sometimes is a condition for student commitment. In addition, the improved way of working has led to a systematization of the continual improvement work. The stimulating results constitute an example of how the quality of higher education can be improved, simply by utilizing the unexploited potential of its own customers.
References
Akao, Y., (1990), Quality Function Deployment - Integrating Customer Requirements into Product Design. Productivity Press, Cambridge, MA, USA.
Bergman, B. and Klefsjö, B., (2001), Kvalitet från behov till användning (in Swedish).
Studentlitteratur, Lund, Sweden.
Clare, C. (1995). Introducing TQM in a New University: Practical Lessons, In Proceedings of the First World Congress on Total Quality Management, (ed. G.K. Kanji), Sheffield, Chapman & Hall, pp.442-445.
Cleary, B. A. (1997). Improving Classroom Learning with Quality, In Proceedings of the 50th ASQC Quality Congress, pp.1-11.
Dale, B.G., (1999), Managing Quality, 3rd ed., Blackwell Publishers Inc., Malden, MA.
Dutka, A., (1994), AMA Handbook for Customer Satisfaction. - A complete Guide to Research, Planning & Implementation. NTC Business Books, Lincolnwood, Il.
Granström, K., Hedquist, A., Nordström, J., Persson, M., Schön, K., Uhlin, J., (2001), Examinationsmodell genom QFD, (in Swedish), report, IEK044 vt2001. Division ocf Quality technology and Statistics, Luleå University of Technology, Luleå
Green, P. E. and Wind, Y., (1975), New way to measure customers' judgements Harvard Business Review, Vol. 53, July-August, pp 107 - 117.
Gustafsson, A., Ekdahl, F., and Bergman B., (1999), Conjoint Analysis: a useful tool in the design process, Total Quality Management, Vol. 10, No. 3, pp. 327-343.
Harvey, L., Green, D. (1994). Employer Satisfaction (Birmingham, QHE).
Harvey, L. (1995) Student Satisfaction, The New Review of Academic Librarianship, Vol.1, pp.161-173.
ISO 8402 Quality Management and Quality Assurance Terminology.
Mintz, J.A. (1997) When learners become the teachers: student involvement in course design, curriculum choice, and teaching, In Gibbs.,G., Rust, C. (Eds.) Improving Student Learning –
Improving Student Learning through Course Design, Oxonian Rewely Press Ltd., Oxford, pp.179-187.
Nilsson, K.A. (1998) Course evaluations – for the students’ participation in the work with quality. (In Swedish), Lund University: Evaluation office 1998:202.
Peterson, S.L., Kovel-Jarboe, P., Schwarts, S.A. (1997) Quality Improvement in Higher Education: implications for student retention, Quality in Higher Education, 3(2), 131-141.
Rockler, M.J. (1979). Teaching Students to Conceptualise the Future through Simulation/
gaming for Creativity; in J. Megarry (Ed.), Perspectives on Academic Gaming and Simulation 4: Human Factors in Games and Simulations (London, Kogan Page/SAGSET).
Rowley, J. (1996) Measuring Quality in Higher Education, Quality in Higher Education, Vol.2, No.3, pp.237-255.
Sandvik Wiklund, P., and Wiklund, H., (1999b) Application of Quality Tools for Course Design - Improvement of Student Learning and Satisfaction. Improving Student Learning, Vol.6, pp.135-145, Oxonian Rewley Press Ltd., Oxford.
Snyder, B., 1968, Hidden Curriculum. MIT P, MA, USA.
Trowald, N. and Dahlgren, L.O., 1999, Student views on knowledge measurements (in Swedish).
Pedagogiskt utvecklingsarbete nr. 15, Sveriges lantbruksuniversitet. Uppsala, Sweden
Sandvik Wiklund, P. Wiklund, H., (1999a) Student Focused Design and Improvement of University Courses, Managing Service Quality, Vol.9, No.6, 434-443.
Wiklund, H. (2000) Student-Driven Improvement Work, Towards Best Practise – Quality Improvement Initiatives in Nordic Higher Education Institutions (Copenhagen: Nordic Council of Ministers, Tema Nord 2000:501, pp.74-86).
Yanhong, R., Kaye, M. (1998) A Case Study for Comparing Two Service Quality Measurement Approaches in the Context of Teaching in Higher Education, Quality in Higher Education, Vol.4, No.2, pp.103-113.
Zeithaml, V.A., Parasuraman, A., Berry, L.L. (1990) Delivery Quality Service (New York: The Free Press).
