GPS-PERFORMANCE IN TECHNOLOGY EDUCATION part II
Eva Hartell
Royal Institute of Technology
There are strong connections between assessment and learning. Assessment can have many purposes. One purpose is when the teacher acquires information in order to adjust their teaching to better meet the pupils’ needs for future progress on their
learning journey. This paper provides findings from a qualitative study that explore and describe the process of assessment in Technology education in the Swedish compulsory school. How do teachers follow up their pupils' progress? What equipment/assessment tools do they use, in order to 'locate' their pupils and move them forward on their learning journey? The results are based on classroom observations and the teachers' written assessment documentation.
Key words: technology education, formative assessment, follow-up, Global Positioning System (GPS)-performance in education
Introduction
This paper aims to describe findings from a study regarding teachers’ work with assessment with the purpose of moving their pupils forward on their learning progress in the mandatory school subject technology. Thus, emphasizing assessment as the link between teaching and learning.
Navigare Necesse Est
The importance of navigating at sea is familiar to me, after years of sailing with our family boat. Teaching could, to my experience, in many respects be seen as a similar activity to teaching and learning, which put demands on all the participants. Despite thorough planning; you still need to make frequent checkups since you neither know exactly what will happen during the journey in advance nor which way to take to the wanted destination. Sailors use different equipment to navigate e.g. stars, nautical charts and their former experience from sailing and from the surroundings. Lately it is also possible to use a Global Positioning System device, a GPS-device. In order to navigate with the GPS, to alter the gap between your current and your wanted position, you need four things: (1.) a GPS-device, (2.) accurate software, (3.) information from three different satellites, and last but not least (4.) knowledge to interpret and translate the information into the current settings. Depending on the performance of your GPS- device and your own prior experience, you can get various accurate position precision.
Neither teaching nor sailing is an easy laid back activity. This, I find, is part of the fascination with the journey, both as a sailor and teacher.
Where are we and Where are we Going?
The teacher has to start from where their pupils are currently situated and take their teaching from there (Kimbell, 2007). Teachers often assume that they are able to determine if their pupils are ‘on track’ or not. This assumption is often based on what the teachers themselves think that they have taught their pupils or not, or that pupils are moving forward in the same pace as the teaching (Kimbell, 2007; Wiliam, 2011). This is somewhat problematic. Firstly everything taught is not necessary learnt by the pupil.
Secondly pupils learn stuff in other surroundings as well. The teacher’s interpretation of gathered information can also be influenced and biased by the teacher’s prior experience and view about the pupils as individuals and as a group. Both the teacher’s and the
learner’s expectations regarding her/his possibilities to achieve the goals set up may also affect the results (Rosentahl & Jacobson, 1992; Kimbell, 2007, Hattie, 2009).
Teachers need to be aware of the difficulties of assessment in order to deal with it properly and consciously (Kimbell, 2007). Moreland et al (2008) cues the importance of the teachers knowing where they are going; it helps the pupils learn. When the teachers involve their pupils and clarify the criteria for success it has a countervailing effect where all pupils’ benefit but the low-achievers benefit the most (Jönsson, 2010).
According to Hattie (2009) and Jönsson (2010) it is successful for teachers to share worked examples of prior pupil work of different quality.
It is important to be aware of the variety of knowledge to build upon and how the learning takes place for the individual and the group as a whole. The teacher’s ability to adjust their teaching based on the pupils’ different starting points and to bridge the gap between the current and the wanted destinations is the characteristics of assessment for learning (Black & Wiliam, 1998). This requires thorough planning including questions to ask along the way and adjust what is happening in the classroom from the evidence gathered to meet the pupils’ needs (Kimbell, 2007; Wiliam, 2011). Considerable research shows that this planning benefits from being done together with others (Leahy et al, 2005; Moreland et al, 2008; Black, 2008; Pettersson, 2009; Wiliam, 2011).
Teachers who lack others to discuss with and only have their own experience to rely on are at risk of getting non-aligned with the current view of learning and steering
documents (Pettersson, 2009).
Documenting and Follow-up in Sweden
Within the Swedish context the sidelined technology subject has been mandatory since the beginning of the 1980s and is still lacking thorough roots and tradition (Hartell, 2010). Sweden has a strong tradition of classroom assessment, and relies on the belief in teachers’ ability to independently assess pupil’s knowledge and decide on what grade to be given (Klapp Lekholm, 2009). The state of knowledge among the Swedish
youngsters in technology is not easily interpreted and described (Hartell, 2011). Some national tests are provided as support material by The Swedish National Agency for Education (NAE) but none in technology. The importance of discussions among teachers regarding learning and assessment has been highlighted (Pettersson, 2009;
Klasander, 2010).
In 2008 the Swedish teachers were provided with a navigational tool, the Individual Development Plan with written assessment (in short IDP). The intention of this public document is to support the future learning of the individual pupil, i.e. to be formative. It shall not include any harmful information. It falls, according to Hirsch (2011), into the long cycle of formative assessment. In short the intention of the document is similar to a GPS-device (Hartell, 2010). The design of the document is decided upon at every school and shall include written assessment in every school subject given and is not to be used to compare between schools or individuals. Similar documents are found internationally but with the difference that the Swedish IDP phenomenon is mandatory for all pupils and not just for those in special needs (Hirsh, 2011). The NAE report (2010) showed that the introduction and in-service training offered to teachers regarding the IDP-document were not sufficient.
Purposes of Assessment
Teachers work with assessment all the time; by asking questions, looking for glimpse of understanding and so on (Kimbell, 2007). However, the aims and purposes of the
assessments obviously differ (Newton, 2007; Pettersson, 2009). By being clear on what to be assessed and what the results shall be used for, as well as foreseeing the
consequences of the assessment, the validity increases (Nyström, 2004). If the purpose of the assessment does not include the pupil’s future progress, one must question the usefulness of it (Nyström, 2004). By feeding back to the analogy between sailing and teaching; I would like to point out that there also are considerable differences. When sailing you can change the pre-planned route and destination to something more suitable. When teaching you can change targets and align along the prevailing
conditions that exist but the Swedish National curricula stipulate two tuning positions that the pupil is entitled to exceed during their nine years of compulsory school. To
“navigate” pupils towards the goals of the curriculum and, making sure to keep every pupil ‘onboard’, is a challenge worthy of a professional around-the-world-sailor.
Assessment for Learning
Not all teachers who teach technology grade their pupils but they all work with assessment somehow. Teachers who fail to assess what the pupils cannot conclude if they are contributing or impeding their pupils’ process. Refusing to assess is really a concession to those who argue that no learning takes place (Lindström, 2006). Lots of information concerning pupils’ performance and skills is gathered, but seldom used (Wiliam, 2009). However well the intention of the gathering and the interpretations of the information are conducted; it is only formative when the information is used by the teacher, pupil or peers to modify and adapt the surrounding conditions for the learner to better suit the learner’s needs (Wiliam, 2009).
There is an unbreakable connection between successful teaching and formative assessment and learning, where assessment can be seen as the bridge connecting the teaching and the learning (Wiliam, 2011). International research is conclusive about the effectiveness of formative assessment but also the misuse and difficulties of the concept are identified (e.g. Hattie, 2009; Black & Wiliam, 1998; Bennett, 2011; Wiliam, 2011).
Wiliam (2009) stresses the necessity to divide formative assessment into three different cycles of time; long, medium and short. The effect of the formative function decreases with time elapsing between collection and use. The most effective formative assessment is that which occurs minute by minute every day in the classroom and is, according to Wiliam the one that should be seen as formative in its true sense. The long cycle (e.g.
IDP-documents) is not to be diminished as it may have an effect on a more comprehensive and strategic long-term basis.
Formative Assessment in Technology Education
In the next section a brief overview of previous research concerning formative
assessment with the focus on technology education is presented. The review has been made through Swedish and English literature and selected to fit the theme of this part of the study and not to be a complete review.
Discussions and the use of pupil’s questions in technology are cued by Black (2008) and Moreland et al (2008). They highlight the permissive classroom climate as well.
The possibility for the pupils to ask questions, the possibility of working with the feedback and the notion of learning from one’s experience including mistake are
stressed. Black emphasizes the wait time both before and after pupils reply as crucial for pupils’ learning. By increasing the amount of wait time from the average 0.9 seconds to 3 seconds the possibility for learning increases extensively according to Black.
Moreland (2008) says it is beneficial that the teachers themselves understand technology since it helps the learning of the pupil. Kimbell (2007) stresses the
importance of planning questions in advance. Blomdahl (2007) argues that a lot of time and effort are spent dealing with other matters e.g. finding equipment, time and group size instead of planning the tutoring in the technology subject. Their practice mainly focuses on managerial in the classroom and the teaching and learning come in second place. Blomdahl (2007) cues the lack of habit, among the Swedish teachers, to describe their teaching practice. Collegial discussion regarding teaching is rare among
technology teachers according to Bjurulf (2008) and Klasander (2010). Bjurulf and Klasander show that the teaching of technology is influenced by the teachers’ prior experience and they are not covering the whole of the syllabus e.g. technological systems. Discussion regarding grading between pupils and teacher occurred occasionally in the end of the theme (Bjurulf, 2008). Bjurulf says the teachers’
assessment does not reflect the teaching practice in technology e.g. the teachers seem to value theoretical skills higher than practical skills despite most of the teaching time is spent practically. Bjurulf (2008) cues the notion of a hidden agenda; where the enacted syllabuses recognize criteria connected with personality, i.e. working alone, not asking questions and being thorough, as criteria for success. This was known among the high achievers.
Results
This paper provides results of a qualitative study that explores the process of assessment for learning in technology education with the focus on teachers. This results presented are based from two teacher-focused sub-studies (1.) regarding the documenting of pupil’s attainment and (2.) teachers’ assessments practice within the classroom. The results build upon authentic samples of assessment documents in technology (IDP with written assessment in technology) and from classroom observations and teacher
interviews. The sub-studies are presented in short separately with some clarification specific to the sub-study and then discussed together.
Sub-study 1. IDP-documents in Technology Education
This sub-study have investigated the possibility to navigate the pupils in technology education using the IDP-documents. The results rely on samples of 351 authentic IDP- documents from school year 1-6, from five different municipalities. According to the regulations every school can decide on the design of the template (some schools within a municipality can use the same). This sample includes 14 differently designed IDP–
templates. Some IDP-templates included a box to tick for the goal achievement of the pupil in a particular subject where others included pre-formulated goals and/or space for the teacher to write additional comments.
Every individual pupil is entitled to a written assessment in every subject given. When looking for information about pupil knowledge in technology; the information is limited. Most (9 out of 14) of the IDP-templates totally lacked written assessment in technology. Thus the hypothesis presented at TERC 2010 (Hartell, 2010) was
confirmed. Technology was not the only missing mandatory subject in this sample but the least frequent. One municipality even lacked technology in all their six templates.
None of the IDPs, which included technology, contained individual information about the individual pupil regarding where to go next. Thus this study shows that IDPs are not useful in navigating pupils in technology education.
Sub-study 2 Short Cycle Assessment
Wiliam (2009) emphasizes the short cycle of formative assessment in the classroom.
Lindberg (2005) cues the lack of knowledge about teacher’s classroom assessment within the Swedish context. This sub-study investigates and describes the short cycle of
formative assessment in primary technology education. There are many things going on in a classroom. The five key strategies for formative assessment (Black and Wiliam, 1998) were used as spectacles in an observational scheme (which was inspired by Kimbell & Stables, 2008). By using observations, photography and sound-recording (which has been transcribed with high accuracy) and complementary teacher-
interviews, traces of the five key strategies for formative assessment have been captured and described in order to describe these teachers’ work with assessment. The purpose was to describe two teachers’ work with assessment together with their pupils during a technology theme in school year 4 and 5 (10- 11 year olds).
The results show traces of the five key strategies and these teachers’ un-doubtable intention of moving their pupils forward on their learning journey. They show, through e.g. glimpse, intonation, voice etc., their high expectations of their pupils’ ability to learn. They assess for them, not to them. Unfortunately previous results (e.g. Black et al, 2008) show that they are not likely to be as successful as their intention. They keep track of their pupils solely on regards of what happens during the classroom activities and keep the information within themselves. The results also show the lack of teacher training and teaching material. These teachers sorted it out by bringing equipment and stuff from their own home and from other subjects. Lot of time and energy are spent on this and they are not just teachers in the technology subject. They ask for organized collegial discussions regarding assessment in general and in technology in particular which neither of them is given the opportunity. This lack of equipment and collegial discussions are consistent with previous findings (Blomdahl, 2007; Bjurulf, 2008).
The conclusive results show that these teachers are left to their own prior experience in planning, assessing and teaching for their pupils in technology education. Neither the validity nor reliability of these teachers’ enactment in the classroom is investigated here.
Still it can be questioned through previous results (e.g. Kimbell, 2007; Pettersson, 2009
& Wiliam, 2011) which show that planning teaching and assessment are done profitably together with others.
Discussion
The results presented here show that the teachers are alone in making sure their pupils reach the stipulated tuning points stipulated by the national curricula. They are left to their own experience in finding the starting points, planning, executing, managing and assessing in technology. It might be considered trivial to use the GPS-analogy but it is worth reminding of when navigating at sea with a GPS-device you need information from at least three different satellites in order to determine the position where you are at. The accuracy in the position increases when more satellites are involved. When using signals from only one source the risk of getting lost increases considerably, even though your previous experience can help to bridge the gap from where you are to where you want to go. The importance of teachers working with assessment both consciously and carefully, e.g. looking for glimpse of understanding in the eye and the importance of awareness concerning the interpretation building upon one’s own
experience, is identified. There are many ways of finding out where the pupils currently are on their learning journey. IDP-documents are however not an option, in technology at least. It is not just the information that is needed. The evidence needs to be put into action. As a sailor I keep a record in my log to position us and as a teacher I need some similar notations to keep track on the pupils during the journey. The gathering of information does not by itself move the pupil forward; it is how it is used that matters.
By being clear on where we are going the likelihood of getting there increases for all participants. The results presented here show the evident lack of available equipment to
the teachers in making sure their pupils reach the stipulated targets. The importance of discussions regarding technology and assessment is identified previously and the results presented here emphasize the lack of it. Feedback is the description on what next step to take to alter the gap between the current and the wanted destination. The GPS-
performance of the teachers I would say is the ability and possibility to do so. And the results presented here show that some tuning is needed.
Conclusion
Sweden has a tradition of teacher’s assessment which I think shall be acknowledged and valued. However, the results show that the teachers are left alone and this awakes new questions concerning consequences. I stress the need for further investigations on regards how this affect the pupil’s results. What I find even more interesting, is to investigate how to support teachers in moving their pupils further on their learning journey, thus how to tune their “GPS-performance”. The GPS-navigator is part of a complex system and the results from this study show that the teachers are alone in managing all this. I argue that the risk of getting lost within the educational context increases when you are left alone. The need for discussions with others and the need for organized discussion regarding assessment and technology are both asked for here and previously identified as beneficial. Still, here, it is confirmed as an area of improvement.
Research shows that when teachers discuss with other professionals both the reliability and validity of the assessments increase. To my experience this is not always so easy to organize. That is why I find it intriguing to investigate further how different digital tools can be used to shrink time and space (c.f. e.g. Hartell, 2012; Kimbell & Williams, 2012) and what possibilities it opens up for.
References
Bennett, R. E. (2011), Formative Assessment: a critical review. Assessment in Education:
Principles, Policy and Practice, 18:1, 5-25 Online publication
http://www.tandfonline.com/doi/abs/10.1080/0969594X.2010.513678 (retrieved 11-11-27) Black, P.J. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education:
Principles, Policy and Practice, 5(1), 7-73.
Black, P. (2008) Formative assessment in the learning and teaching of design and technology education: Methods and techniques. Design and Technology Education: An International Journal, 13(3), 19-26.
Black, P. J., Harrisson, C., Lee, C., Marshall, B. & Wiliam, D. (2008/ 2003) Assessment for learning. Putting it into practice. Open University Press.
Blomdahl, E. (2007), Teknik i skolan- en studie av teknikundervisning för yngre skolbarn, HLS förlag. Stockholm. Doctoral thesis.
Bjurulf, V. (2008) Teknikämnets gestaltningar. En studie av lärares arbete med skolämnet teknik. Karlstad University studies, nr 2008:29 Doctoral thesis.
Hartell, E. (2010) GPS-performance in technology education. In Middleton. H. (eds) Knowledge in Technology Education. Vol I (II). (pp:171-177). Brisbane, Griffith Institute for Education Research
Hartell, E. (2011), Hur sätter man betyg i teknik? In Hansson, Nordlander, Skogh (eds) Teknikutbildning för framtiden- perspektiv på teknikundervisningen i grundskola och gymnasium. (pp: 75-87). Stockholm: Liber.
Hartell, E. (2012), En kort redogörelse om utprövning av programmet e-scape för skapande av bedömningsunderlag i ämnet teknik. Med tillägg av resultat från parvisa jämförelser.(Report to The Swedish National Agency for Education, Skolverket).
Hattie, J. (2009), Visible learning. A synthesis of over 800 meta-analyses relating to achievement. Chippenham: Routeledge.
Hirsh, Å. (2011), A tool for learning? An analysis of targets and strategies in Swedish Individual Educational Plans. Nordic Studies in Education 2011 (1):14-30.
Universitetsförlaget.
Jönsson, A. (2008) Lärande bedömning. Malmö: Gleerups.
Kimbell, R. & Stables, K. (2008), Researching Design Learning, UK:Springer
Kimbell, R. (2007), Assessment, In de Vries, Custer, Dakers, Martin (eds), (2007), Analyzing Best Practices in Technology Education, (pp:247-258), Rotterdam: Sense Publishers
Kimbell, R.& Williams, J. (2012) Conclusions, International Journal of Technology and Design Education 22:2012 (pp: 265-270).
Klapp-Lekholm, A. (2009) Vad säger betygen? In Lundahl & Folke-Fichtelius (eds) Bedömning i och av skolan - praktik, principer, politik.(pp: 129-140). Lund: Studentlitteratur.
Klasander, C. (2010) Talet till tekniska system. PhD thesis, Linköping University, Department of social and welfare studies, Norrköping, Sweden. Doctoral thesis.
Leahy et al (2005) Leahy, S., Lyon, C., Thompson, M. and Wiliam, D. Classroom assessment:
Minute by Minute, Day-by-Day. Educational leadership, Vol 63, no 3, 19-24.
Lindberg, V. (2005) Svensk forskning om bedömning och betyg. Studies in Educational Policy.
Lindström, L. (2006). Creativity: What is it? Can you assess it? Can it be taught? The International Journal of Art & Design Education, 25 (1), 53-66.
Moreland et al (2008), Moreland J., Jones A. and Barlex D. Design and technology inside the black box. Assessment for learning in the design and technology classroom. London. GL assessment.
NAE/ Skolverket (2010) Individuella utvecklingsplaner med skriftliga omdömen i grundskolan.
En uppföljning och utvärdering av skolornas arbete. Rapport 340.
Newton, P. E. (2007). Clarifying the purposes of educational assessment. Assessment in education; Principles, Policy & Practice Vol 14(2), 149-170.
Nyström, P. (2004) Rätt matt på prov. Om valideringar av bedömningar i skolan. Umeå universitet, Pedagogiska institutionen, No 71.
Pettersson, A. (2009/2005), Bedömning- varför, vad och varthän? In Lindström & Lindberg (eds) Pedagogisk bedömning. Stockholm: Stockholms universitetsförlag.
Rosentahl & Jacobson (1992) Pygmalion in the classroom. Teacher expectation and pupils’
intellectual development. P x-y Crown House Publishing Company. USA.
Wiliam, D. (2009), Assessment for learning: why, what and how? An inaugural professerorial lecture by Dylan Wiliam. Leading education and social research. Institute of Education University of London. Elanders.
Wiliam, D. (2011), Embedded formative assessment. Solution Tree Press.
