Audio Recordings of all Data Excerpts
Excerpt 7.1
Excerpt 7.2
Excerpt 7.3
Excerpt 7.4
Excerpt 7.5
Excerpt 7.6
CASS test recording [32111231] “Henrik and Fred”
About the test
The test recording (File name CASS recording 32111231_Henrik_Fred) is a high-stakes NEST recorded in 2014 and used as data for assessment in the rater moderation studies presented in Chapter 7 (Criteria application: The case of interactional competence) and Chapter 8 (Case study 2b). The test, in the form of an anonymized audio-recording of a paired NEST from the 2014 Testing Talk corpus, featured 15- to 16-year-olds Fred and Henrik and their co-present teacher Kajsa.
About the rater discussion setup
As moderation should work from samples of authentic learner performances, one test recording was selected for the teacher groups to work with. In the workshop, the teachers listened to the test recording in full while taking notes and making initial assessments individually. Raters were then divided into four groups and assigned rooms for their moderation meetings. The NEST recording was also accessible via a web link so that participants could choose to re-listen to parts of the recording together on a smartphone. In the group meetings, raters were instructed to discuss their individual impressions of the two learners in relation to the NEST analytic assessment factors, and finally, agree on a holistic grade for each learner on the A–F scale. Conversation analytic transcription and procedures of analysis were employed in describing raters’ discussions of test-taker conduct.
Using the test recording
The full test recording can be used in several ways:
· Use the scoring rubrics provided in the Appendix of the book (“Assessment factors for NEST 9, translated from the Swedish National Agency for Education, 2013”) and follow the procedure for rater moderation described above, with individual assessments and subsequent group moderation. Do teachers interpret the criteria in similar or different ways? Where are points of disagreement?
· Use the recording for broader purposes of rater training on the assessment of L2 oral proficiency and interactional competence. Suggested questions for discussion are provided below.
· Use the recordings with one or more of the other scoring rubrics provided in relation to Chapter 5. Was the assessment different?
Study questions
· Proficiency-wise, what evidence do you find for a grade between A and F for these two test-takers? Make note of specific features of language, expression, and interaction, and discuss their relative weight in setting a grade.
· In relation to the discussion topics test-takers read from the topic cards, how can their topic management be assessed?
· What features of test-takers’ L2 interactional competence are particularly salient in relation to the discussion tasks? What are the affordances of the test format and the discussion topics in terms of what Fred and Henrik can display?
· Identify three aspects of each test-takers’ performance that you initially liked and disliked and explain why to your colleagues. Try to identify any bias and individual preferences in your own selections in relation to the assessment factors. We recommend individual reflection, followed by group discussions. Discuss your own self-reflections with your colleagues.
· The teacher in the test is an active co-participant in the interaction. Listen closely to the teachers’ contributions and discuss places where they prompt or hinder test interaction (questions, acknowledgement tokens, laughter, turn allocation etc.)
· The discussion topics touch upon everyday moral issues. What are the pros and cons of test topics which require test-takers to reveal personal viewpoints? Can you find any evidence as to the role of the topics in this recording in terms of how Fred and Henrik approach them? Can you find any signs of test-taker resistance, and how can you tell?
