Assigning marks to judged work

A recent blog by Tomas Needham described how he used nomoremarking.com to judge GCSE English Controlled Assessment in his school. Once he had finished judging he needed to assign marks to scripts so that they could be moderated by the exam board. How could he do that?

Tom’s solution was to include 8 pieces of work that had previously been moderated by AQA. For example, for the essays on Of Mice and Mean, each piece of work had a moderated mark, with the marks ranging from 10 to 26. The highest mark possible is 30. He removed all marks and annotations from these scripts and included them in the judging.

Once the judging was completed, the first thing I checked for Tom was that the marks and judging agreed. Marking, as Tom describes it, is ‘fiendishly difficult.’ If the marking is too inaccurate then the marks would be of little use. Luckily there proved to be a strong relationship between the marks and the CJ scores. The correlation was 0.89.



Once a relationship had been established between the moderated marks and the CJ marks it was easy to apply this relationship to the unmoderated marks using linear regression and predict a moderated mark for every script. The capping on the mark scheme at 30 meant that scripts we predicted would have much higher marks from the judging could only receive 30 marks.



So, statistically speaking, how successful was the judging? Tom had also asked his teachers to mark the work in isolation. Plotting the original teacher marks against the predicted moderated marks shows that there was a wide scatter of marks from the original teachers. Scripts that, as a group, the teachers had judged to have a mark of 19, had original marks which ranged from 16 to 30!



So, all in all, a really neat experiment by Tom, who has shown clearly here that teacher assessment, given the right support, can work! If we want to keep assessing important practical skills using teacher assessment we need to stop marking and start judging.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s