Putting Comparative Judgement to the Test

Starting in January 2014 we put Comparative Judgement to the test at Ashbourne Community School in Co. Meath.

A general English test was administered to 137 6th year Post Primary students. The test asked the students to outline a character they sympathised with using any text they had studied as a reference. The scripts were anonymised, scanned and uploaded to the nomoremarking website for judging. At the time of writing this report we are awaiting the return of the English scripts from 4 other schools at different locations around Ireland.

Once all scripts, around 450, are uploaded to the website the English teachers from all the different schools will be asked to judge the scripts.

The English teachers’ judgement decisions will be statistically modelled using a computerised algorithm to produce a scaled score. Each school will therefore receive a scaled score that is comparable with the scores of the students at other schools. The scaled rank order for the students at Ashbourne Community School will be analysed at a meeting of their English teachers and grade boundaries will be identified.

The correlation between the Comparative Judgement outcome and pupils’ previous test results will be investigated to test how reliably the Comparative Judgement process has performed.

Feedback from the pupils and teachers through reflective evaluation forms will be gathered.

Pupils will also be offered a huge learning opportunity by engaging in the judging of script themselves through peer assessment.

Reliable and fair?

This Comparative Judgement assessment will involve anonymous assessment of pupils from different schools and the professional judgements will be carried out online by teachers across the country. External (i.e. cross-school) moderation will minimise fears that pupils could claim they were unfairly marked or that grades are unreliable. Moreover this approach to external moderation is not costly and does not lead to an increased workload, thereby allaying any concerns about increased costs and teacher workload of implementing a school-based assessment system for certification.

By Suzanne McMahon, Ashbourne Community School

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s