Friday 26 April 2013

Hoaxes and deceits with exams eg GCSE English

People who design exam systems have special ruses to confuse and bewilder us.

1. Reliability vs validity

If you narrow down what you examine,  you can make an exam more reliable. That's to say it will come up with consistent marking, the curve on the graph of those sitting it will match the standard curve year in year out. However, what you are testing will not necessarily reveal eg the best of what the 'testee' can do, and will not necessarily encourage a wide and deep way of working leading up to the exam. At the end of the day, many people in government either don't know about this or, more likely don't care.  As I've mentioned before, the Spelling, Punctuation and Grammar test has been brought in on a lie about reliability ie the claim that there are right and wrong answers to questions in these three areas. The claim was made in the government's 'Bew' report and acted on but I have not seen anywhere that being taken on as an entirely false claim. It's the rush to reliability that has produced something invalid.

2. If certain methods of testing are producing the 'wrong' results, then that part of the exam producing those false figures, must be abolished.

This is the lie at the heart of the re-design of GCSE English. So, the claim is made that teachers cheating on the speaking and listening section are bumping their students up to A-C. The consequence say the critics is that there is a false picture of who is competent at English, so they are kicking speaking and listening out of the final grade. This is a hoax. Examiners have known for at least a hundred years that if you want to make a grade for which minimum criteria have to be reached in all its sections, then you simply control the marking that way. So, in this case, it would be simple to say that a student getting an A-C needs to reach 3 different minimum levels at the written section to be able to get A, B or C in the final mark. There is absolutely no need to abolish that section of the exam that is deemed to be lifting the final grade into what is thought to be the minimum requirement of being a competent - b good - c outstanding.

Alternatively, examiners dish out  a double mark, one for the written section, one for the oral - as we used to do with foreign languages back in the 50s and you had to pass both to get your 'certificate'. You don't need to abolish one section.

In other words, Ofqual have brought the abolition of talking and listening in off the back of a hoax.