Your grade IS decided by whoever does the assessment

Skrevet af Jannick Friis ChristensenMia Mathilde HansenTranslated by Felix Kasperek - Foto: Jørn Albertus - 17. februar 2014 - 17:520 kommentarer
Unless the MSc BA IB students born in the beginning of the month were consistently more knowledgeable than those born towards the end of it, there was a significant difference in the assessments – and that’s not allowed.

 In early December, approx. 150 students of the MSc BA IB program were divided at random between two assessors after a written sit-in exam. The consequence was a significant difference in grades awarded. Now, all the hand-ins are to be re-assessed.

Hand-ins divided at random
In exam periods, the topic of conversation among the students often pertains to whether or not the grade they got could have been different, if somebody else was doing the assessment.

The students of the MSc BA International Business (IB) program got to feel that first-hand after their recently finished International Financial Markets written sit-in exam. And surprisingly(?) it turns out that it makes a difference.

Due to problems with the implementation of STADS, the program administration was forced to send out a grade transcript containing the grades of all the examinees, rather than the usual individual transcripts. The transcript showed that the examinees had been divided between two internal assessors.

At first glance, it seemed that one assessor had been more generous with the grades than the other. At second and third glace, that still seemed to be the case, and it didn’t change once the average grades were calculated.

Significant difference in average grade
Some of the students noticed the significant difference in the grades awarded by the two different assessors. It spurred on MSc BA IB student Kasper Sonne Nørbygaard to create an Excel sheet to test his suspicion.

The Excel sheet confirmed his suspicion; the students born in the first half of the month had been awarded a grade that was almost 2,31 grade points higher than those been after the 16th in the month.

The likelihood that the academic competencies are actually divided like that is less than very low. The big difference made the students react, and the inboxes of the program administration and the course administration were flooded with emails.

Instructor: grades could also be too high
Ole Risager, course coordinator and instructor of the class this year chose to pass on the responsibility of grading the hand-ins based on his answer key.

- And then it turns out that there’s a significant difference in the grades. But the truth is that nobody knows why the difference is there. We can’t be sure what the reason is, because there are many potential explanations, Ole Risager explained and said he was sorry about the whole ordeal.

- I’ve received lots of emails from students, and the prevalent tone is that the grades awarded have been too low. But the grades could be are too high, too, Ole Risager says.

The students were free to complain
Although the program administration has received a steady stream of complaints from students who felt unfairly treated, the course coordinator chose not to handle the situation on an administrative level.

The attitude of the course coordinator was that the students had been assessed according to the rules and on equal terms, and that it was up to the individual students to try their luck with complaining officially through Legal Services, if they thought differently.

The issue is that according to the rules, the assessors are free to chose a level and interpret the answer key. That’s what the two assessors have done, and they’ve each assessed the students from their respective levels.

But… When held up against each other, there was a significant difference in grades.

Expert: grades are NOT person-independent
CBS OBSERVER got involved in the case in the middle of January, and as part of the research got CBS’ evaluation and accreditation unit to do an independent analysis of the grades given.

The analysis showed beyond any doubts that the students were right in thinking that the assessments weren’t independent from whoever assessed them.

The GPA for assessor #1 is 6.74 while it’s 4.43 for assessor #2, when the grades given to blank hand-ins (-3) are ignored.

Evaluation consultant Michael Møller Nielsen, who did a statistical analysis of the grades, explains:

- Only approx. half of the observed difference in grades between the 2 assessors can be explained with reasonable probability as being one assessor, in this case assessor #2, coincidentally ‘drawing’ worse hand-ins than assessor #1.

Michael Møller Nielsen therefore concludes that the grades given, with 99.9 % probability ISN’T independent of whoever assessed them – which support the arguments of the students.

Outraged students compared grades
The course coordinator’s handling of the case outraged a large part of the MSc BA IB students. They claimed that the problem was that the two assessors clearly had defined different levels.

When an anonymous source, whose name is known by the staff, contacted CBS OBSERVER with the case, it was only to bring attention to the fact that there shouldn’t be an opportunity to discriminate against students in an exam situation.

- I definitely think that this case is about the principle that your grades should be a reflection of your competencies. Everyone who isn’t satisfied should have the opportunity for reassessment, the source thought.

He’d gotten the grade 7 and didn’t think a reassessment would change it. However, he considered complaining out of principle, after the instructor published the answer key following massive pressure from the students.

Practice as usual – the students must go to Legal
Student Jan Vibe Gregersen, who’s one of the three student representatives of the MSc BA IB program’s Quality Board acted as spokesman for the course coordinator.

He tried his best to find a reasonable solution that would serve both parties, but he wasn’t satisfied with the outcome Ole Risager decided on. According to him, the outcome placed the burden of the problem completely on the students’ shoulders.

Ole Risager had chosen – in a declared hope to be able to shut the case with “The friendliest solution for the students” – that the students should go the official way and complain via Legal – in other words, ‘practice as usual’.

In Ole Risager’s eyes, that was the most student friendly solution. He thought that the alternatives – either a reassessment of all hand-ins or an actual re-take of the exam – both entailed risks that the students could end up with worse grades than they got at the original exam.

Wanted the Study Board involved
- I wanted the Study Board to get involved regarding the principle, and we asked them to. The two assessors should agree on what the standard of grading should have been, says Jan Vibe Gregersen and emphasizes:

- It’s not as much the fact that one is stricter than the other; it’s more about the obvious difference in grading.

However, this isn’t a case for the Study Board – except for the principle in the case and how to avoid a similar situation in the future. Henrik Sornn-Friese could have involved himself in the case on his own initiative, but that would have been in his role as Academic Director of MSc program, not as Chairman of the Study Board.

Dean’s Office of Education and Legal made executive decision
As the case was until Thursday the 13th of February, the students couldn’t do anything. The re-examination was Friday, February 7th, and the 14-day deadline for normal complaints through Legal was passed long ago.

Luckily, the Dean’s Office of Education and Legal got involved. In part due to the ripples made by CBS OBSERVER’s research and in part due to the 35 complaints Legal had received, of which the essence was that the assessments were different.

It’s standard procedure for Legal to get involved when a disproportionately large amount of complaints for an exam is received. The Dean’s Office had the statistical analysis repeated by CBS’ evaluation and accreditation department, and came to the same result:

There were no mistakes in the exam itself, so no annulment or extraordinary re-examination was necessary, but the assessment of the two groups had been dissimilar – too dissimilar.

Exam re-assessed with external censors
The Dean Office for Education notes that: “The assessment made doesn’t live up to the Order’s (The Examination Order, § 60, stk. 1, nr. 3) demands for similar treatment and reliable assessment”.

As a consequence, the Dean’s Office decided that “the current assessments are revoked and a new assessment of all hand-ins is commenced” – “As fast as possible”…

The students thereby risk not only getting higher grades, but also lower grades.

Even though the original exam didn’t have external censors, the re-assessment will have – in part to ensure as high reliability as possible and in part due to the risk of the students receiving lower grades.

Additionally, a number of conditions will come into play for students that have attended the re-examination in the meantime, or receive a failing grade in the reassessment.

We here at CBS OBSERVER are currently trying to get our hands on the internal memo, so it can be posted in its entirety in this article.

CBS errors shouldn’t affect students
CBS Students president Charlotte Gjedde considers it a basic right that the students of CBS should receive grades that are fully comparable with those of their fellow students, since they’re the ones they’ll be competing with after their studies end.

She’s outraged by the instructor’s handling of the case, but happy that a reasonable solution has been found.

- I think it’s good that CBS accepts responsibility, when they’re the ones responsible for the mistake, says Charlotte Gjedde.

She finds the solution of sending all hand-ins to reassessment more fair than letting each individual student seek out Legal themselves.

- I can easily see that the solution is a bit of a downer for students who are already satisfied with their grade. It just doesn’t change the fact that a collective reassessment in this case is more fair than individual complaints, Charlotte Gjedde recognizes.