Should I apply for IELTS Remark in writing?

Introduction

According to the recent IELTS statistics, writing is the most challenging part of the IELTS Academic exam. Exam takers typically perform worse here than in other parts. That's why it's a popular question if it makes sense to request a re-evaluation of the writing score to increase the lowest and overall score.

Generally speaking, asking for a remark has a decent chance of success. Especially if there is a big discrepancy between diffent sections of IELTS, say listening and reading is 8 but writing is 6. Often you could improve your score by 0.5 or 1 band. Here is a success story of such a positive remarking result

Understanding the IELTS Evaluation Process

To understand why it makes sense to ask for re-evaluation let's understand how exactly evaluation works and what is the acceptable deviations in scores. IELTS evaluation is made by human examiners that follow an intense 2-day training on the topic. They are also required to pass IELTS writing and speaking for band 9 and match several other criteria, like having a teaching experience. There are procedures in place to improve scoring quality, e.g. IELTS writing consists of two parts, which are graded by separate examiners, if there is a big difference between grades for these two parts third examiner double checks scores. Evaluation rubric and band descriptions are standardized and examiners are monitored from time to time, but eventually, it all leads to a human factor, how each teacher understands each peculiar rule.

An deep dive into IELTS Evaluation Correlation criteria

How can we learn more about the internals of the IELTS examinations? IELTS has a section on the website with scientific articles for research. There is a very nice article named "An investigation of examiner rating of coherence and cohesion in the IELTS Academic Writing Task 2". Authors focus particularly on one aspect of IELTS writing, namely Coherence and Cohesion, but they did extensive testing for all components of the writing task 2. They had 55 IELTS examiners who graded 12 sample essays each. The sample essays also had ideal reference scores provided by IELTS organisation. Let's review the excerpt from the article that summarizes the findings.

Let's translate this text to a human language. What could we see from that? They use a term named correlation. Correlation shows how close the scores given by examiners are to ideal reference scores. The higher this value the better. Typically this number ranges from 0 (no correlation) to 1 (ideal correlation). From the excerpt above we see that IELTS targets correlation of 0.8. It is considered as a good correlation by them.

As shown in the table the majority of examiners had a correlation close to 0.8, one examiner had as low correlation as 0.5, and the couple of examiners had 0.75 and 0.79. Two examiners had correlations around 0.94.

But what does a correlation of 0.8 mean? Does it mean that it's ok to have a difference of 1 band from reference scores? Or 0.5 or 3 bands? To find it out I had to brush up my statistics skills from the time when I studied theoretical physics in university.

The idea is to simulate a situation when correlation is 0.8 and see what the sample distribution of scores would look like. We can create a random set of scores and calculate correlation with reference scores. We would repeat this process until the moment when we create a set of scores with a correlation coefficient close enough to 0.8. This type of simulation has the fancy name "Method of Monte Carlo". You can see the code in Python for the simulation here. This approach allows us to see a sample (one of the variants) how examiner scores could look like in relation to reference (correct) scores.

This is what sample results look like for a correlation of 0.8. We can see that occasionally scores are different from the reference by 0.5-1-1.5 band, rarely 2.5

For 0.9 differences are way less, mostly up to 0.5 band, rarely 1.

Let's look at the case of 0.5 Here we could see a staggering difference of 5!

Summary

As we can see, with a typical IELTS scores correlation (0.8), the IELTS writing band could differ by 0.5 or 1 band between different examiners. And this 0.5 might be exactly what you need to secure your future in a European university or to pass a Canadian immigration test. Of course, it's important to remember that this difference could be on both sides, to lower or to higher bands. As well, remarking requests cost an extra fee. However, often it would be wise to request re-evaluation keeping in mind the difference in scoring between examiners and the high stakes involved.