I have been getting a lot of queries from students about normalization and relative level of difficulty of CAT across slots. CL Academic Team has just released a Research Report on their findings of the CAT 2013 attempts data. I will advise you to read the same. But here are my specific answers:
Q. Is there a normalization post the CAT? – The short answer is NO.
Q. Is the CAT difficulty level random across the slots? – The short answer is NO.
I don’t know whether or not you will agree with these answers, however, logic and data analysis clearly reveals that my answers are in fact highly likely to be true. Here are the long answers:
CAT is ‘pre-normalized’
PROMETRIC: “LOFT carefully balances each candidate’s exam to ensure proper content coverage and overall difficulty, enabling reliable comparisons of performance from one candidate to another.”
Most of us have graduated from one CAT paper till 2008 to around forty CAT papers (one in each time slot) from 2009. However the reality is that students in the same slot also have different papers and Prometric administered around 1,75,000 different tests (Prometric uses the term Test Forms) using LOFT algorithm which more or less ensured that the difficulty level of the test is almost the same for any two test takers, leaving very little room for any normalization post the test. In fact as per LOFT each candidate gets randomly presented with a pre-set equal number of questions from each difficulty level (Easy, Medium-Difficulty and Difficult) from their item bank.
Also, since there were 1,75,000 different tests, it will be almost impossible to do any find of ‘slot’ wise normalization post the CAT. In fact there is probably nothing called a slot, there are only Test Forms.
Data from CAT 2012 suggests that some kind of post exam normalisation is there but it is minimal. In CAT 2012 a large number of CL faculty members in different slots correctly answered only the 9 LR questions in VALR (and nothing else) and their score in this section was in the 79.11 to 79.99%ile range, a difference of only 1 mark in the scaled score.
Breaking the Myth –
Myth: CAT is tougher in certain slots
Reality: CAT difficulty level is more or less the same for every test taker
I can claim this with complete confidence because of the analysis that CL has done.
More than 8,000 students admit card data and their attempts in the CL True Percentile Predictor have been analyzed by CL. This is a statistically significant quantity of data and the results of our analysis confirms that CAT is a standardized test.
The main reason for the ‘noise’ – that CAT is easier on earlier dates – is because of the fact that students who scored well in the Mocks are taking the CAT early. Our data suggests that more ‘Mock toppers’ (better prepared students) are taking the test early and therefore, the ‘noise’ that CAT seems to be easier on earlier test dates. In reality, the CAT difficulty level is almost the same! It is profile of students which is impacting the attempts and therefore the ‘noise’.
The average attempt in CAT was around 37-38 across slots. The movement of the average attempts has the same trend as the percentage of ‘Mock toppers’ among the test takers. I will seriously doubt if even Prometric can create this kind of insight that CL has generated.
Bottom-line, CAT is not a random test and therefore yes, CL can almost accurately ‘predict’ the outcome of the CAT based on statistical analysis. One word of caution – the accuracy of the prediction depends on the accuracy of your estimated correct responses.
Be that as it may, let’s look forward. Whatever happened in the CAT, happened – the key is to take informed next steps. The True Percentile Predictor will go a long way in helping you on the same. If you have not yet used the CAT Percentile Predictor, please use it. The real benefit in using the tool is that it gives you insights into things beyond the CAT – the B-schools that match your profile.