Manhattan GMAT CATs accuracy

[anil4410]

Hi,

I was wondering how accurate the results on Manhattan CATs actually are.
I've seen a great deal of variation in my scores on these Manhattan CATs.
On the last 4 tests, I've ended up with the following scores:
680, 750, 700 and 770.

As you can see, there is a great deal of variation in the scores.
Also, I noticed that the questions in my last CAT were generally easier. Mostly in the 600-700 level.

Are there great chances of my score being inflated because of the level of questions being easier?. or is this issue catered to by the scoring algorithm?

Thanks,
Anil

[RonPurewal]

Hi,

I was wondering how accurate the results on Manhattan CATs actually are.
I've seen a great deal of variation in my scores on these Manhattan CATs.
On the last 4 tests, I've ended up with the following scores:
680, 750, 700 and 770.

As you can see, there is a great deal of variation in the scores.
Also, I noticed that the questions in my last CAT were generally easier. Mostly in the 600-700 level.

Are there great chances of my score being inflated because of the level of questions being easier?. or is this issue catered to by the scoring algorithm?

Thanks,
Anil

the reason for the extra load of 6-700 problems at the end is that you ran out of 7-800 problems!

we are currently beefing up the pool's content of 7-800 problems, but, as it currently stands, extremely high-scoring students are not going to have enough of the 7-800 problems to last for six full exams.

680/750/700 is a fair amount of variation, but it's not ridiculous.
the OFFICIAL test has a "standard error" of 30 points for individual scores. this means that, if your "true average" score is N, then there's about a 70% chance that your score will be between N - 30 and N + 30.
turning this statement around, we can see that there's actually a 30% chance of scoring < N - 30, or > N + 30!!

so, assuming (for the sake of argument) that your true score is, say, 720, then you could fully expect 30% of your scores to be <690 or >750. given this fact, that's not an unreasonable spread of scores.
the 770 is probably a bit inflated, since you ran out of the harder problems.

good luck.

[jtspam]

My question somewhat relates to this.

If the question bank ran out of higher level questions and gave 'easier' questions, wouldn't that lower your score, as opposed to inflate it, since the level of questions you are getting correct is on average lower?

The reason I ask is that I just took my 5th CAT, and was very disappointed at my verbal score (37) given how good I felt about it upon completion. In reviewing my assessment report, I got a total of 28 correct, and only 13 incorrect. That was comparable to my 3rd CAT where I got 27 correct and scored a 41, and much better than my 2nd CAT where I got 25 corrected and scored a 40.

I understand it's not about how many you get correct, rather it's the level of questions that you get correct. However, I didn't feel I got enough 'opportunities' to nail 700-800 questions to raise my score, given the fact I was doing pretty well. I never got more than two consecutive problems wrong, and at one point got 11 in a row correct. Nevertheless, 18 of the last 20 questions were 600-700 level questions!

By no means am I complaining about the CATs; I think it's a wonderful practice tool. I'm just trying to get a better idea of how the scoring works, and whether my score may be a bit inflated or deflated. I have an appointment to take the exam in a week and a half, and I'm a bit discouraged by my latest verbal score after I was able to break 40 a couple of times in previous CATs. My goal is a 42, but my scores seem to be heading in the other direction, even though I'm getting more problems correct. That leaves me baffled at what I should be working on! Obviously I'd like to believe that my score was just deflated, but after reading in this thread that it may actually be inflated, that worries me a bit more

[RonPurewal]

My question somewhat relates to this.

If the question bank ran out of higher level questions and gave 'easier' questions, wouldn't that lower your score, as opposed to inflate it, since the level of questions you are getting correct is on average lower?

The reason I ask is that I just took my 5th CAT, and was very disappointed at my verbal score (37) given how good I felt about it upon completion. In reviewing my assessment report, I got a total of 28 correct, and only 13 incorrect. That was comparable to my 3rd CAT where I got 27 correct and scored a 41, and much better than my 2nd CAT where I got 25 corrected and scored a 40.

I understand it's not about how many you get correct, rather it's the level of questions that you get correct. However, I didn't feel I got enough 'opportunities' to nail 700-800 questions to raise my score, given the fact I was doing pretty well. I never got more than two consecutive problems wrong, and at one point got 11 in a row correct. Nevertheless, 18 of the last 20 questions were 600-700 level questions!

By no means am I complaining about the CATs; I think it's a wonderful practice tool. I'm just trying to get a better idea of how the scoring works, and whether my score may be a bit inflated or deflated. I have an appointment to take the exam in a week and a half, and I'm a bit discouraged by my latest verbal score after I was able to break 40 a couple of times in previous CATs. My goal is a 42, but my scores seem to be heading in the other direction, even though I'm getting more problems correct. That leaves me baffled at what I should be working on! Obviously I'd like to believe that my score was just deflated, but after reading in this thread that it may actually be inflated, that worries me a bit more

well, the algorithm is going to do its best to extrapolate your performance.
what this means, in practice, is that there are probably more opportunities for variation on both sides of the fence:
(1) since you won't be presented with 700-800 problems, you will have a much greater chance of "cleaning up" on the questions that are presented to you. if this happens -- i.e., you get an insane number of questions correct -- then your score will probably be a bit inflated, as may be the case with the poster above.
(2) on the other hand, since the problems are on average easier than the problems you encountered on previous exams, each missed problem will carry a relatively more severe penalty. this effect makes it more possible to have artificially low scores as well.

also, don't forget that there is a great deal of statistical "noise" in all gmat cat exams, including the official test. therefore, you shouldn't really be concerned with small upward and downward fluctuations in your exam scores.
and one more thing: how far apart have you taken these exams? your true level of preparation, and hence your "true score" (the thing you're presumably trying to approximate with these practice tests), is very, very unlikely to change much in a week or two. therefore, unless you're taking these exams only about once a month, don't ascribe too much meaning to the variation from one exam for the next -- instead, try to look at the whole big picture.