ManhattanGMAT Test Scores Vs GMAT Test Score

[direstraits007]

Hello MGMAT Instructors,

Well I've given the GMAT Prep and my scores are Prep1:740 (Q50/V40) and Prep2: 720(Q50/V39).

Now I want to assess myself because my GMAT date is 20 march and I need to know where I'm standing as far as the GMAT exam is concerned. I heard that GMAT Prep is the real indicator of the main GMAT score, but I could see a few questions from OG12/OG11 in the GMAT Prep. So I don't know whether the scores I got in Prep are the real indicators.

So, I gave the MGMAT test series and my scores are as follows:

Test 1 (Free): 720 (Q50/V38)
Test 2: 680 (Q49/V34)
Test 3: 740 (Q49/V42)
Test 4: 680 (Q48/V35)
Test 5: 770 (Q51/V44)
Test 6: 760 (Q51/V42)

So, How close are the MGMAT Scores, when we compare them to the real GMAT scores? I know our instructors would be having fair enough data handy, which can rightaway relate the MGMAT scores to the real GMAT score with a potential delta. So, please let me know where I stand and if I consider all other factors same then how much would be my range of score in the GMAT ?

Also, I noticed two surprising things in the MGMAT:
1: Even if I do 25 questions correct in my verbal, my verbal score ranges >=38. But in GMAT Prep I didn't see this phenomenon. I did one GMAT prep and intentionally marked wrong answers and got 25 correct in verbal out of 41. My Verbal score came out to be 30. Perhaps my analysis is wrong, but that's what I want to know from the instructors.
Please explain.

2: The last two tests of MGMAT skewed my score to 750+. Is it because MGMAT wants to boost the morale of the students during its last two tests? I'm asking this because I answered the same number of questions correctly in the last two tests as in the fourth test in which I got 680. But the score delta was too much. Please explain.

waiting for the early response.
Thanks!

~GeeMate.

[RonPurewal]

hi --

Test 1 (Free): 720 (Q50/V38)
Test 2: 680 (Q49/V34)
Test 3: 740 (Q49/V42)
Test 4: 680 (Q48/V35)
Test 5: 770 (Q51/V44)
Test 6: 760 (Q51/V42)

those are some serious scores. nice job.

So, How close are the MGMAT Scores, when we compare them to the real GMAT scores? I know our instructors would be having fair enough data handy, which can rightaway relate the MGMAT scores to the real GMAT score with a potential delta. So, please let me know where I stand and if I consider all other factors same then how much would be my range of score in the GMAT ?

nah, we actually don't track statistics like that -- there are just too many variables that could invalidate any data we might collect.
the four most important variables that would render those data suspicious, if we were to go to the effort of collecting them, are:
(1) response bias: the only official scores we're going to hear are those from students who actively choose to come back and report them to us. as you might imagine, students with middle-of-the-road results aren't normally going to be motivated to do so. as with pretty much any other self-reported survey, we would receive disproportionate numbers of extreme results -- both from students who were elated with a performance beyond their wildest dreams, and also from students who were disappointed in their performance. as a result, we would probably wind up with some sort of bimodal distribution that would in no way reflect the true nature of our students' performances.
(2) confounding variables in test administration: we would basically have to trust that our students had taken their practice tests in perfect exam-like conditions. of course, this isn't always going to be the case: many students will cut corners with such things as taking excessively long breaks, pressing the pause button, and so on.
(3) study plan: many of our students exhaust our practice tests early, continue to study for another month or two, and then evaluate the rest of their preparatory trajectory with other practice tests (such as gmat prep). it would be disingenuous at best, and downright dishonest at worst, to compare such students' results to the results obtained by students who took the official exam right after finishing our battery of tests.
(4) random variation: remember that the official exam, even despite its adaptive nature, still has a relatively sizable standard error -- approximately 30 points. this means that there is going to be considerable "noise" in your exam results. therefore, if the delta between our practice tests and the real test is small enough, it will be completely obliterated by this random noise.

the short answer is, basically, no, we don't collect aggregate data -- and we have plenty of good reasons not to.

despite the lack of exhaustive data, however, from many students' feedback we CAN claim a high degree of fidelity between our practice tests and the real thing.

Also, I noticed two surprising things in the MGMAT:
1: Even if I do 25 questions correct in my verbal, my verbal score ranges >=38. But in GMAT Prep I didn't see this phenomenon. I did one GMAT prep and intentionally marked wrong answers and got 25 correct in verbal out of 41. My Verbal score came out to be 30. Perhaps my analysis is wrong, but that's what I want to know from the instructors.

well, remember, it's an adaptive test!

if you're taking our practice tests and giving an honest effort, then you are probably missing the harder questions on the section. on the other hand, if you're simply marking random wrong answers on gmat prep, then you are probably giving wrong answers to a bunch of easier questions in addition to the harder ones.
this discrepancy would of course have a massive effect on your overall score -- even with an identical number of incorrect answers, you're going to have a much higher score overall if those incorrect answers occur almost exclusively on difficult questions.

2: The last two tests of MGMAT skewed my score to 750+. Is it because MGMAT wants to boost the morale of the students during its last two tests? I'm asking this because I answered the same number of questions correctly in the last two tests as in the fourth test in which I got 680. But the score delta was too much. Please explain.

you may want to go back to those tests and take a look at the reported difficulty levels of the questions.
one problem that currently exists on our cat tests -- a problem that we're working on remedying -- is the lack of sufficient 700-800 problems to accommodate students with scores as high as yours. as a result, such students (including you, i would presume) tend to run out of these questions around the fifth exam. as a result, the adaptive algorithm uses the hardest questions remaining in the pool -- the 600-700 questions -- but must extrapolate your performance therefrom. if you nail those questions, it's going to attribute an extremely high score to your performance.

[direstraits007]

hi --

Test 1 (Free): 720 (Q50/V38)
Test 2: 680 (Q49/V34)
Test 3: 740 (Q49/V42)
Test 4: 680 (Q48/V35)
Test 5: 770 (Q51/V44)
Test 6: 760 (Q51/V42)

those are some serious scores. nice job.

So, How close are the MGMAT Scores, when we compare them to the real GMAT scores? I know our instructors would be having fair enough data handy, which can rightaway relate the MGMAT scores to the real GMAT score with a potential delta. So, please let me know where I stand and if I consider all other factors same then how much would be my range of score in the GMAT ?

nah, we actually don't track statistics like that -- there are just too many variables that could invalidate any data we might collect.
the four most important variables that would render those data suspicious, if we were to go to the effort of collecting them, are:
(1) response bias: the only official scores we're going to hear are those from students who actively choose to come back and report them to us. as you might imagine, students with middle-of-the-road results aren't normally going to be motivated to do so. as with pretty much any other self-reported survey, we would receive disproportionate numbers of extreme results -- both from students who were elated with a performance beyond their wildest dreams, and also from students who were disappointed in their performance. as a result, we would probably wind up with some sort of bimodal distribution that would in no way reflect the true nature of our students' performances.
(2) confounding variables in test administration: we would basically have to trust that our students had taken their practice tests in perfect exam-like conditions. of course, this isn't always going to be the case: many students will cut corners with such things as taking excessively long breaks, pressing the pause button, and so on.
(3) study plan: many of our students exhaust our practice tests early, continue to study for another month or two, and then evaluate the rest of their preparatory trajectory with other practice tests (such as gmat prep). it would be disingenuous at best, and downright dishonest at worst, to compare such students' results to the results obtained by students who took the official exam right after finishing our battery of tests.
(4) random variation: remember that the official exam, even despite its adaptive nature, still has a relatively sizable standard error -- approximately 30 points. this means that there is going to be considerable "noise" in your exam results. therefore, if the delta between our practice tests and the real test is small enough, it will be completely obliterated by this random noise.

the short answer is, basically, no, we don't collect aggregate data -- and we have plenty of good reasons not to.

despite the lack of exhaustive data, however, from many students' feedback we CAN claim a high degree of fidelity between our practice tests and the real thing.

Also, I noticed two surprising things in the MGMAT:
1: Even if I do 25 questions correct in my verbal, my verbal score ranges >=38. But in GMAT Prep I didn't see this phenomenon. I did one GMAT prep and intentionally marked wrong answers and got 25 correct in verbal out of 41. My Verbal score came out to be 30. Perhaps my analysis is wrong, but that's what I want to know from the instructors.

well, remember, it's an adaptive test!

if you're taking our practice tests and giving an honest effort, then you are probably missing the harder questions on the section. on the other hand, if you're simply marking random wrong answers on gmat prep, then you are probably giving wrong answers to a bunch of easier questions in addition to the harder ones.
this discrepancy would of course have a massive effect on your overall score -- even with an identical number of incorrect answers, you're going to have a much higher score overall if those incorrect answers occur almost exclusively on difficult questions.

2: The last two tests of MGMAT skewed my score to 750+. Is it because MGMAT wants to boost the morale of the students during its last two tests? I'm asking this because I answered the same number of questions correctly in the last two tests as in the fourth test in which I got 680. But the score delta was too much. Please explain.

you may want to go back to those tests and take a look at the reported difficulty levels of the questions.
one problem that currently exists on our cat tests -- a problem that we're working on remedying -- is the lack of sufficient 700-800 problems to accommodate students with scores as high as yours. as a result, such students (including you, i would presume) tend to run out of these questions around the fifth exam. as a result, the adaptive algorithm uses the hardest questions remaining in the pool -- the 600-700 questions -- but must extrapolate your performance therefrom. if you nail those questions, it's going to attribute an extremely high score to your performance.

Thanks Ron for providing wonderful clarification. Yes I saw the last test while doing the analysis and I found that there were only 4 questions in maths and only 11 questions in verbal of 700-800 difficulty level. :(. Oops..after seeing my scores in the last two tests I thought I'm all set for the D-day.

So from your above detailed description, shall I infer that in order to have a good GMAT score one should have at least all the easy questions correct and should also try to get as many as difficult questions correct. So if someone gets one difficult question wrong and one easy question wrong, then does that mean, the penalty would be more on the latter case ?

Thanks!

~GeeMate.

[StaceyKoprince]

To answer your last question, yes, it hurts your score more to get a lower-level question wrong than a higher-level one.

Also, re: the data on student scores, we do actually track the data, but we tell everyone what Ron said up above in order to make sure that everyone is aware that the data may be skewed.

The median official test score (for the students who replied to our surveys) is between about a 680 and a 690. The standard deviation of the official test, as Ron mentioned above, is about 30 points.

The standard error between an MGMAT student's last MGMAT practice test and that same student's official test score is about 50 points. Generally speaking, that means most (but not all) of the students who responded to our surveys scored within 50 points (plus or minus) of their final practice test score when they took the real test.

We do our best to get people to tell us how they did (seriously - we ask multiple times over a period of months after the class ends), but obviously not everyone replies.

[direstraits007]

To answer your last question, yes, it hurts your score more to get a lower-level question wrong than a higher-level one.

Also, re: the data on student scores, we do actually track the data, but we tell everyone what Ron said up above in order to make sure that everyone is aware that the data may be skewed.

The median official test score (for the students who replied to our surveys) is between about a 680 and a 690. The standard deviation of the official test, as Ron mentioned above, is about 30 points.

The standard error between an MGMAT student's last MGMAT practice test and that same student's official test score is about 50 points. Generally speaking, that means most (but not all) of the students who responded to our surveys scored within 50 points (plus or minus) of their final practice test score when they took the real test.

We do our best to get people to tell us how they did (seriously - we ask multiple times over a period of months after the class ends), but obviously not everyone replies.

Thanks Stacey!

So, If I understand it correctly, then is it right to assume that we can expect a delta of 50 points in the GMAT score than the score we get in the final MGMAT test. But, I've again a question for you. I remember somewhere above Ron mentioned that as most of the 700-800 questions were pulled off in the previous MGMATs, the final MGMAT was not showing substantial number of 700-800 GMAT level questions, which itself makes the test easier as compared to GMAT. right ? So, ideally speaking, shouldn't we take a delta of 50 points on the first or second MGMAT test to estimate the GMAT score ?

Thanks!

~GeeMate

[StaceyKoprince]

is it right to assume that we can expect a delta of 50 points in the GMAT score than the score we get in the final MGMAT test.

Statistically speaking, 2/3 of people can expect a delta of 50 points and 1/3 of people can expect a delta of >50 points. So it's more likely that you'll be within 50 points than otherwise, yes, but it's not a slam-dunk.

It isn't necessarily the case that "most of the 700-800 questions were pulled off in the previous MGMATs." That's ONLY for people who are scoring very highly right from the beginning. Your quant score was already in the upper 80s right from the very first test - so, for you, yes, you may have run out of 700+ questions by the last test (and we can generally confirm that based on your report of the # of 700+ questions on your last test). Your verbal score was a bit more up and down, so I'm not as sure about that one. It may be a bit inflated but it may not.

Most likely, your final overall test score is inflated. (And possibly the 2nd to last? Check the data on that one.) So, yes, you should take that into account when trying to anticipate your score. (Though I will note that we don't actually adjust for that when we're doing our own calculation - for whatever that's worth.)

I will say that it's not easy to use up all of our 700+ questions before the 6th test, so the fact that you did tells me that you're ready to take the real test. :)

[direstraits007]

is it right to assume that we can expect a delta of 50 points in the GMAT score than the score we get in the final MGMAT test.

Statistically speaking, 2/3 of people can expect a delta of 50 points and 1/3 of people can expect a delta of >50 points. So it's more likely that you'll be within 50 points than otherwise, yes, but it's not a slam-dunk.

It isn't necessarily the case that "most of the 700-800 questions were pulled off in the previous MGMATs." That's ONLY for people who are scoring very highly right from the beginning. Your quant score was already in the upper 80s right from the very first test - so, for you, yes, you may have run out of 700+ questions by the last test (and we can generally confirm that based on your report of the # of 700+ questions on your last test). Your verbal score was a bit more up and down, so I'm not as sure about that one. It may be a bit inflated but it may not.

Most likely, your final overall test score is inflated. (And possibly the 2nd to last? Check the data on that one.) So, yes, you should take that into account when trying to anticipate your score. (Though I will note that we don't actually adjust for that when we're doing our own calculation - for whatever that's worth.)

I will say that it's not easy to use up all of our 700+ questions before the 6th test, so the fact that you did tells me that you're ready to take the real test. :)

Thanks Stacey for the clarifications.

Thanks!

~GeeMate

[StaceyKoprince]

good luck!

[kartik.ibm]

Hi "Direstraits",

I've been reading this very interesting conversation you've been having with the instructors, and was wondering if you could let us know how your GMAT went? Wasn't it supposed to be on March 20th?

Thanks,

KK

[StaceyKoprince]

Thanks for your interest, kartik. direstraits, we would certainly be interested to know how things went if you are willing to share!