GMAT Scoring System Revealed?

[mindadze]

Dear All,

Recently I did GMAT Prep test #2, only Verbal Section. I got 42, with only 5 incorrect answers. I clearly understand that its not about how much u get correct or incorrect, but please have a look at this screen-shot:

http://postimage.org/image/qk47hhr5d/

Incorrect Qs:
#5 - SC
#27 - CR
#33 - RC
#37 - SC
#38 - CR

I am just trying to understand how the scoring of this test works. This particular example of my scores seems strange to me. Judging from the scoring experience of taking first GMAT Prep test, score here is not adequate (though I am not sure).

I know how sensitive tutors are towards the questioning of the correct answers of official problems and I can imagine that my questioning of the scoring system may cause some controversy. But maybe there is something that we are missing in terms of scoring system of the GMAT.

First of all what do you think about this particular example? (please refer to the screen-shot)

Now lets discuss some issues concerning the scoring system in general:

1) Do you think that proportions of SC, RC and CR in the INCORRECT answers affect the score? i.e. provided ceteris paribus condition towards ALL other aspects, the two scores will differ if u get a) 5 incorrect and all of them are SC b) 5 incorrect and say 2 SC, 2 RC and 1 CR are incorrect.

2) I actually did not have time left for last 5 questions and guessed all of them, randomly. Did the test acknowledge this, provided I chose answers in a very short time, and therefore gave a low score?

3) Is GMAT Prep scoring system "equivalent" to that of the real GMAT?

4) Does the test actually score timing? Does the time one spends on a particular answer, which he or she gets correct, affect the score?

Frankly, I am not even sure if this result is somehow inadequate, I just guess so, provided previous GMAT Prep experiences.

Getting many questions incorrect or leaving some of them blank at the end effects the score significantly, this was confirmed by GMAC officially:

"There is a severe penalty for not completing the GMAT test."
- Larry Rudner

But what about earlier questions being worth more? Stacey Koprince kindly delivered news from GMAC Test Prep Summit, and we are now more confident that this is a myth, but previously it was a commonly held belief. Some respected GMAT textbooks explicitly mentioned that early questions are worth more.

All I want to say is that GMAT Scoring system is not fully revealed and exchanging ideas about it should be interesting.

Thank you for your comments.

P.S tried to choose correct folder for this post, sorry if I missed.

[StaceyKoprince]

I'm going to take these questions in a different order than you gave.

First, there is an enormous amount of data that we simply cannot see from the score results; the most glaring omission is difficulty level. Trying to analyze any of this without the difficulty level data... well, that's really the core of everything. We can't even start having a good conversation about what's going on with the algorithm without that data (and they don't, of course, give us that data). And that's for what we do know about the algorithm! Many of the details are a closely guarded secret.

GMATPrep is either the same as the real test or close to it with some differences - but they haven't told us for sure. For instance, the real test includes experimental questions, but GMATPrep consists only of questions that have already been tested, used in the "live" test pool, retired, and released as GMATPrep questions. On the real test, the experimentals don't impact your score; my guess is that GMATPrep does count all of the questions, so that would be at least one difference compared to the real test.

Next, we do know that the real test puts in quite severe penalties for a high proportion of wrong answers in a row, or a higher proportion of wrong answers in a sequence than would be expected given the scoring level to that point.

For example, someone scoring at the 50th percentile who gets the last two questions wrong will not see much of a negative impact on the score because it's not unusual for someone at that level to get two in a row wrong. The score might not even change.

On the other hand, someone scoring in the 99th percentile would see a serious negative impact from getting two in a row wrong at or near the end because that's not expected for someone at that scoring level. They do this specifically because it's not enough to get a bunch of hard questions right. You also have to get through the test - you have to be able to maintain that high of a performance all the way to Q41. If you can't, your score is going to drop - big.

42 on verbal is the 95th percentile. You missed 2 of the last 5 Qs. That proportion is very much not expected at that level. So it's quite possible that you were at the 97th (44) or 99th (45) level and those two questions brought you down - just a bit, to the 95th percentile, but it feels like a huge drop because it's "only" 42. Further, at that level, the drop "compounds." You get the first one wrong, that drops the level, you get the next one wrong, that drops it even more. Chances are you dropped down to maybe 40 or 41 and had to work your way back up to 43.

There's an old report that talks about someone scoring at a 780 level. The report specifically says that this person would have to answer *at least* the first 5 questions in row correctly in order to offset guessing on just one question at the end of the section. (This report is old enough that the specific details may have changed by now, but this is generally the way a CAT works.)

The test does not take into account how long you spend on any question - so, no, there was no penalty based simply upon rushing on the last 5 (besides the fact that you got two wrong).

I do not think that the proportion of incorrect answers (in terms of question type) affects the score, no, although I have never specifically asked them this, so I may not be right.

A little more on the early questions myth: I was working in the industry when the myth was first propagated in the late 90s (and believed it myself - in fact, if you have half an hour, I can tell you exactly how the whole thing started, because I was there. :) But it is definitively a myth - I have that straight from the mouth of the man in charge of the algorithm (Dr. Lawrence "Larry" Rudner), not to mention I've read enough academic papers to convince me. If you'd like to learn a bit more about why, you can read our free e-book The GMAT Uncovered Guide. (If you already have an account with us - even a free account - the Guide is already in your student center account.)

This is a great topic to discuss, though. I think I'm going to write an article about it - both what we do know and what we don't know!

[mindadze]

Thank you Stacey for your reply and good luck with the Article, cant wait for it. :)

[StaceyKoprince]

Check it out here:
http://www.beatthegmat.com/mba/2012/03/ ... tions-myth