From GMATPrep 2.0 to GMATPrep 2.1.277 - surprising results!!

[catennacio]

Hey guys,

I took the GMATPrep CAT and found very surprising results which I can't explain:

Yesterday, I took 2 CATs with GMATPrep 2.0 (the very early version of GMATPrep released on April 3rd from mba.com)
I got the results:
- CAT1: 640 (Q48, V30, IR2). Quant = 24/37 right (13 wrong). Verbal = 26/41 right (15 wrong). I didn't know that I couldn't go back the review screen so I ended the test.

Fine, I decided to take it again on the afternoon because in CAT1 somebody called me on the phone and we talked for 5 mins:
- CAT2: 700 (Q49, V35, IR2)
Quant = 26/37 (11 wrong)
Verbal = 27/41 (14 wrong).
This time, I reviewed my session and found that the software has flaws in detecting right answers in IR (I heard somewhere on this forum).

So today I went to mba.com and downloaded the 2.1.277 version (latest one that they have) and retook the test again... and here comes the shocking results:
- CAT1: 590 (Q49, V21, IR7)!!!
Quant = 28/37 (9 wrong).
Verbal = 27/41 (14 wrong)

Quant is fine, although I still slightly doubt that to have a Q49 on the 2.1.277 version is more difficult than to have a Q49 on the 2.0 version. This is illustrated by the fact that my Quant hit rate has to be 28/37, as compared to 26/37 in the previous version, in order to have a Q49. OK, easy questions and difficult questions factors come into play, so I'm OK with that.

However I can't explain how my Verbal fluctuates that much, provided that I have the same hit rate in the new version and the old version: both at 27/41 (14 wrong). In the previous version, I got a V35 whereas in the new version I only got a V21!!

My questions:
- If my hit rate is around 27/41, what is the APPROXIMATE V score do I get after all? It can't fluctuate from V21 to V35 as in my case, no matter how they tweaked the scoring algorithm.
- Does the scoring algorithm differentiate among wrong answers? I mean does it consider an answer to be wrong, very wrong, very very wrong and extremely wrong? This is the only explanation I can offer to myself to understand how the algorithm works. So even if I got the same number of wrong answers, because I chose a lot of extremely wrong answers, the software thinks I'm so stupid that it gives me a V21 instead of a V35.
- Has anyone experienced this before? How did you explain it?

Thanks!

-C

[StaceyKoprince]

If my hit rate is around 27/41, what is the APPROXIMATE V score do I get after all? It can't fluctuate from V21 to V35 as in my case, no matter how they tweaked the scoring algorithm.

Yes, it can - this is exactly how the test works. :) The score is specifically *not* based on the percentage correct. The whole point of an adaptive test, actually, is that people at most scoring levels get about the same percentage of questions correct - even at very different scoring levels. What changes drastically is the difficulty level of the questions each person sees, not (so much) how many they get right. This is not like the tests you're used to from school.

It is concerning, of course, that your verbal score dropped so much. We have to figure out why. Read this article and see what you think might apply in your case:

http://www.manhattangmat.com/blog/index ... ent-wrong/

Then come back here and let me know.

Also, I'd like you to get a better idea of how the scoring works, because that has a major impact on how you take the test (in particular in terms of how you handle the timing).

Read the "Scoring" section of our free e-book The GMAT Uncovered (look in your student center). Then read this:
http://www.manhattangmat.com/blog/index ... to-win-it/

[catennacio]

Hi Stacey,

Thank you for taking time to respond. I have read your suggested links.

When I took the CAT2 of the old version (the one with 700 score), I did take the essay section as well as the IR section. I was almost the same with the real test, not to mention that it was the 2nd CAT I took on the same day (I know, I know, I'm not supposed to do that.. but it's a different story).

In the CAT1 of the new version (the one with score 590), I didn't take the essay section because for some reason, I got the same essay question with the real GMAT exam I took a month ago. I did take the IR section, but this section was almost identical with the IR section in CAT2 of the old version (I don't know why), so I quickly selected the answers and didn't spend too much brain power. In the Quant and Verbal, I had a feeling that they were a little easier than before because I got some redundant questions (as illustrated by the fact that I got more questions right on Quant, because of these redundant questions). I felt I performed on par with the previous test in Verbal.

For both CAT sections I had about 1 minute left.

After reading the "In It to Win It" article, I think this is what I was doing. I didn't try to score right on all the questions (I know it's impossible), I was willing to let things go by guessing. This is shown by the fact that for each question that I flagged for review because I guessed, when I reviewed, mostly they were wrong. But I accepted that because those are either questions I had no clue, or questions I already spent too much time on, so I moved on.

When I reviewed the Verbal section of the low-scored CAT, I got a lot of RC wrong, mainly because I hated it, so I ended up guessing most of them. Out of 14 wrong questions, I got 6 wrong RCs. It could be because those wrong RCs are really easy ones, which I should have gotten right. For the other 8 wrongs (3 CRs and 5 SCs), I made silly mistakes, and most wrong SC are from 700 level, I think - the ones with meaning (the economy of country A, B, C vs. the economies of country A, B, C), with idiom (strenthen something with vs. strengthen something through) etc..

I posted the same discussion on gmatclub and got a response from an official GMAC officer. She too agreed that it is unusual to have such difference in my Verbal score and they are looking into my log files. For now, the only way to understand the score is to think as what you suggested, that I have too many wrong way-too-easy questions. The reason I'm concerned is that:

1. I want to know, or at least of rough idea on how the scoring algorithm works in order to manage/pace my time and balance between really letting things go and trying a little harder.

2. Because GMATPrep2 is a completely new piece of software (based on the fact that it is written on a different platform - Adobe Air), I want to make sure that it is bug-free, or at least stable enough before retaking another real GMAT.

Thanks.

-C

PS: For some reason I can't see a download link in this page http://www.manhattangmat.com/gmat-uncovered.cfm. Can you please check?

[StaceyKoprince]

Unfortunately, there is only one static IR section in the GMATPrep 2.x software - every time you take the test (including the 2nd of the original 2 tests), you will see the exact same 12 questions. :(

As a result, I recommended pulling together a set of 12 questions from some other source so that you can mimic the real test conditions.

If you happen to see an essay prompt that you recognize, open up your OG12 or OG13 and pick a different one - but don't skip that essay. It's important for you to practice stamina.

It's always possible that, when they update the software, there's a glitch somewhere (actually, when they first launched GMATPrep 2.0, there was a glitch with the scoring on the IR section, and they quickly fixed it). But it is also possible to have a wide range of scores while getting about the same # of questions correct. Let us know what they say - I'd love to know.

You can't download the GMAT Uncovered Guide from that page. You have to log into your student center in order to access it. Once you log in, go to the Resources and Downloads section and look on the right-hand side for the link. And then yes, I agree, go read the Scoring section to get a better understanding of how the scoring works.

[sameersinha10]

Same issue faced by me.
On May 12 2012 when I started GMAT Preparation I got 680 (Q45V38) in CAT 1
After 3 months, 2/3rd way into my preparation I took CAT 2 .
(The software synched up and updated to Adobe AiR earlier)
I got 640 (Q48V30). In Verbal I got only 11 wrong and 4 out of first 5 wrong. As many as 17 questions were correct in a go.
I postponed my GMAT date from August to Sep and need to work on RC and CR . I got 5 RC wrong and 4 CR Wrong.
Will try to reinstall and try out again.

[StaceyKoprince]

I know there were some bugs in the early software for IR. I'm not sure whether there were any bugs anywhere else.

Still, we can assume that if there were any bugs early on, they've likely caught and fixed them now.