Lower level of harder questions in later exams

[alok.anant]

Hi Stacey,

I have bought the 6 sets of MGMAT CATs and am using them for my practice for the real thing along with the GMAT Prep. Below are the details of my scores:

GMAT Prep 1: 760 (However there were a lot of repeat questions from OG and some others that I had practiced before)


MGMAT CAT 1: Q 45, V 34 Total: 650

MGMAT CAT 2: Q 47, V 36 Total: 680

MGMAT CAT 3: Q 48, V 41 Total: 730

MGMAT CAT 4: Q 48, V 42 Total: 730

However, in the last test, I saw that the number of 700-800 questions had reduced by a large percentage and I could finish the Verbal section with 8 minutes remaining. Given that I used to struggle to finish the test towards the end till CAT 3, I really got a feeling that because I was not challenged as I had expected from an MGMAT CAT, I had done badly. However, the 730 came as a surprise.

I have seen posts from around 2007 around similar issues where you have commented that your staff is working on adding in more difficult questions so that the pool is not exhausted by CAT 3 or 4.

Could you kindly let me know if this change has been made?

Also, does your predicted SD of 50 still hold in such cases?

If someone could look at my tests and check if the questions have been presented in an expected manner then that would be really helpful.

In the second test (Score 680), 1 question had remained unanswered in the Quant section. Could you tell me how much of an impact would it have on the overall score if I had attempted this with a wrong answer?


Thanks for your help in advance!

[StaceyKoprince]

We did add more questions, but right now the level in the pool has dropped a bit again because we have pulled a number of problematic questions in order to fix them. As we fix them, we add them back into the pool, and I'd say we're about half done with this task.

Your first two tests were only at 34 and 36 for verbal, so I'm a bit surprised that this would happen in your case. Normally, when it does occur, it occurs when someone's scoring in the 40s (verbal) from the start. If you want someone to take a look at your last test to see whether this did happen to you, please send an email to techsupport@manhattangmat.com and they will follow up. Make sure to include your account info and specify which test you want them to review.

The SD calculation that we do does not filter out these situations, so the statistic holds in general for everyone. Practically speaking, though, I would assume that a person who hits this situation has a bit of an inflated score - so it would be worth following up with tech support, above, to get an idea of whether we should discount your score a bit when evaluating your progress. (Question: you only noticed this in T4, right? T3 was also 730 but you didn't notice anything odd on that one? If that's the case, then we can assume the 730 score on the 3rd test is a valid indicator, +/- the SD.)

Re: your question about the unanswered problem, there isn't enough info to be able to answer that question exactly (it depends upon many, many factors), but I can give you an idea. Leaving the question blank would result in a drop of 3 to 4 percentile points. Answering the question incorrectly would likely have resulted in a drop of around 1 to 2 percentile points (again, depending upon many factors, including the fact that we're talking about this happening with only one question; 4 questions in a row would have resulted in a higher per-question penalty).

[alok.anant]

Thanks for your inputs.

Yes, I did see the reduced level of difficulty only in T4.

Let me send out that email to tech support.

Thanks once again!

[StaceyKoprince]

Well, that's good at least - that means T3 is definitely valid! :)

In terms of whether it accurately reflects your current scoring level, did you take the test under full official conditions? (Do the essays, don't use the pause button, take only two 8-min breaks between sections, etc.) If you did take it under official test conditions, then that score does reflect your current scoring level, +/- 50 points (the standard deviation).

If you did NOT take it under official conditions, let us know exactly how you deviated from official conditions and we'll give you an idea of how much that might have affected your score.

[alok.anant]

Hi Stacey,

I gave the GMAT yesterday and wanted to share my feedback on the CATs and check on a doubt that I had.

Feedback:

1) Though the 6 CATs are really good practice and do help in predicting the actual score, I felt that by the 5th and 6th tests, the difficulty level had decreased a lot, as only a few 700-800 questions were left. If only the pool of questions could be strengthened to cater to people who score above 700 in most tests, we would have a tremendous GMAT-like experience. For me, only the 3rd and 4th really predicted my actual level (in both I scored 730), as they were very challenging, whereas during the first two I was raw and the 5th and 6th were just too easy (I actually scored a 780 on the last test with more than 5 minutes to spare in each section)

2) As most people predict, my actual score (730) was a bit over my average MGMAT score (716) and the same as my 3rd and 4th tests

Doubt:

My score breakdown is as follows:

Quant: 50 (93%ile)
Verbal: 40 (89%ile)

Total: 730 (96%ile)

With a 50 and 40, I would have expected a score of over 730, as recently I have seen people score 730 with a 50 and 38. I understand that the algorithm has certain gray areas in calculating the score, but could you share your views on this?

In most of my MGMATs a 730 reflected the 97th percentile. However, in the actual test, it was the 96th. Could you shed some light on this? (My doubt is that GMAC might have updated their percentile and actual score data recently for the observations)

Finally, thanks for building an excellent support system with the books, tests and classroom courses that MGMAT offers. Most of the GMAT takers have benefited in one way or the other from your services.

Regards
Anant

[tim]

First, congratulations on your 730! To answer your questions at once - the GMAT scoring algorithm is a closely-guarded secret that we can only reverse-engineer so effectively. On top of that, the GMAT updates this mechanism whenever it feels like it, so just keep in mind that however they say the scores and percentiles match up is how they match up. :) We do have some information indicating the GMAT recently updated its system so that it now assigns lower percentiles to scores than it did a few months ago. Obviously this new information trumps any speculation or any empirical results from older tests..

[alok.anant]

Hi Tim,

Thanks for the response. That was really helpful.

Regards,
Anant

[tim]

Glad to hear it..