CAT- irregular precentile- blip

[Burfi]

Stacey
I have been taking Manhattan CATS and scoring in mid 600's.
CAT1A: 630
CAT2A: 640
CAT3A: 660
CAT4A: 650
CAT5A: 560- BOOM- I took this yesterday

Not sure what happened ... I took all above tests the same way- AWA+ IR and simulated test conditions. What puzzles me is why my score dropped like 100 points. I am yet to finish my analysis and then will post what could have caused, but I did see one abnormality which made me wonder if there was a blip in the scoring algorithm.

I answered first 5 questions in quant correctly which pushed my percentile to 98% but then I answered the 6th question-700-800 level incorrectly and it dropped my percentile to 73%( 25% drop??). So my question is: " is that normal"? if so, are we being penalized that high?

In verbal also I faced similar situation ( although I answered first question incorrectly but I answered next 6 questions correctly and it pushed my percentile to 99%. I answered the next question incorrectly and it dropped it to 85%( a drop of 14% only).

Looks like algorithm is penalizing more if we don't answer correctly in first 10 questions? Can you please address this particular scenario- especially the quant one ?

[StaceyKoprince]

I'll ask our algorithm guru about this but I do know one thing already: the "percentile" thing is not literally your exact current scoring level. There are "tiers" within the percentile thing and you can be put into, for example, the top tier (which then reads 98th percentile) even if your score is not literally 98th percentile. The underlying algorithm is calculating the exact percentile - it's just not showing you that precise data on the report.

The bigger thing, though, is that this was not likely the cause, or the only cause, of your 100-point drop, so we need to dig deeper to figure out what happened there.

Do two things. First, read this:
http://www.manhattangmat.com/blog/index ... ent-wrong/

Second, use the below to analyze your most recent MGMAT CAT, then do the same for the 1 or 2 higher scores before that:
http://www.manhattangmat.com/blog/index ... ice-tests/

Then come back here and tell us the results of all of your analysis and what you think you should do based on that analysis. (Note: do share an analysis with us, not just the raw data. Part of getting better is developing your ability to analyze your results - figure out what they mean and what you think you should do about them!) If you have questions about anything, of course let us know. (And I will let you know what I hear from our algorithm guy!)

[StaceyKoprince]

okay, I spoke with our algorithm guy (thanks, Tate!) and he explained things to me. The test basically allows "swings" within a 4-point range (scoring point, not percentile point), because it is managing so many variables when deciding which question to choose next (it has to hit a certain mix of all kinds of different categories, and it also tries to prioritize questions that are more "discriminating" - which is the term we use for questions that are really good at discriminating between higher and lower-level testers at some particular scoring level). Your test "swung" from 51 to 47 (4 points), so within the allowed range - and note that it can also go up. You likely jumped up right at the beginning as well, since you got the first several Qs right. :)

It used to be that the 4-point range typically covered a bit less than 20 percentile points, but as the pool of test takers has changed over time, the percentile rankings assigned to the various scores have changed. That changes the percentile rankings you see but not the way the underlying scoring algorithm works.

Further, this one question would not (by itself) be the cause of your overall score drop - if you have the skills (and you clearly do or you wouldn't have been getting the other scores that you got), you will climb back up again. So something else was going on during that test that caused such a big score drop. Do the analysis I described in my first post and come back here to tell me what you found.

[Burfi]

Thanks Stacey for your detailed explanation! I took another CAT the following day and scored 640. At least I know 560 was just an anomaly? or something else. After more thinking I came up with 2 possible reasons that could have caused such a huge drop.
First Reason: I always take my tests either in the afternoon or in the evening. 560 CAT was taken in the morning. In fact I took my real test also in the morning( same time) and bombed it as well. The problem is all afternoon slots are take away hence was forced to take real test in the morning. But Big revelation 560 CAT confirms I am peak of my performance in the afternoon. I read this phenomenon before, but frankly did not take it seriously but seeing my score fluctuations I must admit I am starting to believe time of day makes hell lot of difference.

Second thing is I spent little more than usual time in the initial questions and I had to rush in the middle.

I always review my CATs . Since last 2 CATS I have been reviewing the same way you suggested in your articles. But I exhausted all my CATS and I am afraid if I review it question by question then when I reset the pool I will remember the questions. I already did my review based on the reports generated but did not review each question. Do I still do the review? Please advice. Thanks again Stacey!

[StaceyKoprince]

Time of day can make a HUGE difference. I'm not a morning person either and I do much worse in the morning - I avoid that time slot like the plague. :)

FYI - when they first release time slots, they typically mostly release the 8a slots. As things start to fill up, they then make additional later morning or afternoon slots available. (They administer many different tests that are all of different lengths - to maximize the space used, they first fill up the 8a slots and then see what else they can offer based on however long those various tests are.)

And timing can be huge, of course. Good to know that you did rebound on your next CAT - that's reassuring.

Re: reviewing the last 2 CATs, yes, that is a concern. I'd base my decision about what to review on the results themselves. For any that are in the "I went a bit long but I did get it right" category, do review (because you want to learn how to do those questions more quickly). For any in the "I thought I got this right - how did I get it wrong?!?" category, especially if the time was normal or on the fast side, again you need to review (because you need to figure out what mistake(s) you made and how to minimize them in future).

For correct-in-normal-time, yes there are still things you could learn, but... you also don't want to "spoil" yourself for your next CAT. For "too-long-and-wrong," you already know what the issue is: you should have cut yourself off faster. Again, you could learn something by reviewing those individual problems, but there's that "spoilage" tradeoff again. So for those, you don't have to review (or not as much, anyway).

I will say one thing, though - you really shouldn't be taking tests more frequently than once every couple of weeks*. And you also have other test sources, such as GMATPrep. So if you make your next test GMATPrep (assuming you haven't done both of those already), then the next MGMAT test won't be for like a month, by which time you'll have forgotten many / most of the individual questions.

* Caveat: if you're getting close to the real test date, then you might take two tests 1 week apart: the first 2 weeks before the real test and the second 1 week before the real test.

[Burfi]

Hi Stacey
Thanks for the info on time slots. I did not know about it. so how do I go about getting an afternoon slots? do I just keep checking for afternoon slots for desired dates? I desperately want late morning or early afternoon slots if possible.

I usually take one test every weekend. Since I got 560 I wanted to take another test at a different time just to see 560 was an anomaly, so I took the next day. Other wise I take every weekend or sometimes once every 2 weeks. BTW I took gmatprep several times and I know the answers very well hence taking MGMAT only for now. MGMAT too I have taken twice already. Hoping to forget answers so I can repeat the CATS. Please advice any other CATS that I could possibly take. But Frankly I like only gmatprep( which is best) and MGMAT.

I will review my tests and post the analysis. Meanwhile I have not set the date. I want to achieve my desired score and score at least 700 several times on MGMAT before I book the exam. Thanks again for your detailed analysis/info!

[StaceyKoprince]

Stop taking so many practice tests! Once a week is NOT a good use of your time. Each practice test gives you a HUGE amount of data about your strengths and weaknesses. Until you make substantial progress in multiple areas of weakness, taking another practice test mostly just tells you the same things you already know!

Not more than once every 2 weeks. And, actually, not on a pre-determined, set calendar or timeframe. You take a practice test when you feel that you've made substantial progress in addressing the issues with your last practice test, however long that takes. THEN take another, get new data, and start tackling whatever the new issues are.

So, one way to forget the questions? No more practice tests for you for at least 3 weeks. Go do something about these weaknesses before you bother taking another test!

For instance, how many careless errors do you make? Taking more tests won't minimize your careless errors. Read this.
http://www.manhattangmat.com/blog/index ... -the-gmat/