How important is the first 10 questions and "wrong in row" ?

[ZhenM102]

Hi, I mock tested myself four times using prep-Gmat software, and I have to say the myth is true about "The importance of first 10 questions and wrong answer in row"

I used to think if I get better accuracy rate, my score should be higher, but it is not the case at all! Below is my summarized report on 4 tests

Please ignore the first test, it was taken before I started studying (And it has very low accuracy rate). And the recent 3 tests are taken on 6/9/2016 6/13/2016 6/14/2016, after two months of painful study

And as you can tell, the accuracy rate is pretty much the same, not very dramatic difference.
So the only explanation for my declining score is the position of wrong answers as well as whether they are in row or not.

The 2 test is highest, because it only has 3 wrong after first ten questions
The 3 test is the median, but i have two times "wrong in row", which is 4 times and 5 times in verbal section (I was stunned that even making so many mistakes I can still get a 690) But luckily, they are after first 10 questions.
The 4 test, says best about the adaptive software, a higher correct rate than test 3 but lower score, which is caused by my verbal "wrong in row" in the first ten questions.
So the evidence is very clear, it doesn't matter how many right answer I get, what matter is where you make mistakes (before or after first 10) and "wrong in row"

Conclusion: Altough Accuracy rate plays a role, (otherwise my first test should also get closer to 700), but definitely not as important as the "position and wrong in row" factor. I would say 30% 70% ?

My final thought: There is no way for me to significantly increase accuracy rate in short period of time(will have first real test this coming Saturday), and also considering accuracy rate contributes so much less than the "position and wrong in row" factor, I wouldn't want to bother to study more. So the best is just to take as many GMAT as possible until I am LUCKY enough to have questions that not so hard in the first 10? (Very opportunist thought lol but I am kind of desperate now when each mock test results a lower score than before, even with a higher accuracy rate )

Any thought, advise, comfort guys?

[img]file:///C:/Users/Zhen/Desktop/report.PNG[/img]

[ZhenM102]

emmmm, the image doesn't come out, do you guys know how to upload image?
it is not online image, I saved it on my desktop, but i don't think this command is working...

But anyway, you pretty much get the ideas, (I will upload the image when i figure it out)

[StaceyKoprince]

The scoring algorithm is...well, it's an actual

algorithm

. There's no way to backwards-engineer how it works by taking the test 4 times and looking for patterns based on right and wrong answers. Someone could perhaps come up with some good theories by running 10,000+ Monte Carlo simulations. But even then, there are limitations to what that can show. (I've read the academic papers that were based on such studies.)

The scoring algorithm is based on a well-known theory (Item Response Theory). You could actually read all about it, if you really wanted to know. Of course, most people find it incredibly boring and obtuse.

It is the case that accuracy is not how this test is scored. Everyone will answer approximately 50-60% correct at almost all scoring levels (except at the very top and bottom). That's how the algorithm works. Most of the myths that arise about the scoring (including the idea that the earlier questions are worth more) occur because our brains just keep going back to the idea that accuracy must be at the heart of the score. We're so used to that from school that we can't really conceive of a test that works completely differently.

Short answer: the first 10 questions are not more important than the last 10 or the middle 10 or any particular 10 anywhere in the test. There are other factors in the algorithm that compensate for what might appear superficially to be more emphasis placed on earlier questions.

By the way, we (the instructors at my company) have run mock tests where we've gotten the first 5, 7, or 10 in row wrong. Then, we worked to the best of our ability for the rest of the test. We still scored in the high 40s on quant (out of 51) and the high 30s to low 40s on verbal. That wasn't to show how the algorithm works (you really can't from such a limited test), but to show that recovery is possible.

(Note: it is the case that you can say the earlier questions are worth a bit more IF your goal score is about a 500. The higher your goal score goes from 500, the less this is true, however.)

It is the case that getting many questions wrong in a row does have the potential to severely impact your score, primarily because it pulls your score down very low and then you have to recover. If you also spent a lot of time and mental energy on some those questions, then you may be running low on time or feeling mentally fatigued, making it even harder to recover.

(Very opportunist thought lol but I am kind of desperate now when each mock test results a lower score than before, even with a higher accuracy rate )

This is what I really want to address. Your score isn't where you want it to be, you're not sure why, and so you're looking for anything (desperately!) that could help you "beat" the test. You've been studying hard and this whole thing just seems baffling and futile. Right?

I don't know for sure what the problem is, but I have a very good guess (because I've seen it so many times before): you are trying to study and approach this test in the ways that worked for you in school...but those ways aren't working on the GMAT. And this goes back to the underlying assumption in your brain that the goal is to try to get more questions right.

Read these:
http://tinyurl.com/executivereasoning
https://www.manhattanprep.com/gmat/blog ... -the-gmat/
http://www.beatthegmat.com/mba/2016/02/ ... n-the-gmat

Really think about them. Read them again. Then come back and tell me what you think.

Also look at this:
http://tinyurl.com/2ndlevelofgmat

Tell me how your approach so far does and does not match up with all of that stuff. And tell me any ideas you have for what you think you need to do differently. (I'm not talking about individual study sessions. I'm talking about overall approach / mindset.)

[ZhenM102]

The scoring algorithm is...well, it's an actual

algorithm

. There's no way to backwards-engineer how it works by taking the test 4 times and looking for patterns based on right and wrong answers. Someone could perhaps come up with some good theories by running 10,000+ Monte Carlo simulations. But even then, there are limitations to what that can show. (I've read the academic papers that were based on such studies.)

The scoring algorithm is based on a well-known theory (Item Response Theory). You could actually read all about it, if you really wanted to know. Of course, most people find it incredibly boring and obtuse.

It is the case that accuracy is not how this test is scored. Everyone will answer approximately 50-60% correct at almost all scoring levels (except at the very top and bottom). That's how the algorithm works. Most of the myths that arise about the scoring (including the idea that the earlier questions are worth more) occur because our brains just keep going back to the idea that accuracy must be at the heart of the score. We're so used to that from school that we can't really conceive of a test that works completely differently.

Short answer: the first 10 questions are not more important than the last 10 or the middle 10 or any particular 10 anywhere in the test. There are other factors in the algorithm that compensate for what might appear superficially to be more emphasis placed on earlier questions.

By the way, we (the instructors at my company) have run mock tests where we've gotten the first 5, 7, or 10 in row wrong. Then, we worked to the best of our ability for the rest of the test. We still scored in the high 40s on quant (out of 51) and the high 30s to low 40s on verbal. That wasn't to show how the algorithm works (you really can't from such a limited test), but to show that recovery is possible.

(Note: it is the case that you can say the earlier questions are worth a bit more IF your goal score is about a 500. The higher your goal score goes from 500, the less this is true, however.)

It is the case that getting many questions wrong in a row does have the potential to severely impact your score, primarily because it pulls your score down very low and then you have to recover. If you also spent a lot of time and mental energy on some those questions, then you may be running low on time or feeling mentally fatigued, making it even harder to recover.

(Very opportunist thought lol but I am kind of desperate now when each mock test results a lower score than before, even with a higher accuracy rate )

This is what I really want to address. Your score isn't where you want it to be, you're not sure why, and so you're looking for anything (desperately!) that could help you "beat" the test. You've been studying hard and this whole thing just seems baffling and futile. Right?

I don't know for sure what the problem is, but I have a very good guess (because I've seen it so many times before): you are trying to study and approach this test in the ways that worked for you in school...but those ways aren't working on the GMAT. And this goes back to the underlying assumption in your brain that the goal is to try to get more questions right.

Read these:
http://tinyurl.com/executivereasoning
https://www.manhattanprep.com/gmat/blog ... -the-gmat/
http://www.beatthegmat.com/mba/2016/02/ ... n-the-gmat

Really think about them. Read them again. Then come back and tell me what you think.

Also look at this:
http://tinyurl.com/2ndlevelofgmat

Tell me how your approach so far does and does not match up with all of that stuff. And tell me any ideas you have for what you think you need to do differently. (I'm not talking about individual study sessions. I'm talking about overall approach / mindset.)

Wow! This is quite stunning. Knowing this "business mindset" certainly help me to be more comfortable with guessing a question and moving on, but for 700+ maybe I should still not guess too often but to work hard unless it is absolutely clueless question?
For my first GMAT I got

IR 6 Writing 5.9
Quan 49 77%
Verbal 34 72%
Total 690 86%

And now I just feel like I meet a bottleneck, I come to realize the "high score hard question" often doesn't test my knowledge anymore, but "tricks". Usually whether I could spot that hidden clue is the key to unlock right answer. Because every time I looked at the OA, it is always "OMG, that's the approach? I would never notice that, or I will never think of expanding out that equation." But thinking of these, the real world tricky problems are usually all beyond basics, and you have to approach it in a different way, smart way to solve it. Since these hard questions are hard because they deviate more from standardized methodologies, I feel it's useless to do more problems, because every hard problem could have its own trickery, which is not standardized. As a result, even if i learned the trick for this question, next one could be completely different.

As you pointed out in your article, "light bulb in my mind" is the key, but people act and solve problem in their life mostly based on standardized logic and methodologies, I am no Edison or Einstain who can have a completely new idea every day, some of the problems even take creativity to do I'd say. But to be honest, on my first GMAT, I rarely encountered a problem I had no idea what it was talking about, and I thought I got everything right actually, but it turned out not to be the case.

In general, 600-700 is an area where solid basics and normal-people-logic can achieve very easily, which can be obtained by standardized study (Or old school study), but 700+ is an area that requires one's creativity, smartness and well above average insightfulness, (Business mindset) which people cannot easily improve unless you have worked for more than 5 years and dealing difficult people and problems everyday (Or people who were born with these superiority).

Conclusion：maybe hard for me to get up 700+ in my next test, which will be in less 30 days.But I will try

[StaceyKoprince]

Even for 700+, you're going to get quite a number of questions wrong—more than you would think based on a "school" mindset.

You were at Q49 before; the top score is 51. Even when they're going for Q51, I advise my students to guess immediately / randomly on 4 questions throughout the section. (And I have had multiple students use that approach and still score Q51!)

If you're at V34 and trying to break 700, you'll want to try to increase this by 2-4 points (or more!). In that range, you can still guess quickly / randomly on ~6 questions in that section. It's only once you get above about V40 that I recommend decreasing to 4 questions.

I never recommend going below 4 random guesses—even when you are going for a top score in either section.

What you describe for quant is likely the case, because you are almost at the top; you can only get 2 more points on quant. You can study those approaches, but there's more opportunity on the verbal side at this point. I would make that your primary focus as you try to push forward. (You still need to practice quant, of course—need to keep your skills up!)

Finally, yes, it's hard to score 700+. Only 10% of all test takers reach that level—and the pool of test takers is already very superior! Everyone is a university graduate (or about to be) and wants to go to graduate school; these are smart people!

But...you already scored 690. You're almost there! Keep working hard and you just might get there!

[ZhenM102]

Hi, thanks for the reply
I just finished the second time, 680 as a result, 10 points lower...(devastating face)
So here is what I did after the first test.
1. Review all wrong questions on PrepGmat software and my notes
2. Redo all the 6 tests on PrepGmat (I bought the 2 exams packs from official website)

Before my first real gmat test, the average scores of 6 tests were 680 and I scored 690 on real test
Before my second real gmat test, the average scores of 6 tests were 730 (with one highest to 760) but I only scored 680 on the real test.

To be honest, I was really confident I could score over 700 the second time, since I took the Prepgmat tests as very accurate benchmark, but the result is just confusing. When I was doing the actual test, I felt really good about my verbal, yet, it still graded me a similar score as first one, 34. (My math didn't improve that much, since it is already approaching to 50 score, I figured, improving verbal has much more space than improving math, so I put my main focus on verbal.)

So here are my thoughts:
1. Since I redid the 6 tests, inevitably, I encountered repeated questions so it is more likely I picked the right answers. (Although I don't memorize answers and I feel I don't recall majority of the mock tests questions when redoing them. Maybe the mysterious subconsciousness is doing the bidding? )

2. The Prepgmat tests are easier than the actual tests (I also heard this from some other students that GMAC deliberately gave out low level difficulty questions to sell and put high level difficulty questions on real tests so more people will fail and take longer to pass the exams, meaning more people would buy materials from them so that GMAC able to make more moeny---Interesting conspiracy theory but one of the possibilities)

3. No matter what, there is no one to blame but myself in the end and to study harder is the only way. So I am planning on buying all the CATs exams, which should be better indicator than the easy 2 exam packs sold by GMAC.

Question:
I guess the third attempt at GMAT will be my last shot, so any suggestion on my study?

Thank you!

[ZhenM102]

I forgot another thing that I heard:

The GMAT test is not only adaptive but the score is also curved, which means not only does the computer is giving me hard time if I am trying to score higher, but other competing students are also the obstacles to higher scores.

For example,

If the raw score of mine is 720, but after taking into account that over 10% of students scored over 700+ for the past one month in this test center (or in the whole data base, not so sure how GMAC measures this but you get the idea.) the program will start curving: If it counts number of people over 740, and calculate that these people alone already constitute top 10%, then my raw 720 will be pushed lower than 700 (maybe to a 690 or 680) so that the 700+ could still be the same percentage.

If it is the case, I think it kind of explains why I did so well on my mock tests and feel so good about myself but still did a bad job on the real test: I just beat the machine, the program of adaptivity, but not other students, who did much better than me. Although I cannot verify this (Not knowing how many questions I got wrong for each section in the real test is annoying, and maybe it is why GMAC doesn't give you too much information, so that you wouldn't find out how ridiculously they curve the scores even given that you did fairly well on the real tests) I think it is truth, otherwise without curve, GMAC cannot maintain the percentage for each section that well solely based on the adaptivity.

My conclusion: I have to score 750+ as my average on mock tests in order to get a 700+ on real test.

Any thought?

[StaceyKoprince]

Starting with your last post first: Nope.

The scores correspond to a certain level of knowledge and that does not change. It never changes; it has been the same for decades.

What does change is the percentiles. 10 years ago, a score of 45 was somewhere around the 80th percentile. Today, that same score, which reflects the same ability level, is the 59th percentile. As the pool of test-takers changes, the percentiles change; they are updated every July. (It was also the case, 20+ years ago, that 730 or 740 was 99th percentile. Now, you have to hit 760 to get 99th percentile.)

Next: Going from a 680/690 to a 710/720/730 isn't about practicing hundreds and hundreds of questions. That's a school-test approach: if I just repeat this enough, I'll essentially "memorize" it and be able to do it on the test, too.

The real test isn't going to include essentially the same questions but just with different numbers or details or something. The real test is going to include similar-but-new questions that you still have to figure out how to think your way through. So that's what you need to study: what's the best approach "through" this question? If it's SC and there are these 4 differences among all of the answers, which ones are best for me to use and in what order? Why? Which ones should I ignore or only use if I have to because I'm not good at them?

What kinds of traps do I tend to fall for in SC? CR? RC? What can I do to learn to avoid them? (Note: traps on verbal fall into two broad categories: I was tempted to pick a wrong answer and I crossed off a right answer. You have to learn from both angles: how am I going to avoid falling for this type of tempting wrong answer and how am I going to avoid crossing off this right answer?) And so on.

It's also usually the case that *anyone* can pick up another 10 to 30 points simply by executing better on the test—which means letting go of too-hard questions so that you can be really careful on the points that you do know how to get. You minimize careless mistakes, and you give yourself a chance to answer just-slightly-hard questions because you haven't expended too much time and mental energy on questions that weren't worth it. It's like you're a world-class athlete trying to maximize your performance: sometimes, you let this point go because it's less important than the next one and you need to conserve your energy. So where / how can you get better just at execution?

[ZhenM102]

Hi Stacey,

Thanks for your advice, you are the one who gives me hope when I really need it

Interesting fact you said about the score and percentile, that means, with time goes by, people tend to score higher, so in order to be on higher percentile, you have to score higher. Just as you said, 730 was 99% 20 years ago, but now you have to get 760 to be 99%. Although the scores don't change, however, I can also observe higher GMAT requirement on top schools increase over the year; in the past the top B school average GMAT may only hover around 720, now many of them require 750+ (At least the average says so). I figure maybe the top schools don't necessarily want 700+, but they want the top percentile of group, the score is just a number, but they want the best of best. So I can predict in far future, with more and more people can reach 750+ more easily, the GMAT requirement for top B schools will go near 800. In a sense, it sort of confirms my theory that not only am I competing with the GMAT questions, but also other students who are taking them.

As for the study approach you mentioned, I think you are right. Doing more questions is no better than understanding one question thoroughly, so instead of breadth, I will focus more on depth. But when I need to practice the whole pacing thing and put myself "in the zone" before real test, I do need some complete test. Since I have done the official test packages for two times, I can't do it anymore (memorized most of answers), do you think the Manhattan CATs would be a good proxy to go?

Thanks!

[StaceyKoprince]

in the past the top B school average GMAT may only hover around 720, now many of them require 750+ (At least the average says so)

Not quite. In the past, the average hovered around 690/700. Now the average hovers around 720/730 for the very top schools.

And note the large difference between saying the average is 730 (or whatever number) and the schools require 730. If they required 730, then the average would have to be higher. If the average is 730, then by definition they're admitting lots of people who are below that average...that's what average means. So it is not the case that they require a 730 or a 750.

I'm pointing this out because, while it is important to do well on the GMAT in order to give yourself a competitive shot at the top schools, you don't have to score a 750+ or even a 730+. In fact, there are plenty of people who do score 750+ but don't get into the top schools because the schools are not interested only in your GMAT score. And there are plenty of people who get into those top schools with sub-730 scores. There are a lot of other factors involved.

I think that you are on to something when you describe why the averages have been increasing. But I also personally don't think that this average score inflation trend can (or should!) continue. If b-schools were to accept only people who score 750+ on the GMAT, they'd have a very narrow style of thinking in their classrooms. Yes, the GMAT is good at testing certain analytical skills, but those aren't the only skills that can lead to success in business! We'll see what happens, I guess.

I will answer your question about practice tests, but I am biased: part of my job is developing and maintaining our CATs! I do think that they are high quality and representative of the real test. Also, last year, we hired the former Chief Psychometrician of GMAC (aka, the guy in charge of the official test algorithm) to conduct a thorough review of our exams and tell us how we could make them better. Here's the outcome of that study:
https://www.manhattanprep.com/gmat/blog ... -endorsed/

In other words, I may be biased, but an independent expert also thinks our exams are high quality. So it's not just me.

In future, just remember not to do too many exams. As we talked about already, that's now how you really get better!

One other quant-specific resource: GMAT Focus. This is an official product, so go read about it at www.mba.com—but it's really good practice for the timing and decision-making on quant.