How closely related is the MCAT scores to.......

[Guest huh]

How closely related is the MCAT scores to performance in Med School

[Guest not rex morgan]

Not sure of the point of this question, but I'll give you a short answer, anyways. I don't think the MCAT tests much of anything they are concerned with testing. It's all about who can speed read. The writing section is a joke. They want you to basically undo the decent style you've built up over the years and format your style to fit their horrible set up. With that said, I have no idea how the score reflects your performance in med school and I don't really understand why you're asking. But given the fact that I don't believe it's an accurate test of anything, I don't think it reflects alot about med school. In my experience, it has just been used for extrememly competitive premedders to clash heads.

[Guest huh2]

Hmm..

 

I believe there is substantial evidence that the MCAT *IS* in fact a better predictor of medical school performance than undergraduate GPA (uGPA).

 

Several studies are available from the AAMC MCAT website (perhaps a bit of bias though??) But regardless, there is some evidence for this:

 

Here's the link:

 

www.aamc.org/students/mca.../start.htm

 

And here is part of an abstract from one of the more recent studies:

 

Veloski, J.J., Callahan, C.A., Xu, G., Hojat, M., & Nash, D.B. (2000). Prediction of Students' Performances on Licensing Examinations Using Age, Race, Sex, Undergraduate GPA, and MCAT Scores. Academic Medicine, 75, S28-S30.

 

CONCLUSION: As expected from many earlier studies, MCAT scores were consistently more valuable than were undergraduate GPAs as predictors of performance on licensing examinations, supporting their continued use in selection decisions. These relationships are stable across three decades and apply to the three examinations. Verbal scores tended to be better indicators of performances in the clinical and postgraduate tests. There was no independent effect for older, nontraditional students after controlling for their undergraduate academic performances and MCAT scores.

 

Have a good new year all.

[Guest Ian Wong]

Heya huh2,

 

I took a look at the above website, and it seems like an awful lot of those journal articles were published in the journal <!--EZCODE ITALIC START--> Academic Medicine<!--EZCODE ITALIC END-->, or were not published but rather just presented at conferences. My PBL background (and yes, this is somewhat tongue in cheek) has taught me to be skeptical of any information that only seems to be able to make it into print in just a small number of journals. It only takes one journal with a political or organizational bent to publish a lot of stuff into worldwide literature. As you've alluded, the MCAT folks have an awful lot to gain by ensuring that med schools continue to use the MCAT. Each year, some 45,000 Americans apply to med school, many having taken the MCAT multiple times. That's an awful lot of MCAT test fees!

 

In any event, I can only comment on my experiences in med school with my own class, and here, I can't see that MCAT scores have much to do with clinical performance when it comes to extracting vital information from your patient interviews, or hearing that fine heart murmur, or subjectively being able to feel that your patients ACL is lax when attempting to stretch it. The people in the class that I have discussed the MCAT with haven't shown, to my eyes, any correlation with their clinical skill.

 

MCAT scores, in my opinion, are only as good as the time and products you expend to prepare for them, in addition to one's previous science background. Similarly, the licensing exams that you speak of are much like the MCAT's in that they require a certain style of preparation. However, sitting down and writing in answers to a multiple choice exam simulates very little about an actual clinical encounter, and so I wonder, to a degree, why people hold the licensing exams to such a high level. For my fourth year classmates, it's just another "stupid exam", with a 93% (or thereabouts) national pass rate.

 

I suppose the licensing exams can be touted as an objective measurement for all medical students across North America, much as the MCAT is for premed students, but that doesn't mean that they are great predictors of how you function clinically. I don't think that any multiple choice exam with bubble sheets could really be expected to simulate all the variables in each doctor-patient encounter. It's the best we've got now, but that doesn't mean it's all that good.

 

Ian

UBC, Med 3

[Guest BennyG]

Hi Ian (and Mr. huh2),

 

I suspect that it is quite likely that MCAT performance does not correlate with clinical skill. However, the same thing can probably be said for GPA or any other academic measure one wants to apply. The ultimate question should be "is the MCAT a better way to assess an applicant's academic abilities than the GPA?" If (BIG IF) it can be objectively demonstrated that MCAT scores display a stronger correlation with the academic component of medical school performance than undergraduate GPA, then why not make MCAT a larger portion of the admissions process? After all, a school wants its students to succeed, right?

 

I assume that admissions committees look at academic performance for two fundamental reasons: (1) they want to know if you are intelligent/dedicated/organized enough to handle the training and (2) they want to know that you can handle the intellectual demands of being a doctor. Is the MCAT better at assessing your brain-power than any other test we wrote in undergrad on alkene reactions, population ecology or newtonian mechanics?

 

For those of us who took life sciences, think back to what most of your courses were like. The topics also had nothing to do with medicine, and many exams were also multiple choice and written under time constraints. And one ALWAYS has to study "a certain way" for each type of course or test; that is no different from the MCAT. The one thing I can say in defence of the MCAT is that it is standardized. All 45,000 of us around the world write the same test (of course, with some shuffling of Qs and different test forms) at the same time. The gaussian statistics don't fluctuate that much, the raw scores get adjusted to account for variations in test form, etc. It is designed to be a consistent, standardized, academic test. On those criteria, it suceeds. Of course, the AAMC would like to profess that the MCAT predicts great doctors, but, as we discussed before, they are obviously not objective considering the profits they derive from it.

 

When you compare two people with different scores, you can be clear about what the difference means. Student A performed better academically than Student B. Period. This is not always the case when you look at someone's GPA. What kind of courses did the student take? Advanced nuclear physics or a whole bunch of intro life sci courses to keep the GPA high? What kind of program was he/she in, and what university did they attend? Was it even humanly possible to get 90+ in the course? If there is so much variation in the type and difficulty of the undergraduate programs of the applicants, why are we rejecting students over a 0.1 or 0.2 difference in GPA?

 

I'm not trying to be some big MCAT-pusher. I disliked writing the test as much as anyone else. I also found it to be yet another test I had to do along the way to meds, another one of the many hoops we jump through for admission. I spent a lot of time studying that could have been spent on other things that help my med school admission chances. But at least, at the very least, I felt like I had a equal chance against everyone else, and that my mark represented my real academic performance compared to the other examinees, no matter how artificial and non-applicable to medicine.

 

Happy holidays and new year to everyone,

 

piece (of pie)

[Guest not rex morgan]

What is not included in the statistics is the med school performance of bright people who didn't make it in b/c of their low MCATs. (as they have no med school performance to speak of). Some people just don't do well on standardized tests, or can't shell out the Kaplan money. By the way, it's been a while since my stats course, how different does a graph with random distribution look from a "Gaussian" graph? I'm assuming Gaussian is representative of a normal distribution.

[Guest BennyG]

Hi not rex morgan,

 

You make a good point - the sample is biased because only the "data" with high MCAT AND high GPA is included in the statistical analysis. How many people there are with high GPAs and low MCAT ("low" meaning not good enough to get accepted, so let's say, depending on the university, between eight and ten) is an interesting question and for the sake of objectivity should probably be examined by the same people who produce the pro-MCAT material. If the number of people in this category is sufficiently high, I doubt it can be explained by just writing off those students as people who weren't that smart and just took bird courses. Not only would that be unfair, it wouldn't be true.

 

There are two possible scenarios. One - the GPA statistics of the MCAT-writing population approximate the GPA statistics of the undergrad life sciences population (which, despite exceptions, still comprise the vast majority of the MCAT writers). Or, the big OR - most of the MCAT examinees are high GPA achievers from undergrad, which means that there are a good number of people who are above, say, the 75th percentile in their undergrad classes but who can't break the 50th percentile of the MCAT sections (around eight or nine). Very interesting indeed, especially since the MCAT is based on material that students have already been tested on in school. So maybe the MCAT is just designed to be plain harder so that it can knock some people down and produce the much-desired bell curve distribution. The more I think about it, there are still many studies and statistical analyses that should be performed on the MCAT.

 

As for the use of prep courses (Kaplan, for example), the all-seeing and all-knowing AAMC states in its examinee handbook that the score differences between people who took a prep course and people who didn't was minor and likely attributable to just having a more structured study schedule. From my memory (I took the April 2001), I didn't remember seeing the words "statistically not-significant (p < 0.05)" beside the word "minor". There is no reason to trust the AAMC statement - it would kill their reputation if it could be demonstrated objectively that prep courses improve student scores. From my anecdotal experience, I found a prep course very useful, not so much for the information (although the instructor was very good) but for the practice material. However, in my class at least, the people who took the class after a poor performance on the MCAT the first time did not necessarily improve. Don't forget, the prep course companies (Kaplan, Princeton, etc.) also use propaganda potentially as f.o.s. as the AAMC. Kaplan says that "2/3 of our MCAT students get into med school" and other such stuff. And yeah, they are damn expensive. Everyone makes money off this stuff, man, everyone.

 

Re: the stats, yeah gaussian, normal, same thing, it's that bell curve. BTW, I looked at the graphs for the April MCAT and they are mostly bell-like (although a histogram can sometimes be misleading) except for the writing sample, which has two peaks, one near M and one near Q.

 

happy new year!

[Guest not rex morgan]

Hey Benny. I commend your objectivity. I like to see lateral thinking. Thanks for clearing up the stats. Just one comment. I don't know if I'd call the MCAT "harder" than uni courses. I think the depth required for the material is substantially less than undergrad courses. It's quite the bredth of material to cram into one day. Plus, the testing style isn't for everyone. I'm a slow reader, but always did well in English in university. I think anyone would be hard-pressed to find a professor who would deem the MCAT English harder than undergrad English. Verbal reasoning was my worst section, and my writing samples weren't so hot either. I guess I couldn't bastardise my writing style enough for them. In any case, from my experience, I don't really value the MCAT, and I don't believe it represents my knowledge. It's a little old-boys-club in my view. However, some people found it a valuable, challenging experience. Happy New Years Folks.