Mcmaster Medical School "chance Me": A Statistical Approach

[ralk]

 

 

This gets back to a major problem with your method: the Z-scores McMaster used to select invited applicants, and the means and standard deviations used in their calculation, had to exist before the invited applicants were selected. McMaster had to use different Z-scores than the ones you've calculated because the data used to calculate your's didn't exist when McMaster selected their invitees. The only thing they have in common is that they're both Z-scores. The interview data reflects the entire range of possible scores. This method interprets that range for the applicant using z-scores and provides an estimate of how competitive he/she is based on where him/her fall along that range. It doesn't have the precise SD/mean from the general applicant pool because it doesn't need it to answer the question of competitiveness! 

 

 

 

If you had claimed this from the beginning, that you simply used the results of applicants to infer competitiveness without taking into consideration how McMaster actually selects candidates, I would have had very little issue with what you've posted. Why? Because that's exactly what I did over a week ago. At that point you argued pretty strongly against such a method, so it's a tad odd to see you coming back with effectively the same model now, but fine, if it helps people figure out their chances, I'm all for it.

 

But no, you justified your model with this:

 

 

 

2) The knowledge that z-scores are used by McMaster's MD Admissions Office

 

What is a z-score? A z-score is a 'statistic' that describes your position relative to other peoples' on a distribution curve. 

For instance, a z-score of +1.64 means that you are in the top 5% of the data set, whereas a z-score of -1.64 means that you are in the bottom 5% of the data set. 

 

So? McMaster calculates a z-score for your performance on CASper, your MCAT Verbal Reasoning score and your Undergraduate GPA. These z-scores are then translated as an overall score on the 32% for each of these components. This means that the 32%-score that you receive for, say, your MCAT VR is dependant on how you fared relative to other people who applied in your cohort. 

 

What do you need to calculate a z-score? You need the mean of the data set and the standard deviation (SD), which is a description of how "dispersed" the data is. A relatively low SD reflects a narrow distribution whereas a high SD reflects a wider distribution. Using this information, a z-statistic can be calculated as follows:

 

Z= ("Your score" - "Mean value of that score")/"SD of that score"

 

Where is the standard deviation? Most medical schools don't make this data available and simply disclose a "mean" to the public. McMaster, however, happens to give us a distribution of the data. From this, we can estimate the Standard Deviation of the Verbal Reasoning and Undergraduate GPA components (from the link above). CASper information is unfortunately not available.

 

 

 

As I've demonstrated - and you've made no attempt to refute thus far - your calculated Z-scores have nothing to do with McMaster's Z-scores. Yet you give a long explanation about how your model is based off the fact that McMaster uses Z-scores. During our discussion last week, the one point I conceded was that numbers give the impression of authority when none exist. I edited my post to more explicitly reflect my lack of authority.

 

I will put that back on you now: your attempt to explain your model using the fact the McMaster uses Z-scores gives the impression that your Z-scores are estimations of McMaster's Z-scores and that you are therefore replicating McMaster's pre-interview selection process, when you are doing no such thing. Please make it clearer in your original postings that your Z-scores are in no way reflective of the Z-scores used in McMaster's process. Better yet, remove the mention of McMaster's Z-scores utilization entirely, as it has no bearing on your model.

[Organomegaly]

If you had claimed this from the beginning, that you simply used the results of applicants to infer competitiveness without taking into consideration how McMaster actually selects candidates, I would have had very little issue with what you've posted. Why? Because that's exactly what I did over a week ago. At that point you argued pretty strongly against such a method, so it's a tad odd to see you coming back with effectively the same model now, but fine, if it helps people figure out their chances, I'm all for it.

 

But no, you justified your model with this:

 

 

 

 

As I've demonstrated - and you've made no attempt to refute thus far - your calculated Z-scores have nothing to do with McMaster's Z-scores. Yet you give a long explanation about how your model is based off the fact that McMaster uses Z-scores. During our discussion last week, the one point I conceded was that numbers give the impression of authority when none exist. I edited my post to more explicitly reflect my lack of authority.

 

I will put that back on you now: your attempt to explain your model using the fact the McMaster uses Z-scores gives the impression that your Z-scores are estimations of McMaster's Z-scores and that you are therefore replicating McMaster's pre-interview selection process, when you are doing no such thing. Please make it clearer in your original postings that your Z-scores are in no way reflective of the Z-scores used in McMaster's process. Better yet, remove the mention of McMaster's Z-scores utilization entirely, as it has no bearing on your model.

 

My model used standard deviations using McMaster's data. It's true that these z-scores aren't the ones that would be used for their ranking of applicants, but they provide information that allow us to quantify the relative worth of VR and GPA and what combinations will produce good or poor chances. The reason I wrote this preface about the Z-scoring method is to inform people of why the Z-score method is relevant and superior to a linear one which does not take into account distributions. Certainly, however, I can edit my original post to reflect the conclusions we reached regarding the meaning of the Z-scores and speak more about why this method is more appropriate for the data we have. 

 

Edit: Within a reasonable extent, the SD of the data of interviewed applicants will be the same as the SD of the data of the general pool. So the SD's I calculated actually hold true for the data McMaster would be working with, and would still provide information about how much VR/GPA are worth relative to one another outside this interviewed data-set. 

[ralk]

My model used standard deviations using McMaster's data. It's true that these z-scores aren't the ones that would be used for their ranking of applicants, but they provide information that allow us to quantify the relative worth of VR and GPA and what combinations will produce good or poor chances. The reason I wrote this preface about the Z-scoring method is to inform people of why the Z-score method is relevant and superior to a linear one which does not take into account distributions. Certainly, however, I can edit my original post to reflect the conclusions we reached regarding the meaning of the Z-scores and speak more about why this method is more appropriate for the data we have. 

 

Edit: Within a reasonable extent, the SD of the data of interviewed applicants will be the same as the SD of the data of the general pool. So the SD's I calculated actually hold true for the data McMaster would be working with, and would still provide information about how much VR/GPA are worth relative to one another outside this interviewed data-set. 

 

1) You never mentioned a linear model until I did, especially not in your original post. And neither your original post, nor anything since, has provided any rationale as to why using a Z-score is superior to what I did. That makes me seriously question your explanation the your reason to explain that McMaster uses Z-scores was truly meant to highlight the differentiation between your model and mine. Still, I'll look forward to any changes you make that would clarify your model's underlying theory for me and anyone else reading. However, I'll mention again, since you keep claiming superiority, that our end results are pretty much identical.

 

2) You do realize that you've produced a linear model too... right? You haven't posted it, but your formula is Z-score = (VR - 10.99)/1.07 + (cGPA - 3.83)/0.146 - it's entirely linear. You used a different approach than I did to get there, Z-scores instead of my regression approach, but it's still a linear model.

 

3) Please justify the theory in your edit. On what grounds do you claim that the standard deviation from a biased subset of applicants is approximate to the standard deviation of the whole pool?

[fightoffyourdemons]

Sorry to bring up this super old thread, but I just wanted to add some thoughts/calculations that I've done on my own, and taking THT's work further, went on to calculate chances for actually being accepted vs. just interviewed.

 

I went through the interview invite/regrets threads for the past 2 cycles and this year's cycle, since they were all weighted the same way (32% GPA/32% VR/32% CASPer, 1% and 4% bonus for complete Masters or PhD), and inputted everyone's stats into Excel using z-scores. I included the Masters/PhD bonuses.

 

Assumptions: mean applicant GPA: 3.8, since everyone applies, low GPA or not.

                      SD: 0.15 based off literature from Mac publications

                      mean VR: 9.5, since mean VR is 8.1 and people with lower VR probably don't apply as frequently

                      SD: 1.3 again based off of literature from Mac publications

                      Mac takes the top 10% of applicants to interview, since it's generally been around that range (this year 552/5200+, etc.). Therefore you need a z-score total of >1.28 if you don't have a Masters, 1.225 if you do and 1.08 if you have a PhD).

 

Mean invited applicants GPA + MCAT z-score: 1.59 (median: 1.55, range: -1.39 to 4.03)

Mean CASPer needed: -0.31 (median: -0.27, range: -2.75 to 2.67)

 

Mean rejected applicants GPA + MCAT z-score: 0.51 (median: 0.7, range: -3.15 to 3.89)

Mean CASPer needed: 0.76 (median: 0.56, range: -2.61 to 4.43)

 

This sort of makes sense, given the skew towards higher GPA and VR on this forum.

 

After the interview is done, the formula switches to 15% GPA, 15% VR and 70% MMI. To calculate z-scores here, I used the average GPA and MCAT over the last two years (available on Mac's website),

 

GPA 3.825 (SD 0.1) and VR 11.03 (SD 1.16)

 

Assuming that Mac takes the top 40% of interviewees, you would need an overall z-score of >0.25 to be in that top 40%. Average MMI score is approximately 55% according to the JAMA article published by Mac, with an SD of 10.2 (they have 12 stations in their paper so I scaled it down to 10 stations). 

 

Mean accepted GPA + MCAT z-score: 0.37*0.3 = 0.11 scaled (median: 0.51, 0.15 scaled)

Mean MMI z-score needed: (0.25 - 0.11)/0.7 = 0.2 scaled (median: 0.14 scaled, need to be in top 42% of interviewees)

 

Rejected GPA + MCAT z-score: 0.18*0.3 = 0.054 scaled (median: 0.36, 0.11 scaled)

Average MMI z-score needed: (0.25 - 0.054)/0.7 = 0.28 scaled (median: 0.2 scaled, need to be in top 39% of interviewees)

 

Thus, it looks like the interview doesn't differentiate people that well, at least according to the stats posted by people in this forum. The margin of difference between the interview scores barely exists, which I guess works for Mac because they give everyone a good shot at the interview stage. However, getting that first invite is greatly aided by a high GPA and VR score.

[Organomegaly]

Sorry to bring up this super old thread, but I just wanted to add some thoughts/calculations that I've done on my own, and taking THT's work further, went on to calculate chances for actually being accepted vs. just interviewed.

 

I went through the interview invite/regrets threads for the past 2 cycles and this year's cycle, since they were all weighted the same way (32% GPA/32% VR/32% CASPer, 1% and 4% bonus for complete Masters or PhD), and inputted everyone's stats into Excel using z-scores. I included the Masters/PhD bonuses.

 

Assumptions: mean applicant GPA: 3.8, since everyone applies, low GPA or not.

                      SD: 0.15 based off literature from Mac publications

                      mean VR: 9.5, since mean VR is 8.1 and people with lower VR probably don't apply as frequently

                      SD: 1.3 again based off of literature from Mac publications

                      Mac takes the top 10% of applicants to interview, since it's generally been around that range (this year 552/5200+, etc.). Therefore you need a z-score total of >1.28 if you don't have a Masters, 1.225 if you do and 1.08 if you have a PhD).

 

Invited applicants GPA + MCAT z-score: 1.59

CASPer needed on average: -0.31

Rejected applicants GPA + MCAT z-score: 0.51

CASPer needed on average: 0.76

 

This sort of makes sense, given the skew towards higher GPA and VR on this forum.

 

After the interview is done, the formula switches to 15% GPA, 15% VR and 70% MMI. To calculate z-scores here, I used the average GPA and MCAT over the last two years (available on Mac's website),

 

GPA 3.825 (SD 0.1) and VR 11.03 (SD 1.16)

 

Assuming that Mac takes the top 40% of interviewees, you would need an overall z-score of 0.25 to be in that top 40%. Average MMI score is approximately 55% according to the JAMA article published by Mac, with an SD of 10.2 (they have 12 stations in their paper so I scaled it down to 10 stations). 

 

Accepted GPA + MCAT z-score: 0.37*0.3 = 0.11 scaled

Average MMI z-core needed: (0.25 - 0.11)/0.7 = 0.2 scaled (need to be in top 42% of interviewees)

 

Rejected GPA + MCAT z-score: 0.18*0.3 = 0.054 scaled

Average MMI z-core needed: (0.25 - 0.054)/0.7 = 0.28 scaled (need to be in top 39% of interviewees)

 

Thus, it looks like the interview doesn't differentiate people that well, at least according to the stats posted by people in this forum. The margin of difference between the interview scores barely exists, which I guess works for Mac because they give everyone a good shot at the interview stage. However, getting that first invite is greatly aided by a high GPA and VR score.

 

"The average scores the 117 candidates received across 10 MMI stations ranged from 3.2 to 6.55, with a mean of 5.02 (standard deviation 1⁄4 1.46). "

 

From the paper. In the paper, the scale was originally actually out of 7 points. That's why the mean score sounds like it is 50% but actually it is 5/7. 

 

As far as the interview not differentiating people, I'd say with a mean of 5 and a standard deviation of 1.5 that there is still quite a bit of room to tell people apart there. Especially when it is worth 70% of the score. I don't see a restriction of range issue? 

[fightoffyourdemons]

I grabbed the MMI scores from this paper : http://www.ncbi.nlm.nih.gov/pubmed/23212501, Table 2.

 

EDIT: Also to note, the study I quoted used the MMI as part of the admissions process, but I think the original paper in Med Educ was the year right before it was formally implemented.

[Organomegaly]

I grabbed the MMI scores from this paper : http://www.ncbi.nlm.nih.gov/pubmed/23212501, Table 2.

 

EDIT: Also to note, the study I quoted used the MMI as part of the admissions process, but I think the original paper in Med Educ was the year right before it was formally implemented.

 

Oh ok sorry I see now - I assumed it was from the paper in the invitation package

 

That's actually interesting that the average rejected score is 5/10 and accepted is 6/10. You're right - it does seem like a very small margin! 

[bearded frog]

Uh, you did a bunch of math using made up assumption data and your conclusion was "A high GPA and verbal score helps to get accepted to Mac..."

 

Real groundbreaking work there!

[fightoffyourdemons]

I admit that the pre-interview assumptions are less sound than the post-interview ones, but I don't see why those would be wrong considering that they DO take the top 40% of interviewees. It's not hard to find the z-score given that they give the actual accepted data on their website. The whole problem that was raised against THT's calculation was that it was based on Mac's data on accepted candidates, which I've tried to modify here using real data. Unless people lied about their GPA/VR, it should be pretty accurate. 

 

But I also admit that yes, my conclusions are pretty useless think of it as someone who wanted to numbers to reassure them on how well they have to interview.

 

Uh, you did a bunch of math using made up assumption data and your conclusion was "A high GPA and verbal score helps to get accepted to Mac..."

 

Real groundbreaking work there!

[jlkhhylyiluh6]

Uh, you did a bunch of math using made up assumption data and your conclusion was "A high GPA and verbal score helps to get accepted to Mac..."

 

Real groundbreaking work there!

 

lol, for sure.

 

It's not that hard guys. Get good grades (3.85 or higher), get a good VR (11 or higher), and do ok on Casper. Don't worry about splitting hairs with numbers.

[MrSchwartz3]

I mean I got a 10 in verbal and I got an interview...

[GrumpyMoriarty]

So while I do like crunching numbers as an alternative to recounting things I should/should not have said during MMI, I have a few problems with the current "chance me" models under proposition.

 

1) Where does it actually say that McMaster uses z-score? I tried googling "McMaster Med" and "z-score" and the only significant search result I got was from this forum? Is there some "insider info" that I am unaware of?

 

2) Doesn't z-score assume a normal distribution? While I can see that as the case for VR (based on interviewee data), the cGPA data looks to have abit of a left skew... in which case I would have thought that calculating z-score isn't the best statistical method -- would probably use median and IQR in that case.

3) I have been scouring past posts of acceptance/waitlist/rejection for MacMed, and I notice that there was quite a few people who had a z-score of >0.20 before factoring in MMI getting completely rejected, with many of them saying that they thought they performed well on MMI. So does that mean most of them have a really flawed notion of how well they performed (since I think they need to be in like the bottom 30-40% to be out right rejected)?

[fightoffyourdemons]

1) It's from this article: http://www.ncbi.nlm.nih.gov/pubmed/15141133

 

2) It is left-skewed from what you see when you plot out all the data on a graph

 

3) There's also a lot of people who have to score pretty high on the interview to get in who DID end up getting in... so I have no idea how interview scores are distributed =/

 

So while I do like crunching numbers as an alternative to recounting things I should/should not have said during MMI, I have a few problems with the current "chance me" models under proposition.

 

1) Where does it actually say that McMaster uses z-score? I tried googling "McMaster Med" and "z-score" and the only significant search result I got was from this forum? Is there some "insider info" that I am unaware of?

 

2) Doesn't z-score assume a normal distribution? While I can see that as the case for VR (based on interviewee data), the cGPA data looks to have abit of a left skew... in which case I would have thought that calculating z-score isn't the best statistical method -- would probably use median and IQR in that case.

3) I have been scouring past posts of acceptance/waitlist/rejection for MacMed, and I notice that there was quite a few people who had a z-score of >0.20 before factoring in MMI getting completely rejected, with many of them saying that they thought they performed well on MMI. So does that mean most of them have a really flawed notion of how well they performed (since I think they need to be in like the bottom 30-40% to be out right rejected)?

[GrumpyMoriarty]

1) It's from this article: http://www.ncbi.nlm.nih.gov/pubmed/15141133

 

2) It is left-skewed from what you see when you plot out all the data on a graph

 

3) There's also a lot of people who have to score pretty high on the interview to get in who DID end up getting in... so I have no idea how interview scores are distributed =/

 

Okay so I gather that Mac at one point suggested the possibility of standardizing the score distributions (which makes sense). But I have to point out that there is no direct evidence that the admissions team actually used z-score and not some sort of more fancy stats with higher accuracy.

 

Furthermore, I think that trying to extrapolate percentages from the calculated z-score is not entirely accurate since the back-conversion assumes that the final scores are in a Gaussian distribution, which may not be the case there is a skew in the final distribution. It is far more likely, in my humble opinion, that the MD admissions team would calculate the final z-score for each candidate and rank this score in descending order, offering admissions to the first 203/552 (36.7%) and subsequently to those on the wait-list.

 

Also with regards to skew, did a quick bit of stats (instead of studying for final exams) to find that while the average cGPA for interviewees from PM101 is 3.83, there is indeed a fairly significant left skew as the median cGPA is actually 3.86, with the interquartile range being 3.75-3.95.

[Organomegaly]

 So does that mean most of them have a really flawed notion of how well they performed (since I think they need to be in like the bottom 30-40% to be out right rejected)?

 

http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect

 

Peoples' perceptions of how they did is useless. Don't bother seeking the information. 

[DarkKnight]

So this is probably pointless (as it isn't representative of everyone), but I data-mined this years replies and compiled it just to see what it looks like.

 

Someone said the first round of replies to the offer of acceptance should be made by May 26, so the first-round should come around then.

 

#Accepted 31    

3.88 GPA  

11.00 VR

# declined 8.5  (0.5 for undecided)

 

#Waitlist 11

GPA   3.77

VR   11.55  

 

# regrets 4    

3.70 GPA   

11.50 VR

 

% accepted                    67.4

% declined of accepted  27.4  

%waitlisted                     23.9 

% regrets                       8.7

[GrumpyMoriarty]

The percentage figures are not very valid nor useful. Its not by any stretch of imagination to assume that less people would be willing to post regrets and more people will be willing to post acceptance. I guess by extension we cannot be sure if we have a representative sample even amongst accepted/waitlisted people either. But from what I gather, there seems to be a general trend of higher GPA in accepted people.

 

 

% accepted                    67.4

% declined of accepted  27.4  

%waitlisted                     23.9 

% regrets                       8.7

[Organomegaly]

The percentage figures are not very valid nor useful. Its not by any stretch of imagination to assume that less people would be willing to post regrets and more people will be willing to post acceptance. I guess by extension we cannot be sure if we have a representative sample even amongst accepted/waitlisted people either. But from what I gather, there seems to be a general trend of higher GPA in accepted people.

 

It should be pretty easy to presume this based on the fact GPA is worth 15% of the selection formula. GPA definitely didn't decide anyone's admission result, but combined with VR it would slightly modify how "good' or "bad" your MMI score must/can be to receive admission. Most of the variance will come from the MMI though. 

[sdj78we9jtf2o3mgfvj298j]

In case anyone wanted the new numbers for last year's admissions, they were:

 

GPA

Average: 3.843

STDEV: 0.131

 

Verbal

Average: 11.21

STDEV: 1.127

 

I used the same approach as Turkey. I took the middle of the range for each score and this proved to be accurate since I got approximately the same average as the official statistics.

[Maverick]

I posted in the wrong thread, sorry.

[Eliza123]

How likely would it be for me to apply with a cGPA of 3.04. I haven't written CARS yet, but do I stand a chance of getting accepted? Has anyone heard of someone getting accepted with an average that low?

Please be honest. I really want to go to med school and have made a lot of mistakes in my undergrad which I deeply regret, but am trying to pick myself up and the proper extracurricular experience at least. Does MAC look at ECs as much as they do with GPA? Cuz I've done a couple of years of research, no publications though. I'm going in to my 5th year now.

[greensurR]

How likely would it be for me to apply with a cGPA of 3.04. I haven't written CARS yet, but do I stand a chance of getting accepted? Has anyone heard of someone getting accepted with an average that low?

Please be honest. I really want to go to med school and have made a lot of mistakes in my undergrad which I deeply regret, but am trying to pick myself up and the proper extracurricular experience at least. Does MAC look at ECs as much as they do with GPA? Cuz I've done a couple of years of research, no publications though. I'm going in to my 5th year now.

 

You can see class statistics for the last 3 years here: http://mdprogram.mcmaster.ca/md-program-admissions/how-we-select. Every year it seems they accept a few people with a 3.00-3.49 GPA, but I imagine you probably need a pretty good CARS. Mac does not look at ECs, although it's good to have experiences you can mention on CASPer (potentially) and in the interview. You can see their selection formulae on this page http://mdprogram.mcmaster.ca/md-program-admissions/how-we-select/selecting-our-students

[hollyjolly]

How likely would it be for me to apply with a cGPA of 3.04. I haven't written CARS yet, but do I stand a chance of getting accepted? Has anyone heard of someone getting accepted with an average that low?

 

I just got accepted with GPA of 3.30 and CARS 130 if that's any comfort... but us low-GPAers are a pretty rare breed according to the stats!

 

 

Does MAC look at ECs as much as they do with GPA? Cuz I've done a couple of years of research, no publications though. I'm going in to my 5th year now.

 

I spoke with Mac about this directly and they were clear they do not look at any of your application until after interviews. The invite to interview is a mathematical formula, and that's all there is to it. Doesn't matter how special you might be, they get 5000+ applications and so it makes sense to me that they can't read them all. After interviews they review the full file, mostly looking for red flags to rule out admission, before assigning the final ranking.

 

So... as brutal as it is, getting the highest possible CARS + Casper is all you can do! I would suggest lots and lots and lots of prep.

[Eliza123]

.