How Sex, Race and Socio-economic Status impacts MMI performance:

[Aristotle]

Sex

Seven papers investigated the effect of sex on MMI score. Four papers examined medical school admissions; two, nursing school admissions; and one, pharmacy admissions. Two papers identified a significant impact of sex on MMI score, while the other four did not.

Leduc et al (2017) found that female students performed significantly better in MMI’s for three French-Canadian medical schools than their male counterparts, but only in the subgroup that spoke French at home (p=0.003). This implies a previously unexplored interaction between sex and language, which warrants further study.

Ross et al (2017) examined the MMI scores of 526 medical applicants at the University of Calgary. It found that female students achieved a higher score than their male counterparts (p<0.001), even when they controlled for confounding variables (e.g. MCAT score, applicant age etc). Two interpretations for this difference are proposed. Firstly, that female applicants are more likely to possess the attributes of interest in the MMI assessment (communication skills, critical thinking, ethical decision making etc) which, if true, this would support the validity of the MMI process. The other is that the interviewers are victims of confirmation bias, in that they expect female applicants to possess these skills, and therefore mark them more highly. The authors of this study recommend a longitudinal, health outcome based approach to investigate this further. None of the other studies included in this analysis showed any significant effect of sex on MMI scores (Traynor et al. 2017; Kim et al. 2017; Pau et al. 2013; Gale et al. 2016; Cox et al. 2015).

Race

Four studies investigated the impact of race on MMI scores; three on medical school applications, and one on pharmacy (Leduc et al. 2017; Pau et al. 2016; Cox et al. 2015; Jerant et al. 2015). One of these studies reported a significant impact. Leduc et al (Leduc et al. 2017) found that applicants who self-declared as “Chinese” or “Southeast Asian” performed significantly worse than all other ethnic groups (p=0.013, 0.003 respectively). It also found that applicants who identified as “White/Caucasian” performed significantly better than other ethnic groups (p=0.012). In a qualitative investigation of the potential reasons behind this, it was reported that many of the interviewers had come across a stereotype that Asian students are “shy”, and that this may have impacted on the scores given through a confirmation bias. The other three papers reported no significant impact.

Socio-economic Status

Three papers analysed the impact of socio-economic background (SEB) on MMI scores (Leduc et al. 2017; Taylor et al. 2015; Jerant et al. 2015).

Leduc et al. used a self-assessment form to categorise applicants into those with a family income of either <$100,000, $100,000 to $250,000, or >$250,000, and compared the MMI scores for applicants within these categories. It found that students who declared a parental income of >C$250,000 had significantly higher MMI scores (M=252.8) than those in the C$100,000 to C$250,000 (M=243.3, p=0.035) and <C$100,000 (M=237.1, p<0.001) groups. This study is limited by the fact that it relies on the applicants’ self-assessment of family income, but nonetheless draws into question the validity of the process.

Jerant et al. (Jerant et al. 2015) examined eight different factors, including parents’ education level, whether the applicant had contributed to family income, and whether the applicant was in receipt of income support, to create a composite score for SEB. This was applied to 1,420 MMI interviewees at the University of California. They found that candidates from a more deprived background had a significantly lower MMI score (p=0.03), but that this was only a small a small impact of less than 0.2 S.D. However, being from a more disadvantaged background was associated with a greater probability of being accepted onto the course (OR 3.28, p>0.001), since the final decision was made on a wide range of factors beyond MMI score, including personal statements and MCAT scores. This shows that biases within the MMI system can be compensated for by the admissions process as a whole.

Taylor et al. (Taylor et al. 2015) found no significant relationship between the MMI score achieved by an applicant and the status of the applicant’s school (selective/fee paying vs non-selective and non-fee paying). Nor was there a relationship between MMI score and Higher Education participation rates in the area in which the applicant lived.

 

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Also: "some universities may have financial conflicts of interest because both CASPer and MMI have now turned into revenue generating corporations for their creators and possibly their affiliated universities"

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

I believe that there should be a discussion about the inherent biases of MMI and Casper!