Did I Just Kill My Application?!

[docmm]

I do agree with you in that we can't jump to any conclusions - after all we are just sharing our personal opinions here and have no facts to share.

 

However, I assure you that if CASPer was such an excellent and validated tool, more universities would have adopted it, similar to the MMI. CASPer is definitely not as flawless as the MMI, and as you made it sound like. 

 

It's only a few years old - NOSM is piloting it this year. It's still too early too tell how widely it will catch on. I believe the MMI was created in 2002 (I might be remembering this wrong) and some schools only recently began to adopt it or some variation of it (ie. UofT).

 

Regarding your #4, do you really think all applicants can show the skills they gained throughout these years by typing an answer in 1 minute? 

 

The CASPer is testing primarily your written communication, critical thinking (on-the-fly) skills so yes - I do think applicants who do well on the test demonstrate the level of proficiency in theses areas that they've attained through their experiences. With careful reflection surrounding your past experiences before the test you should be able to have a concise idea of how you demonstrated particular CANMEDs roles and how particular experiences have impacted you. You won't be able to fully explain this but all applicants have 1 minute so it's all relative. With the questions you fumble (which is understandable), there's the hope that they really do drop 1-2.

 

No matter what, being a fast typer helps (yes, not a requirement, but you can't say it doesn't help) If it takes me 10 seconds to type what takes you 30 seconds, then I have 20 more seconds to think?

 

And as a side note, when you say there is no evidence to suggest practicing for CASPer has any real correlation with performance, do you mean this has been looked into and no evidence was found? How can you expect validity and evidence regarding something that is being secretly done. Just like the MCAT, do you think doing 20 full lengths wouldn't help you? 

 

I guess what I meant is that no one can really predict what will be on CASPer year to year. The best thing test companies can do is offer applicants the ability to practice their timing and practice sample "MMI" questions". There are many available online for free. You can easily do both of these things on your own without paying any money. The feedback component you can get for free from friends, medical student friends/family, and even then, it might not be useful as they are not the ones marking it (and neither are the companies).

 

I think MCAT + interview would do enough to applicants that there is no need for any schools to display their creativity.

 

We should be happy that the admissions is dynamic to prevent people from gaming the system as much as possible and to hopefully overtime find a system that is effective at recruiting student physician to meet the demands of the country. While changes and experiments can be frustrating, I think every schools intention is good. 

 

[Organomegaly]

Can't find it right now, but there are results posted in a presentation (by the royal college?) showing that the CASper was correlated around 0.5 with MMI success (which is strong correlation) and had inter-rater reliability of 0.7-0.8, which is substantial agreement. 

 

It's more reliable/valid than you might think. And 4 years isn't enough to see if it will be used in the future. McMaster also piloted MMI, and that took a while to catch on too. 

[rmorelan]

Can't find it right now, but there are results posted in a presentation (by the royal college?) showing that the CASper was correlated around 0.5 with MMI success (which is strong correlation) and had inter-rater reliability of 0.7-0.8, which is substantial agreement. 

 

It's more reliable/valid than you might think. And 4 years isn't enough to see if it will be used in the future. McMaster also piloted MMI, and that took a while to catch on too. 

 

exactly - this is a new tool, in a rather conservative endeavour for medical schools. Nothing changes all that quickly here so the fact that CASper is not used everywhere is not really surprising - even if it is great, it is still too new. The MMI has taken off quite quickly considering the usual rate of change and that has still taken a long time to move around. 

 

People don't like messing with something they know already works pretty well.

[Organomegaly]

found it:

 

http://www.royalcollege.ca/portal/page/portal/rc/common/documents/events/icre/2013proceedings/slides/Admissions_Selecting_Residents/K_Dore.pdf

[Almighty Saguaro]

Yeah except you'd never have to make knee jerk reactions like that in real life, let alone in medicine. Also don't ask me how to achieve world peace and give me 15 seconds to answer it, this isn't the miss USA competition

Exactly! Let's test the aplicant's ability to independently assess an ethical dilemma in mere minutes and come to a hasty conclusion. This will help us select future doctors who will be good at interacting with colleagues and patients, carefullly weighing the issues at hand to achieve some sort of compromise on ethical dilemmas...huh?

 

I can appreciate their intentions in instituting the test but I think its ecological validity is questionable.

[desperate_premed]

Exactly! Let's test the aplicant's ability to independently assess an ethical dilemma in mere minutes and come to a hasty conclusion. This will help us select future doctors who will be good at interacting with colleagues and patients, carefullly weighing the issues at hand to achieve some sort of compromise on ethical dilemmas...huh?

 

I can appreciate their intentions in instituting the test but I think its ecological validity is questionable.

 

Apparently as long as it filters some students out and "research" has shown a correlation, it is acceptable lol.

 

I swear 5 years from now, schools will have multiple interviews (like in different stages). I'm sure a study can be done to back that up!

[Almighty Saguaro]

Apparently as long as it filters some students out and "research" has shown a correlation, it is acceptable lol.

 

I swear 5 years from now, schools will have multiple interviews (like in different stages). I'm sure a study can be done to back that up!

To be fair to the schools, I suppose one one should consider that they have to balance trying to give people a fair chance to demonstrate the qualities their looking for, with using limited resources efficiently.

 

If we take interview to be the gold standard (though I would argue that there are still major issues with this form of assessment), technically, the best strategy might be to review every applicant's entire OMSAS submission, perhaps consider all MCAT marks, and subsequently interview every applicant...but this would, of course, require an inordinate amount of resources and would likely make it harder to weed out applicants. All other considerations aside, this is ultimately the endpoint of the admissions process. 

[Organomegaly]

Apparently as long as it filters some students out and "research" has shown a correlation, it is acceptable lol.

 

I swear 5 years from now, schools will have multiple interviews (like in different stages). I'm sure a study can be done to back that up!

 

Why the quotations around research? Evidence is the only thing we have in using our limited resources to make meaningful choices, given that arguments can be made for and against literally any component of the admissions process. 

[Savac]

Your academic performance and ABS are all done/maintained over many years and give your application a character. I think they should weight a lot more than a test like CASPer and even to some degree the MCAT. I think it's fair to say that the more money you are willing to spend to prepare for these tests, the higher your chances of getting a good mark. Consider someone who hasn't done much clinical work, never interacted with any patients, never shadowed a physician, and haven't done any research and now this person spends a thousand bucks purchasing 12 mock Casper tests, sits down with McMaster students, practices timing, spends an entire month making flashcards with every personal scenario, etc etc. This person will have a higher than average chance of doing well on this exam. Now can you say this person is actually "better" than the ones who score lower? Same can be said with the MCAT to some degree (consider someone who can't afford to buy the AAMC exams or have 20 prep books and a tutor).

 

This part stuck out a bit to me, so I wanted to quickly address it. I'm not sure why it matters what kind of activities an applicant has done. Do you need to do clinical work, shadow a physician, interact with patients, or do some form of research prior to medical school in order to be a good physician? In my opinion the answer is a definite no. All that matters are the qualities that an individual possesses that would enable them to become a great physician with the proper direction and education. Now, obviously many applicants possess these qualities, and certainly more people than there are actual spots in the various medical schools. So the problem becomes identifying people that BETTER possess these characteristics. That's why things like CASPer and the MMI exist.

 

I believe that CASPer is a great test for what it does. I'll preface this by saying that I don't particularly agree with them using it at all, but it's undeniable that it does what it's designed to do exceptionally well. You definitely need to be able to think quickly while considering as many possibilities that you can. Perhaps various experiences in life can help with this sort of problem solving. Regardless of how these skills are acquired, they will be tested during CASPer. You've either got it or you don't... it's as simple as that. In order to get better, I don't think you'd want to really practice anything specifically for CASPer, but rather to experience a wide variety of "things" in your life (ie. your ECs).

[Almighty Saguaro]

This part stuck out a bit to me, so I wanted to quickly address it. I'm not sure why it matters what kind of activities an applicant has done. Do you need to do clinical work, shadow a physician, interact with patients, or do some form of research prior to medical school in order to be a good physician? In my opinion the answer is a definite no. All that matters are the qualities that an individual possesses that would enable them to become a great physician with the proper direction and education. Now, obviously many applicants possess these qualities, and certainly more people than there are actual spots in the various medical schools. So the problem becomes identifying people that BETTER possess these characteristics. That's why things like CASPer and the MMI exist.

 

I believe that CASPer is a great test for what it does. I'll preface this by saying that I don't particularly agree with them using it at all, but it's undeniable that it does what it's designed to do exceptionally well. You definitely need to be able to think quickly while considering as many possibilities that you can. Perhaps various experiences in life can help with this sort of problem solving. Regardless of how these skills are acquired, they will be tested during CASPer. You've either got it or you don't... it's as simple as that. In order to get better, I don't think you'd want to really practice anything specifically for CASPer, but rather to experience a wide variety of "things" in your life (ie. your ECs).

Granted, clinical work, shadowing, interaction with patients, research, etc may not necessarily make one a better physician. But I think those experiences allow for a more serious consideration of why exactly you wish to pursue medicine. It seems odd to me that one could be so convinced that medicine, or even healthcare, is their "calling" when they've never actually observed up-close what being a physician actually entails, nor have they actually experienced being part of a healthcare team or interacting with patients as a healthcare professional would. I'm not saying these should be strict requirements for medical admissions but I do think they should be weighted a little bit more in the minds of applicants and ad coms alike. 

 

Of course, we can launch into debate on the impact of such a "requirement" on applicants of varying socioeconomic status, but that's a different story   My limited experience with clinical epidemiology has led me to the conclusion that SES confounds EVERYTHING!

[desperate_premed]

This part stuck out a bit to me, so I wanted to quickly address it. I'm not sure why it matters what kind of activities an applicant has done. Do you need to do clinical work, shadow a physician, interact with patients, or do some form of research prior to medical school in order to be a good physician? In my opinion the answer is a definite no. All that matters are the qualities that an individual possesses that would enable them to become a great physician with the proper direction and education. Now, obviously many applicants possess these qualities, and certainly more people than there are actual spots in the various medical schools. So the problem becomes identifying people that BETTER possess these characteristics. That's why things like CASPer and the MMI exist.

 

I believe that CASPer is a great test for what it does. I'll preface this by saying that I don't particularly agree with them using it at all, but it's undeniable that it does what it's designed to do exceptionally well. You definitely need to be able to think quickly while considering as many possibilities that you can. Perhaps various experiences in life can help with this sort of problem solving. Regardless of how these skills are acquired, they will be tested during CASPer. You've either got it or you don't... it's as simple as that. In order to get better, I don't think you'd want to really practice anything specifically for CASPer, but rather to experience a wide variety of "things" in your life (ie. your ECs).

 

 

Granted, clinical work, shadowing, interaction with patients, research, etc may not necessarily make one a better physician. But I think those experiences allow for a more serious consideration of why exactly you wish to pursue medicine. It seems odd to me that one could be so convinced that medicine, or even healthcare, is their "calling" when they've never actually observed up-close what being a physician actually entails, nor have they actually experienced being part of a healthcare team or interacting with patients as a healthcare professional would. I'm not saying these should be strict requirements for medical admissions but I do think they should be weighted a little bit more in the minds of applicants and ad coms alike. 

 

Of course, we can launch into debate on the impact of such a "requirement" on applicants of varying socioeconomic status, but that's a different story   My limited experience with clinical epidemiology has led me to the conclusion that SES confounds EVERYTHING!

 

I agree with Almighty Saguaro. I'm not saying these should be strict requirements at all but I do believe that someone who has done research for 3 years and is first author on 2 papers MUST ABSOLUTELY possess certain qualities which I believe are 100% important for medicine. I do also wonder how can anyone say he/she wants to become a doctor with ZERO clinical/hospital experience and having no clue what they are going to be facing both in medical school and when practicing? Seriously.

 

I'm also not saying CASPer is useless - not at all. All I'm saying is that I find it difficult to understand why a school would ignore autobiographical sketch and use CASPer instead - why not do both? why not give CASPer 10-15% weight and consider the applicant's experiences too? 

 

Someone on SDN said he has been a teacher for many years and is now applying to MD. Do you think this quality of the applicant should go unnoticed? Yes, you don't necessarily have to be a teacher to be a good future doctor but how you can ignore the qualities this person possesses (this also applies to any long-term employment position) and just say ok CASPer does it's job and we don't need to look into anything else.

 

I do agree that the qualities CASPer measures are relevant and important but I also agree that 90% of those qualities are going to be tested during MMI. CASPer shouldn't be replacing autobiographical sketch. 

 

 

And when I said I don't think CASPer is flawless or else it would have been adopted by many schools, I was not referring to CASPer in specific, but any online personal assessment tests. How many of the top 30 medical schools in the world have any online personal qualities assessment test? I find it hard to believe that McMaster was the first school who came up with this tool. I'm sure many schools around the world have at least considered it in some shape or form, but none of them actually utilized it.

 

Again, that's just my personal opinion

[Almighty Saguaro]

I agree with Almighty Saguaro. I'm not saying these should be strict requirements at all but I do believe that someone who has done research for 3 years and is first author on 2 papers MUST ABSOLUTELY possess certain qualities which I believe are 100% important for medicine. I do also wonder how can anyone say he/she wants to become a doctor with ZERO clinical/hospital experience and having no clue what they are going to be facing both in medical school and when practicing? Seriously.

 

I'm also not saying CASPer is useless - not at all. All I'm saying is that I find it difficult to understand why a school would ignore autobiographical sketch and use CASPer instead - why not do both? why not give CASPer 10-15% weight and consider the applicant's experiences too? 

 

Someone on SDN said he has been a teacher for many years and is now applying to MD. Do you think this quality of the applicant should go unnoticed? Yes, you don't necessarily have to be a teacher to be a good future doctor but how you can ignore the qualities this person possesses (this also applies to any long-term employment position) and just say ok CASPer does it's job and we don't need to look into anything else.

 

I do agree that the qualities CASPer measures are relevant and important but I also agree that 90% of those qualities are going to be tested during MMI. CASPer shouldn't be replacing autobiographical sketch. 

 

 

And when I said I don't think CASPer is flawless or else it would have been adopted by many schools, I was not referring to CASPer in specific, but any online personal assessment tests. How many of the top 30 medical schools in the world have any online personal qualities assessment test? I find it hard to believe that McMaster was the first school who came up with this tool. I'm sure many schools around the world have at least considered it in some shape or form, but none of them actually utilized it.

 

Again, that's just my personal opinion

 

Since I'm quoted in the above, I feel I should clarify my stance:

 

In summary of my previous posts, if CASPer functions to test how an applicant would approach an ethical dilemma in a real life clinical setting, I argue that its potential ecological validity seems poor. 

 

In my mind, to say that "CASPer shouldn't be replacing autobiographical sketch" is to tantamount to saying that medical schools should not retain the prerogative to decide on what criteria they will select students. To that, I would disagree. 

 

If they want to test an applicant's ability to problem solve and deal with ethical issues but do NOT wish to consider the applicant's extracurricular background, I think that is well within their right...especially since a multiplicity of sound arguments can be made in support of this stance. However, I reiterate that I don't think CASPer is the best way to achieve that objective. 

 

Thankfully, different schools in Ontario seem to have somewhat different ways of looking at applicants so, in support of their chosen admissions policies, each school could technically say that an applicant is free to apply to other universities they feel are more aligned with their personal objectives, ideals, background, aptitudes, etc.