USMLE Step 1 Changes to Pass/Fail Starting 2022

[Monkey D. Luffy]

https://www.usmle.org/inCus/

On the surface, it may seem that this change would not have much of an effect on Canadians writing the test to open up fellowship opportunities in the US, but given the huge emphasis that will be taken off this metric for the NRMP match, I wonder if more Canadians will consider applying to the US match especially for competitive specialties, now that a high USMLE score is no longer a barrier.

[downinggr]

I think a larger concern might be the Canadians who decided to study in the Caribbean or other international schools who may have been targeting US schools for the match.

[bearded frog]

IMHO very good news for Canadians writing USMLE for fellowship, bad news in the short term for Canadian applicants to US residency, as without the absence of standardized test scoring, schools might prefer to go with graduates of medical schools they have context for, IE top 50 US schools, vs carribean or unfamiliar candian med schools.

[circlecircledotdot]

.

[bearded frog]

8 hours ago, circlecircledotdot said:

Maybe you know something that I don't but how is this good news for Canadians writing for the puposes of a fellowship? From all the residents I've talked to,  the goal has always been to "just pass".

They still looked at the scores. To practice in the states the threshold was to just pass.

[rmorelan]

20 minutes ago, bearded frog said:

They still looked at the scores. To practice in the states the threshold was to just pass.

yeah it was always something to make people think this person is potentially good or bad - 

this is going to make things a lot stranger for residency ha. At least with standardized scores you have some form of equalization. I am not sure how you are supposed to evaluate in any timely fashion the flood of applicants to various programs without some standardized tools. Canada is hard enough - imagine well over 10x the number of places in the US. I would be worried they would start using the med school reputation as some form of a proxy (which is bad because those are expensive - so now you introduced another bias)

[circlecircledotdot]

.

[rmorelan]

47 minutes ago, circlecircledotdot said:

Damn well this is concerning... I was planning to write the USMLE during my summer of M2 but I think now I might strongly reconsider until my 4th year or something

just so you can do it in a pass fail situation? if for fellowships way down the line (although that isn't always needed to be clear as someone doing US fellowships - you have to research if it is required) that would very likely work. 

[procrastinating]

1 hour ago, circlecircledotdot said:

Damn well this is concerning... I was planning to write the USMLE during my summer of M2 but I think now I might strongly reconsider until my 4th year or something

But wouldn't you have to relearn all of your M2 knowledge to write until later? That would really suck, even if only for a pass, the knowledge won't be as fresh and would probably require you to study a fair bit.

 

What are people's thoughts on how Step 1 will be used between now and 2022? Will it possibly be less emphasized, in preparation for the inevitable future of p/f?

[Intrepid86]

Probably good for those seeking fellowships, and probably bad for everyone else. When it comes to applying for residency, Step 1 P/F is just going to make programs search for other objective measures to narrow their applicant pools. Like froggy said above, it's also going to make things like school, rank, reputation, nepotism/cronyism, and subjective clinical scores more important and therefore worse.

[rmorelan]

7 hours ago, Intrepid86 said:

Probably good for those seeking fellowships, and probably bad for everyone else. When it comes to applying for residency, Step 1 P/F is just going to make programs search for other objective measures to narrow their applicant pools. Like froggy said above, it's also going to make things like school, rank, reputation, nepotism/cronyism, and subjective clinical scores more important and therefore worse.

Honest that is exactly my main concern with this - at least prior you could come from any school and with great scores (something you can hopefully change) then you potentially move forward even for competitive things. Under this the med school where went to school is really going to matter. 

Every time we get rid of some stressful evaluation system we end up with some later on stress situation to compensate. There are a ton of weaknesses with the USMLE exam to be fair for residency evaluation - it doesn't test a lot of important things that matter a lot more and end performance in residency has at best a fuzzy relationship with your score - perhaps changing the test would have been a better option.

They are going to end up with our CARMS situation but amped up with the fact they have way more schools, and those schools are way less standardized. People who cannot afford to go to high end schools are just going to be further disadvantaged. 

 

Edited February 16, 2020 by rmorelan

[JohnGrisham]

14 minutes ago, rmorelan said:

Honest that is exactly my main concern with this - at least prior you could come from any school and with great scores (something you can hopefully change) then you potentially move forward even for competitive things. Under this the med school you went to is really going to matter. 

Every time we get rid of some stressful evaluation system we end up with some later on stress situation to compensate. There are a ton of weaknesses with the USMLE exam to be fair for residency evaluation - it doesn't test a lot of important things that matter a lot more and end performance in residency has at best a fuzzy relationship with your score - perhaps changing the test would have been a better option.

They are going to end up with our CARMS situation but amped up with the fact they have way more schools, and those schools are way less standardized. People who cannot afford to go to high end schools are just going to be further disadvantaged. 

 

Agreed. Bad news.

I'm the first person to question utility of standardized exams, but i would much prefer them then direct nepotism in clinical practice and the even more huge variability in clinical experiences for LORs etc.  At least standardized exam is a bit more in your control (sure, I will also admit that there are many systemic barriers that come with them as well...but generally once you got into medical school, you probably are able to account for them with individual skill far more than not.). 

[rmorelan]

35 minutes ago, JohnGrisham said:

Agreed. Bad news.

I'm the first person to question utility of standardized exams, but i would much prefer them then direct nepotism in clinical practice and the even more huge variability in clinical experiences for LORs etc.  At least standardized exam is a bit more in your control (sure, I will also admit that there are many systemic barriers that come with them as well...but generally once you got into medical school, you probably are able to account for them with individual skill far more than not.). 

yeah I am not always a fan of them either - because they often are created with a less than perfect philosophy for medical testing. 

But this? No one can come close to understanding the quality of all the schools in the US. They will have to spend a lot more time screening people without a simple method with I would expect a ton of applicants. Resources are limited so it is hard to imagine there won't be some form of short cut taking at many places. 

You are left with research, ECs, and LORs and maybe clinical evaluations which are hit and mess (you may only have a couple of weeks working with in an area of interest during primary clerkship, and electives are messy and complex to). The first two are at best an imperfect tool for studying anything, and the LOR one is automatically biased by the place you did your school. Clerkship evaluations are also quite likely to be biased,.

I wonder if some schools would start looking at shelf exam scores? Maybe those are more focused on relevant areas for each area.  

[JohnGrisham]

15 hours ago, rmorelan said:

yeah I am not always a fan of them either - because they often are created with a less than perfect philosophy for medical testing. 

But this? No one can come close to understanding the quality of all the schools in the US. They will have to spend a lot more time screening people without a simple method with I would expect a ton of applicants. Resources are limited so it is hard to imagine there won't be some form of short cut taking at many places. 

You are left with research, ECs, and LORs and maybe clinical evaluations which are hit and mess (you may only have a couple of weeks working with in an area of interest during primary clerkship, and electives are messy and complex to). The first two are at best an imperfect tool for studying anything, and the LOR one is automatically biased by the place you did your school. Clerkship evaluations are also quite likely to be biased,.

I wonder if some schools would start looking at shelf exam scores? Maybe those are more focused on relevant areas for each area.  

Another thing is, at least for my colleagues who went through the US training system and into residency - doing out of state electives was hardly necessary, outside of unlocking certain hard to get regions like california. Most simply relied on having strong scores and a decent CV and got interviews across the nation in fairly competitive specialties, without needing to fly around and spend $$$ on out of state electives. Only if it was a really top notch big name competitive program would they do an audition rotation. 

Now with no step score differentiator, i wouldnt be surprised if it becomes more like Canada.

[rmorelan]

Just now, JohnGrisham said:

Another thing is, at least for my colleagues who went through the US training system and into residency - doing out of state electives was hardly necessary, outside of unlocking certain hard to get regions like california. Most simply relied on having strong scores and a decent CV and got interviews across the nation in fairly competitive specialties, without needing to fly around and spend $$$ on out of state electives. Only if it was a really top notch big name competitive program would they do an audition rotation. 

Now with no step score differentiator, i wouldnt be surprised if it becomes more like Canada.

oh yeah hadn't thought of that actually - bouncing around and setting those up will be an entirely new sort of pain as well.....and once again anyone that for any reason - financial/personal etc. that cannot do that will be disadvantaged. 

in the end I stand by my original point - ultimately for all these competitive spots there are way more people than positions, so no matter what you do there will be a painful selection process which will be incredibly stressful and many people won't get want they want. You can design the system any way you want but you cannot avoid that underlying fact. So the question becomes how do you design a system that is as unbiased as possible, and with a large degree of fairness allocates those spots while respecting that system also has to include the needs of the hospitals and patient care. In all cases don't pretend that you can avoid the stress/pain of this - you cannot - it is going to hurt (and that won't be the only point in medicine that will be like that either looking further in residency/fellowship or staff positions). People are always going to do extra work to line up with whatever system you have be it the USMLE prior, or the Canada system where people a ton of ECs/research etc which otherwise they wouldn't do (in both cases you will note ha that "studying for the test rather than life" with the USMLE or doing ECs/research for acceptance sake primarily in Canada isn't helping you actually learn the MEDICINE or at least not efficiently in any case). All you can do is move around the point of stress. 

[rmorelan]

as a foot note - some people think this will just result in the Step 2 exam down there being used instead. That is a bit annoying for Canadians as now there two tests prior to applying to residency in the US potentially rather than one ha (although Step 1 is probably the test that is most different than our exams so at least that is a benefit). 

 

[ChemPetE]

13 hours ago, rmorelan said:

as a foot note - some people think this will just result in the Step 2 exam down there being used instead. That is a bit annoying for Canadians as now there two tests prior to applying to residency in the US potentially rather than one ha (although Step 1 is probably the test that is most different than our exams so at least that is a benefit).

This. There’s a lot of anticipation that Step 2 CK will just be the new differentiator. Good news is that the Canadian curriculum is much more in tune with that exam, and will be much easier to score well on that as a Canadian student. I had to study and work hard just to pass step 1, but basically took step 2 and 3 with minimal dedicated prep and had no problem getting competitive scores for those.