Interesting Program Directors Take on CARMS

[rmorelan]

2 hours ago, shematoma said:

The thing is that all the criteria you list are subjective. How many electives is enough to demonstrate interest? Is it 2 weeks, 4 weeks, or should it be all your electives? Is someone who did 6 weeks likely to be a better resident than someone who did 4? What counts as an accomplishment in that field? And reference letters are by definition subjective - they're one person's opinion of the applicant, and worth only as much as you trust the opinion of that person. You also don't know if the reference writer has some kind of vested interest in the applicant.

On top of that, how many of these above factors can be validated scientifically as being significant in choosing residents who are likely to perform well? If we're going to justify some selection method for residents then we should at least do so based on evidence - but I don't think we have any in this case. Whereas if we had a more objective measure - then at least you know that on top of having great references, "accomplishments" in the field, that the person has a solid bank of medical knowledge.

Of course there's always going to be a subjective component, the problem is that right now we have no real objective component whatsoever except for some qualitative and categorical data.

True - I mean no one is saying drop the interview, letters, and elective performance out of the equation. Yet I can still see why a PD would want something else that tests a large part of the required skill set of a resident (actually becoming a medical expert in often a challenging field) and have that as well.

Speaking as an example of one I have been involved in the process of selecting residents - and what we have is often rather limited to separate out the "middle of the pack" people. Top 10-20% and bottom 10-20% ranking is often easy. It is figuring out the central mass of people all with similar letters, similar electives, and similar ECs that becomes quite challenging. 

[Coby]

A system also exists currently for the IMGs to help provide more objective measures with the NAC OSCE grade being used as one criteria for selection as well as the EE (which I believe will be phased out) and some program directors are starting to prefer  IMGs to have passed the LMCC as well as  all IMGs providing transcripts for all of their medical school and often with their class rank being provided by many deans. Could something like this be done for CMGs. ? 

[rmorelan]

1 minute ago, Coby said:

A system also exists currently for the IMGs to help provide more objective measures with the NAC OSCE grade being used as one criteria for selection as well as the EE (which I believe will be phased out) and some program directors are starting to prefer  IMGs to have passed the LMCC as well as  all IMGs providing transcripts for all of their medical school and often with their class rank being provided by many deans. Could something like this be done for CMGs. ? 

Something similar I think it what this group of PDs is thinking of - although that is more extreme than what is being suggested 

In some areas the above examples you provide do give an advantage to IMGs in the second around in some cases. There is a group of PDs that will always default down to something objective when it is provided/available. 

[LittleDaisy]

27 minutes ago, rmorelan said:

Ha very fine I would say - I mean once you say you are leaving there is blood in the water. I wonder how willing the average path PD would be to allowing that. I guess there has to also be some electives in the program overall (otherwise if you don't switch you are now short some blocks, and I am not sure how those would be made up. 

In that person's case, the derm PD is convinced that the resident will be able to transfer into family medicine with no difficulty, and even allowed the last 3 months of that resident's training to be in family medicine (you could be released earlier from your program).

It depends on what program you are trying to transfer into, if you aim for something not family medicine (the MoH and PGME at each university is very flexible for residents transferring into family medicine) , then the chance of transfer might be significantly lower, and I will not approach my PD about the intention of transferring and allowing off-service electives. 

[rmorelan]

2 minutes ago, LittleDaisy said:

In that person's case, the derm PD is convinced that the resident will be able to transfer into family medicine with no difficulty, and even allowed the last 3 months of that resident's training to be in family medicine (you could be released earlier from your program).

It depends on what program you are trying to transfer into, if you aim for something not family medicine (the MoH and PGME at each university is very flexible for residents transferring into family medicine) , then the chance of transfer might be significantly lower, and I will not approach my PD about the intention of transferring and allowing off-service electives. 

Sounds fair  Also I am sure program don't want this done too often - they don't want to be thought of as a stepping stone program ha 

[LittleDaisy]

16 minutes ago, rmorelan said:

Sounds fair  Also I am sure program don't want this done too often - they don't want to be thought of as a stepping stone program ha 

If anyone is thinking of transferring into family medicine (a great specialty! well I am biased ), and is afraid of speaking to your current PD about having off-service electives in primary care, one good person to talk you would be your PGME vice-dean & dean, they would often vouch for your interest in primary care and advocate for you in front of your current PD. However, I am not even sure if doing electives in FM is required for transferring itself at your home school, I know a few surgical residents who transferred successfully without any FM electives. 

[ralk]

8 hours ago, rmorelan said:

Happens to be from my school - but it is response to others to have similar concerns.

http://www.cmaj.ca/content/canadian-program-directors-have-zero-data-select-residency-candidates

Canadian program directors have zero data to select residency candidates

• Matthew D. McInnes, Associate professor of Radiology and Epidemiology, University of Ottawa, The Ottawa Hospital Research Institute, Ottawa, Ont.

11 April 2018

I share Dr. Persad's concern. I supervised nine CARMS matches as diagnostic radiology residency program director in Ottawa and it was plainly obvious that the lack of objective data with which to evaluate candidates is at the crux of a deeply flawed system. Good research(1) has identified that objective data (medical school marks or examinations) are the only reliable indicators of success in residency. Canadian program directors presently have zero objective data to use to select candidates. We are, as far as I am aware, the only system in the world that has both an entirely pass fail system combined with lack of standardized examinations (the LMCC examination is done after Carms).

This frustration from students (subjective/ vague criteria) and program directors (lack of useful data points) comes up year after year(2). Until Canadian Medical schools and the LMCC rectify this by moving the LMCC to third year (for a four-year program), or return to an objective, marks-based evaluation system, these frustrations will not subside.

Dr. McInnes has been beating this drum for a while, to the point of asking for undergrad marks from applicants as part of their assessment.

I think importantly, they've had the power to make positive changes and haven't. He's rallied against subjective or vague criteria, but has never provided transparency as to how they select their residents, especially specifics on how they select one candidate over another, despite clearly having a formalized metric for doing so. There's also nothing against programs testing their applicants' knowledge base, which they could do if it is of such paramount importance.

There are good reasons we moved to a pass/fail system - it meant a heavy focus on information of limited clinical value, emphasized short-term knowledge acquisition, and contributed to student stress and burnout as result. I'm sympathetic to the idea of standardized testing, which at least means less-frequent stressors in evaluating knowledge, but I'd argue we need something a lot better than the LMCC, which functions reasonably well as a pass/fail test, but tests a lot of extraneous or useless knowledge. There's also an issue of stratifying physicians by specialty on the basis of a single test result, as is done in the US - we've made a lot of progress to get strong medical students into traditionally lower-competitive specialties like FM, IM, and Psychiatry, and pushing applicants with weaker test scores into those professions by barring them from more competitive ones works against those efforts. Overall, such testing may allow program directors to better identify the best students, but may be counterproductive to producing the best students. 

[GrouchoMarx]

4 hours ago, LittleDaisy said:

If you are a CMG and are currently a pathology resident, I don't think that any PDs will look negatively at you. I will suggest that you do electives in your field of interest, and get strong LORs. The PGY1 in pathology, you mainly do off-service rotations, and could easily get some training credited. 

If you want to transfer into family medicine, it is relatively easy. I haven't heard anyone being turned down (even with no electives and no family preceptors LOR)

it's not that easy. rmorelan mentioned a lot of the hang-ups.

once you make your intention known to your pd, its not certain how they will react to it. if the transfer gets blocked somehow, youll have a black mark on you for the rest of your residency.

once you do path your electives are basically all path outside of PGY1 which are core rotations more or less assigned to you without much flexibility. I have a lot of ward experience, of course, but little experience in the discipline I'm interested in. family i have seen but im not interested in family.

It also doesn't help that path is looked upon poorly by this particular program. I cant blame them really. but it makes it impossible to stand out because youre assumed to be low-quality from the get-go and are afforded no opportunities to prove otherwise due to rigid pgy1 requirements and a home-program-centric residency. unlike many other fields, path doesnt see a lot of anyone else besides the occasional surgery resident for frozens. even then its just the nurse you see most of the time.

[rmorelan]

2 hours ago, ralk said:

Dr. McInnes has been beating this drum for a while, to the point of asking for undergrad marks from applicants as part of their assessment.

I think importantly, they've had the power to make positive changes and haven't. He's rallied against subjective or vague criteria, but has never provided transparency as to how they select their residents, especially specifics on how they select one candidate over another, despite clearly having a formalized metric for doing so. There's also nothing against programs testing their applicants' knowledge base, which they could do if it is of such paramount importance.

There are good reasons we moved to a pass/fail system - it meant a heavy focus on information of limited clinical value, emphasized short-term knowledge acquisition, and contributed to student stress and burnout as result. I'm sympathetic to the idea of standardized testing, which at least means less-frequent stressors in evaluating knowledge, but I'd argue we need something a lot better than the LMCC, which functions reasonably well as a pass/fail test, but tests a lot of extraneous or useless knowledge. There's also an issue of stratifying physicians by specialty on the basis of a single test result, as is done in the US - we've made a lot of progress to get strong medical students into traditionally lower-competitive specialties like FM, IM, and Psychiatry, and pushing applicants with weaker test scores into those professions by barring them from more competitive ones works against those efforts. Overall, such testing may allow program directors to better identify the best students, but may be counterproductive to producing the best students. 

These points are probably the strongest counter arguments against some form of centralized testing  - particularly the second paragraph  While I am not pushing a side here - more just encouraging a debate - if there was some form of objective testing I would hate it to be say course work grades - too fine a measure etc and suddenly everything is important maximally at all times. Sounds like a recipe for burnout. 

Ottawa has been though quite open about the importance of academic excellence in its selection criteria. If they haven't formalized what that means it would be in part because there isn't a standardized way of evaluating what that means in our standard system (GPA in UG? What about degree difficulty? What about masters/phD work - should that count? What about the MCAT? What do you do if they don't the MCAT? What about..........) All that means it is impossible to quickly define anything in a useful fashion for anyone. Some of Dr. McInne's published work is relatively open about some of this stuff so I am not revealing anything not out there (although I will say we know have a new PD  who will like all new people to the job bring her own take on what is important).

Could certainly test applicant knowledge base - some schools in radiology do exactly that. I have always been a bit wary of say a 30min to 1hr test in the middle of an interview as a selection system. Seems limited, ha. 

 

 

[LittleDaisy]

2 minutes ago, GrouchoMarx said:

it's not that easy.

once you do path your electives are basically all path outside of PGY1 which are core rotations more or less assigned to you without much flexibility. I have a lot of ward experience, of course, but little experience in the discipline I'm interested in. family i have seen but im not interested in family.

It also doesn't help that path is looked upon poorly by this particular program. I cant blame them really. but it makes it impossible to stand out because youre assumed to be low-quality from the get-go and are afforded no opportunities to prove otherwise due to rigid pgy1 requirements and a home-program-centric residency. unlike many other fields, path doesnt see a lot of anyone else besides the occasional surgery resident for frozens. even then its just the nurse you see most of the time.

I am not sure that the PDs assume that pathology CMG residents are low-quality? There are people genuinely interested in pathology and who are happy in their programs (I know a few).

I do agree that English Canada hires a lot of IMG trained pathologists, and the job market isn't that great right now. In Quebec, they mainly hire Quebec-trained pathologists, so the job outlook is a bit better. 

[GrouchoMarx]

10 minutes ago, LittleDaisy said:

I am not sure that the PDs assume that pathology CMG residents are low-quality? There are people genuinely interested in pathology and who are happy in their programs (I know a few).

I do agree that English Canada hires a lot of IMG trained pathologists, and the job market isn't that great right now. In Quebec, they mainly hire Quebec-trained pathologists, so the job outlook is a bit better. 

oh, dont get me wrong, there are some really strong CMG residents in pathology. 

yet the notion persists that a CMG in pathology = someone who didnt match and had to backup and is therefore seen as tainted, or some sort of incel

here is how my email correspondence to the PD went

GM: Dear bla bla bla, I am interested in your field and am wondering if you would be able to inform me if an opportunity exists for transfer, and what steps I should take if one is available. Signed GM pathology PGYx

PD: Dear GM. Sorry, there are no transfer opportunities.

I have wondered if it was just a hard year but I have spoken to other transfer residents and the PD will at least ask for the CV, or arrange a meeting or something.

I have never seen a path resident switch into anything competitive. same goes for psych, and family is very rare.

the hierarchy of fields is real and path is at the dead bottom. the problem is compounded by an overreliance on IMG recruitment. lots of IMGs in path should not be in medicine at all, IMO. poor communication skills and questionable medical knowledge combined with a stubborn misguided faith in it. not all, but there are more than is acceptable.

[bearded frog]

Not having a giant standardized test is one of the things that makes medical school in Canada better than the US. Here, for the most part, they actually teach clinical medicine and relevant topics, as opposed to the US pre-clerkship which teaches how to do good on the USMLE, who cares if some of the USMLE content isn't actually helpful or relevant for a doctor. If they introduced a standardized measure, regardless of what it is, the schools would all change their curriculum to teach and optimize whatever measure that is, and as long as that measure isn't "be the best doctor you can be" then that hurts students.

[medigeek]

3 minutes ago, bearded frog said:

Not having a giant standardized test is one of the things that makes medical school in Canada better than the US. Here, for the most part, they actually teach clinical medicine and relevant topics, as opposed to the US pre-clerkship which teaches how to do good on the USMLE, who cares if some of the USMLE content isn't actually helpful or relevant for a doctor. If they introduced a standardized measure, regardless of what it is, the schools would all change their curriculum to teach and optimize whatever measure that is, and as long as that measure isn't "be the best doctor you can be" then that hurts students.

Studying for the usmle step 2 and 3 teaches you all the clinical medicine you could possibly need to know. The main upside to Canada is students can be a bit more hands on but that's not the norm compared to USA. 

[bearded frog]

I just don't think that optimizing a curriculum around studying for exams is the best way to learn medicine. I mean, we studied for MCCQEI and MCCQEII and those were just a chore that I had to get through to keep on learning on the job of actually being a doctor and didn't really help me learn medicine.

[GrouchoMarx]

16 minutes ago, bearded frog said:

Not having a giant standardized test is one of the things that makes medical school in Canada better than the US. Here, for the most part, they actually teach clinical medicine and relevant topics, as opposed to the US pre-clerkship which teaches how to do good on the USMLE, who cares if some of the USMLE content isn't actually helpful or relevant for a doctor. If they introduced a standardized measure, regardless of what it is, the schools would all change their curriculum to teach and optimize whatever measure that is, and as long as that measure isn't "be the best doctor you can be" then that hurts students.

I disagree. the practical points of medicine have science underpinnings. i have found that knowledge of the basic science of medicine has augmented my clinical skill imo. 

knowing the why is what separates us from the midlevel provider. if we are just doers, and not knowers, then other doers will try to do too.

[medigeek]

1 minute ago, GrouchoMarx said:

I disagree. the practical points of medicine have science underpinnings. i have found that knowledge of the basic science of medicine has augmented my clinical skill imo. 

knowing the why is what separates us from the midlevel provider. if we are just doers, and not knowers, then other doers will try to do too.

Very well said. The extra knowledge and understanding of basic sciences is what separates doctors from midlevels. Anyone can memorize an algorithm. 

[bearded frog]

1 minute ago, GrouchoMarx said:

I disagree. the practical points of medicine have science underpinnings. i have found that knowledge of the basic science of medicine has augmented my clinical skill imo. 

knowing the why is what separates us from the midlevel provider. if we are just doers, and not knowers, then other doers will try to do too.

I didn't say that we shouldn't learn relevant biochemistry, physiology, biology, physics whatever.... just maybe we don't have to memorize all the interleukins or devote as much time to interpreting pathology slides?

[GrouchoMarx]

3 minutes ago, bearded frog said:

I didn't say that we shouldn't learn relevant biochemistry, physiology, biology, physics whatever.... just maybe we don't have to memorize all the interleukins or devote as much time to interpreting pathology slides?

what do you suppose we do instead?

[medigeek]

Just now, bearded frog said:

I didn't say that we shouldn't learn relevant biochemistry, physiology, biology, physics whatever.... just maybe we don't have to memorize all the interleukins or devote as much time to interpreting pathology slides?

I've taken step 1 and a lot of the stuff on there was relevant to general practice. Rare diseases, yes. But stuff you'd still see at an academic center. 

[bearded frog]

Just now, GrouchoMarx said:

what do you suppose we do instead?

What we do now... and learn medicine (all parts from basic sciences, physiology, anatomy, pathology) in a manner which sets us up for success in practice, and not do get 100% on a standardized test.

1 minute ago, medigeek said:

I've taken step 1 and a lot of the stuff on there was relevant to general practice. Rare diseases, yes. But stuff you'd still see at an academic center. 

I'm a resident. Guess how many pathology slides I have interpreted and will be expected to interpret prior to finishing.

[GrouchoMarx]

10 minutes ago, bearded frog said:

What we do now... and learn medicine (all parts from basic sciences, physiology, anatomy, pathology) in a manner which sets us up for success in practice, and not do get 100% on a standardized test.

I'm a resident. Guess how many pathology slides I have interpreted and will be expected to interpret prior to finishing.

I'm also a resident.

guess how many JVPs I've seen in the last three years? or how many babies I've delivered since medical school? or how many bones I've set?

that logic goes both ways.

[ellorie]

15 minutes ago, GrouchoMarx said:

guess how many JVPs I've seen in the last three years?

Guess how many JVPs I've seen EVER?  Hint: 0

I've definitely pretended to see some though  The key is to crouch, look thoughtful, and make vague hand gestures in the direction of the neck.  Bonus points if you fuss around with the bed elevation a little bit.

[GrouchoMarx]

4 minutes ago, ellorie said:

Guess how many JVPs I've seen EVER?  Hint: 0

I've definitely pretended to see some though  The key is to crouch, look thoughtful, and make vague hand gestures in the direction of the neck.

there sa lot of talk about histology being only marginally useful, but at least it is.

the jvp is voodoo

[medigeek]

We can always cut out very low yield material but the right thing to do is replace it with even more high yield material. 

[JohnGrisham]

10 hours ago, bearded frog said:

Not having a giant standardized test is one of the things that makes medical school in Canada better than the US. Here, for the most part, they actually teach clinical medicine and relevant topics, as opposed to the US pre-clerkship which teaches how to do good on the USMLE, who cares if some of the USMLE content isn't actually helpful or relevant for a doctor. If they introduced a standardized measure, regardless of what it is, the schools would all change their curriculum to teach and optimize whatever measure that is, and as long as that measure isn't "be the best doctor you can be" then that hurts students.

I wouldn't be opposed if they made step 2 CK (clinical knowledge) the "standardized Canadian test" for comparison. It is far better a test than step 1, and highly clinically relevant.