On the topic of possible site/examiner variation.. as per the MCC website, "objectivity of scoring is achieved using standardized guidelines for exam administration, the training of examiners and of SPs, and the use of predetermined scoring instruments for OSCE stations."
So the claim is that objectivity is achieved?
The sample rubrics posted on the MCC website show a standardized checklist for examiners, as well as an "interaction rating scale" where they're asked to score our interviewing skills or physical exam skills as "inadequate, marginal, adequate, or superior". https://mcc.ca/examinations/mccqe-part-ii/clinical-stations/case-1-14-minute-station-ken-scott/
This interaction rating scale inherently carries with it a great deal of subjectivity. Further to that, the "standardized checklists" afford the examiner the option of selecting "attempted only" or "completed satisfactorily" for each bullet point. I'm not sure how you can "attempt to ask about fever or weight loss" .. but not satisfactorily ask about fever or weight loss? This would only make sense for physical exam techniques.
Another point worth mentioning is that the examiner checklists assume that the examiner is listening attentively for the entirety of the 14 minute station. Then for the 7 more stations they have to sit through, totalling almost 120 minutes of attentive listening. If a question is asked, but possibly missed by the examiner.. there is no way to know. High stakes exams like the CCFP SOO stations are recorded on video so that exam results can be re-evaluated when significant discrepancies occur.
There is no option for re-evaluation of scores on the LMCC2. You get a pass/fail score, and then a generalized breakdown of your performance with no tailored feedback or insight into specific adjustments that can be made. If you want your score re-checked, it's another $250 yet they say on the website that a re-check does not include a re-evaluation. So essentially whatever the examiner marks on the checklist is set in concrete and a re-check just makes sure that scores were entered into the computer properly.