An important challenge during the COVID-19 pandemic was the conduction of final year postgraduate resident examination in medical colleges of India because it depends predominantly on doctor patient interaction. Conventional exit examination consists of allotting clinical cases to the residents; each candidate is given one long and three short cases. The candidate is expected to elicit history, perform clinical examination, make a differential diagnosis and present the cases to the examiners following which there is a table viva. But because of the pandemic it was difficult to conduct such an examination. First, hurdle was that such a pattern would jeopardise the safety of all including patients. Second, was the difficulty in arranging a large number of patients with different but common diagnosis for examination. Third, was that even if such an examination was conducted it would create an extra burden on protective resources like Personal Protective Equipment (PPE) which could be better utilised during this pandemic. Similar studies were conducted by Malhotra R et al., and Iyenger KP et al., in which they concluded that virtual practical examination may not replace the traditional face to face practical examination assessment, but as confidence in the virtual examination model improves, it can be a useful adjunct to the former [1,2].
So, an innovative method was devised by the teaching faculties of Department of Orthopaedics, GMC Srinagar to ensure safety of the examiners and candidates as well as maintaining the required standard of the examination. This method was employed successfully in conducting exit examinations of final year postgraduate residents in this department. Thus, this study aimed to evaluate the effectiveness of the preformed questionnaire on virtual pattern for exit examinations of final year postgraduate residents of medical colleges in India.
Materials and Methods
This study was a prospective cohort study conducted in the Department of Orthopaedics, GMC Srinagar, Jammu and Kashmir, India in the month of July 2020 after consent from both the candidates and examiners as well as prior approval from the Head of Department. A total number of 10 candidates appeared in the examination. The examination was conducted by a total of four examiners, two from the department itself and two from an external university.
The conventional format was replaced by a digital presentation format in which the cases were prepared by the faculty. A majority of faculty members were asked to prepare individual digital presentations for long as well as short cases. Presentation format consisted of detailed history, clinical examination, photographs of any deformity, elicited clinical signs and radiological imaging. All the cases were mailed to the Head of the Department who compiled a collection of long and short cases with maximum details from the library. A total number of 20 long and 41 short cases were prepared. Each case was given a numeric code and random allocation of the cases to the candidate was ensured by chit-pull system on the day of examination. One common short case (Club foot) was kept for all the candidates to assess the candidate’s knowledge about this common deformity. All the cases were presented to external examiners one day before the examination, to make them familiar with the format and address any issues regarding the format.
A mock test was conducted four weeks before the date of the examination so that the candidates become familiar with the new virtual pattern. On the morning of the examination, the candidates were briefed regarding the pattern of examination. Random allocation of the cases to the candidate was ensured by chit-pull method. Arrangements for the candidates were made in the academic section of the department to ensure adequate social distancing. All the candidates were provided with one N95 face mask and a pair of gloves. Adequate arrangements were kept in case any candidate needed additional gloves and masks. Case presentations, already prepared in power point format, were displayed in the examination room. A total time limit of one hour was set for each long case and 15 minutes for each short case.
Each case began with the first slide of the presentation showing a brief clinical history consisting of patient’s chief complaints and history of presenting illness. The candidates were given 5 minutes to read the slide following which the candidates were asked if they required any other details in history other than those mentioned. Gaps in history were filled by examiner whenever enquired by the candidate. The candidate was then asked for the differential diagnoses based on the history provided. The next slide showed some examination findings and clinical tests. The candidate was asked to explain these findings, whether they needed any additional clinical findings to consolidate his diagnosis, narrow down his differential diagnosis and rule out others. A slide showing a specific clinical test’s photograph/video was used to ask in detail regarding the intricacies of that particular test. Afterwards the candidate was asked to demonstrate relevant clinical tests on a volunteer working in the department who was kept in quarantine for a period of 14 days before examination. Radiological images sought by the examinee were shown in the subsequent slides. Candidate was then asked to summarise the findings and was questioned regarding the final diagnosis and planned management.
A sample of long case scenario is depicted in [Table/Fig-1] showing relevant history, examination, labrotary values and X-ray along with the potential questions that can be asked by examiner (shown in the notes section which will only be visible to the examiner when the slide is in presenter mode).
Long case slide showing radiological images of the patient.
| Relevant History |
|---|
| An elderly man of average height and build. Has history of limp since childhood. Minimal pain. Had been operated around left hip when he was a child. Does not remember much what had happened. |
| Relevant Examination findings |
| There are several well-healed scars over lateral aspect of the left hip. There is a discharging sinus present in the groin/greater trochanter. There are puckered scars suggestive of healed sinuses. The left leg is shortened with gross generalized wasting particularly of the thigh and buttock. There is a pelvic obliquity with the ASIS higher on the left side and a compensatory scoliosis which does not fully correct when sitting down on the couch suggestive of a fixed element to the scoliosis.There is a combination of real and apparent shortening of the left leg. Galeazzi’s test demonstrated that most of the shortening is in the femur, but there is, however, a suggestion of a small amount of tibial shortening. Bryant’s triangle suggests most of the femoral shortening is above the trochanter. There is a definite decreased difference between thumb over the ASIS and fingers over the trochanter on the left side compared to the right. The hip is flexed, adducted and medially rotated. All movements are grossly restricted by pain and spasm. |
| Relevant Lab. values | Potential questions |
| Raised ESR with relative lymphocytosis.CRP raised. | What is your differential diagnosis?What you want next?What is the diagnosis and how will you manage?How do you diagnose tuberculosis of hip?Classify tuberculosis of hip?What is the natural history of the tuberculosis of hip?Conventional and WHO skeletal tuberculosis regimes?Shanmugasundaram Classification. |
 |
The short case viva was conducted in a similar fashion, focusing primarily on the diagnosis and management of the condition as shown in [Table/Fig-2]. The case discussions carried 300 marks and were followed by table viva for 80 marks on X-rays, osteology, orthopaedic instruments, and orthoses and protheses, which were conducted in a conventional manner but with use of hand sanitisers, gloves and adequate social distancing measures. After each candidate, the used instruments or materials were sanitised before the other candidate entered the examination room to avoid fomite transmission. The last part of the examination carried 20 marks and was based on the basics of COVID-19 pandemic and prevention of transmission during orthopaedic surgeries.
Short case sample showing clinical photograph of a two-day-old female patient.
| Potential questions |
| What is the diagnosis?Who was Ignacio Ponseti?What is club foot? What are the deformities in clubfoot?What are the aims of treatment?Describe the sequence of correction of deformities in Ponseti casting technique?How do you treat recurrences in CTEV?How do you manage neglected CTEV in a 2 year old child?How is Ponsetti method different from others?What is Kite’s error?What is the aetiology of club foot ?What is modified Pirani score?What is the brace protocol in corrected CTEV?How would you clinically assess a patient with club foot? |
 |
A questionnaire was designed to assess the virtual examination for safety during the pandemic, quality of case details, clinical photographs and radiological imaging, relevance to the academic training and overall satisfaction with the examination pattern.
The questionnaire also had a complaint section at its end so that the candidates as well as the examiners can state any complaint regarding this pattern of examination. The questionnaire was designed by the Head of the Department to oversee whether this pattern of examination can provide an alternative to the traditional examination or act as an adjunct to the latter. It comprised of 10 questions and each answer was graded on a three point Likert point scale, producing a maximum score of 2 and a minimum score of 0 for each question. After the examination was completed, questionnaire was distributed among all the examiners and their responses were recorded. Candidates on the other hand were called two days after the examination; questionnaire was distributed among them and their responses were recorded.
Statistical Analysis
Descriptive statistical analysis was performed. Mean of the data was calculated using OpenEpi program.
Results
The total score of examiner group was 73 out of 80 whereas that of candidate group was 169 out of 200 [Table/Fig-3]. Among all the questions included in the questionnaire, the safety of examination, quality of clinical and radiological photographs, and the overall satisfaction with the virtual examination rated highest in both the candidate and examiner groups.
Mean scores of the questions asked to candidate and examiner group (maximum possible score of 2 and minimum of 0 for each question).
| Questions score | Mean Score of Examiner Group | Mean Score of Candidate Group |
|---|
| Safety: How safe was the examination? | 2 | 2 |
| Were you comfortable with the seating arrangement made for you in the examination room? | 1.75 | 1.5 |
| Were the case details enough to make differential diagnosis? | 2 | 1.5 |
| Were clinical photographs good enough to elicit the clinical findings? | 1.75 | 1.9 |
| Were radiological pictures good enough to reach up to a provisional diagnosis? | 1.5 | 1.6 |
| Were the examiners/examinee familiar and comfortable with the examination pattern? | 2 | 1.7 |
| Was the examination pattern relevant to the academic training given during residency? | 2 | 1.6 |
| Would you recommend this examination pattern to other institutions? | 1.5 | 1.5 |
| Do you think a mock test helps/helped in preparing for this pattern of examination? ("helped" for examinee; "helps" for examiner) | 1.75 | 1.7 |
| Rate your overall satisfaction in for the ‘Zero-patient contact virtual practical examination’ | 2 | 1.9 |
| Total Score | 18.25 | 16.9 |
A common complaint by the candidate group regarding the pattern of examination was that there was more emphasis on management of the patients whereas in conventional pattern, diagnosis of the patient was considered more important.
Discussion
Assessment of the capabilities of a resident is a continuous process, and the end of training examination is only a part of the overall assessment [1]. There is great variation in the conduct of examinations among the different programs worldwide and within India [3-5].
The pattern of virtual examination is a relatively new and different pattern as compared to the conventional pattern of exit examinations for postgraduate residents in India. This study is among the very first studies done in this part of the world [1]. The pattern, though different, is relatively efficacious in checking the clinical knowledge and skills of the students required for safe orthopaedic practice. The clinical examination of an actual and real patient allows a multidimensional assessment of the patient which is not possible in this pattern with history, images and videos. While the use of actual patients replicates the real-world clinical settings, it can be extremely uncomfortable for the patients to be examined by a number of students [6].
The results of this study were comparable with the results of the study done by Malhotra R et al., with the resultant mean score of 48 (out of 50) and 43.43 (out of 50) for the examiner and candidate group respectively, [1] whereas it was 18.25 (out of 20) and 16.9 (out of 20) for the examiner and candidate group respectively in this study. Another study done by Iyenger KP et al., in which they concluded that comparative studies between traditional and virtual modes of conducting practical postgraduate examination models will have to be undertaken to ensure an even platform of assessment [2].
One important aspect of this study was the availability of a volunteer which made it easy for the examiners to assess the demonstration of clinical methods. No volunteer was kept in the examination pattern demonstrated by Malhotra R et al., [1]. The observation regarding the availability of a volunteer is that it helps in the assessment of clinical skills part of the examination which cannot be assessed by virtual means.
Recently, studies have been done regarding virtual training of residents but the literature is scarce about information on virtual examination for final year postgraduate exit examination in medical colleges of India [1,2]. Schwartz AM et al., concluded that orthopaedic residencies are faced with new challenges to provide care and educate residents in the face of safety, resource redistribution, and erosion of classic learning opportunities. Hence, an adaptive approach aims to be a generalisable tactic to optimise the current landscape [6]. Similarly, Chick RC et al., in their study found that innovative solutions utilising technology may help to bridge the educational gap for surgical residents during this unprecedented circumstance [7].
Limitation(s)
First, limitation was that the number of participants (candidates and examiners) was less, so additional studies in different states need to be done to establish the reproducibility of the facts established. Second, was the fact that patient communication is an important dimension of the conventional exit examination which could not be tested in virtual examination. Third, was the availability of resources like good quality photos of radiographs, and videos. In this study, though the faculty had enough material to compile the same, it could not be the scenario with other institutions. Free accessible images on the internet will have copyright issues with them. Fourth, is the non-availability of devices like interactive screen because of the limited resources available. Interactive screens add a new dimension to this pattern as the student will be able to pin point as well as draw certain landmarks asked by the examiner. Fifth was the issue that with examining a normal volunteer, it is pretty easy to monotonously learn an examination technique (for example, learning through an anterior drawer test or a Lachman test for an Anterior Cruciate Ligament (ACL) injury). The real difficulty is to examine and find whether it’s an ACL injury or Posterior Cruciate Ligament (PCL) injury or both, which cannot happen in a virtual examination. Finally, the clinical case scenarios used in the examination were prepared by faculty with help of the examiners. Therefore, the approval ratings of the examiners might be biased.
Conclusion(s)
This virtual pattern of examination may be a viable interim alternative to the traditional face to face examination, though it may not replace the latter. Since, there is no time frame when this pandemic will end and unnecessary delay in the examination of residents would demoralise them and hamper their career progression, so this model of examination may be further implemented elsewhere to see if it can be a useful adjunct to the traditional examination.