Over the last few decades, there is a changing trend towards WPBA for evaluating the competency of PGs. These assessment techniques have also been evaluated in other specialties such as midwifery and dentistry [1,2]. The PG residency training is largely based on the traditional observer model i.e., learning by observing teachers and seniors. However, assessment of PGs, while they are actively engaged in patient’s related clinical activities is not often a common practice. The long case presentation evaluates case presentation skills of students rather than their actual clinical skills. The apex body regulations clearly mention the PGs’ training in India as competency-based [3].
Mini-CEX, designed by the American Board of Internal Medicine (ABIM) in 1995 has been found to be a valid and reliable tool to assess the clinical competencies of trainees [4]. The Mini-CEX was designed to meet the necessity of clinical skills acquisitions and professional communication between PGs and faculty [4]. It is an effective tool to assess the PGs’ for the actual performance (“does”). A 6-8 encounter Mini-CEX has a reliability of 0.83, which is much higher than a long case or an Objective Structured Clinical Examination (OSCE) of a similar duration [5]. A composite reliability coefficient of 0.8 was observed in the study of Moonen-van Loon JM et al., for eight encounters of Mini-CEX, nine of DOPS (Direct Observation of Procedural Skills), and nine of Multi-Source Feedback (MSF) over a period of one year [6].
Materials and Methods
The educational intervention study was conducted in the Department of General Surgery, of a Tertiary Care Centre attached to a medical college in central India between April 2016 to March 2017. Convenient sampling was used. Study participants were 16 PGs while 13 faculty members evaluated them. Written consent was taken prior to the study from all the students. The faculty participated voluntarily. The permission to implement the study was granted by the Institutional Ethics Committee vide letter No: PCMS/OD/2016/887 Code No: IEC- 2016/04 dated 24.05.2016 People’s College of Medical Sciences & Research Center, Bhopal, Madhya Pradesh, India. Direct observation of PGs during patients’ clinical encounters using Mini-CEX proforma with immediate specific feedback was done. The sensitisation sessions were conducted separately for the students and faculty. The PG sensitisation was done by an interactive small group discussion, which was followed by the role play, and the plenary session in the end. A faculty discussion was held separately and the Mini-CEX generic form available from free downloads abim.org. was used [8]. Subsequently, implementation methodology was discussed and a consensus was reached. Students and faculties were encouraged to have exercises in different clinical settings like ambulatory care (OPD), indoors and the emergency ward. The standard protocol was followed for the Mini-CEX encounters as described by Norcini JJ et al., [9]. The encounter day, place of encounter, and the complexities of the cases were decided by the faculty and the PGs. It was planned to have nine encounters per PG with a focus on four competencies: medical interviewing skills, physical examination skills, counselling skills and professionalism in CEX. The generic form was used for recording the faculty observations during the Mini-CEX encounters. The completed Mini-CEX forms were collected and records were maintained chronologically. On completion of one year, feedback from PGs and faculty was obtained using the predesigned validated questionnaire, with close-end questions on a 5 point Likert Scale (1=strongly disagree, 2=disagree, 3=neither agree nor disagree, 4=agree, 5=strongly agree) and a few open-end questions.
The first two encounters being introductory were considered as preparatory for the study. All the PGs completed seven Mini-CEX encounters, only a few completed 8-9 encounters. Therefore, only five Mini-CEX encounters (E3-E7) scores were considered for the Friedman test to measure the change in the levels of clinical competencies acquired by the PGs.
Statistical Analysis
The data was analysed using descriptive statistics including mean, median, and percentages. The effectiveness of Mini-CEX was evaluated by using serial ‘global rating scale’ scores of the third to seventh Mini-CEX encounters of every PG student. The Friedman test (Repeated-Measures) was applied to calculate the p-values. The test relies on the rank-ordering of the data and allows to evaluate the differences between three or more repeated evaluation scores. A p-value of <0.05 was taken as statistically significant. Data were analysed using IBM SPSS 20.
Results
The median (mean±SD) time taken for observation in the encounter was 10 (13.01±4.13) minutes, with the range from 3 to 25 minutes, respectively. The median (mean±SD) feedback time was 5 (6.6±4.26) minutes, with the range from 2 to 20 minutes, respectively [Table/Fig-1].
Overview of Mini-CEX encounters with surgery postgraduates (PGs).
| No. of postgraduates (n=16)# | M=15 (93%)F=1 (7%) |
| Number of clinical encounters during PG training (n=124)(Individual student=7-9) | First year PG=45 (36.29%)Second year PG=49 (39.52%)Third year PG=30 (24.19%) |
| Clinical settings | OPD=58 (46.77%)Surgery wards=49 (39.52%)Emergency ward=17 (13.7%) |
| Case complexity | Low=10 (8.06%)Moderate=96 (77.41%)High=18 (14.51%) |
| Mean time taken (range) minutes | Observation time=13.01±4.13 (3-25)Feedback time=6.6±4.26 (2-20)Total encounter time=5-40 |
| The mean satisfaction score (scale of 0-9) | Postgraduates=7.3±0.88Faculty=6±0.89 |
#4 PGs joined in First-year while 4 PGs passed out; encounters were recorded as per the year of PG training
Fourteen out of 16 PGs (87.5%) accepted that Mini-CEX was easy to conduct as compared to the traditional long case. About seven PGs expressed that the Mini-CEX exercise increased their anxiety. About 12 PGs felt that the exercises were effective for their clinical learning [Table/Fig-2].
Postgraduates (PGs) feedback on 5-point Likert scale.
Perception of PGs on Mini-CEX (Verbatim responses to open end items)
“Mini-CEX helps us to overcome our fears when it comes to the real world and we are actually watched!”
“This tool helps us to improve the trainee-patient interaction skills effectively”
“Mini-CEX is acceptable to us and should be widely practiced in OPDs”
All the Faculty expressed that Mini-CEX is simple to plan as compared to traditional long cases, and 10 accepted that it was feasible to be conducted with the prevailing professional workload. All the faculty expressed that they will continue using Mini-CEX and recommend the tools to others for use [Table/Fig-3].
Faculty feedback on a 5-point Likert scale.
Perception of Faculty on Mini-CEX (Verbatim responses to open end items):
“Mini-CEX is quick and easy to organise, less time consuming, doesn’t interfere with routine clinical work”
“The deeper knowledge and competencies are difficult to judge by this tool, and traditional long cases are better suited for this”
“Mini-CEX may be made mandatory in the PG and UG Curriculum”
It was observed chronologically during the Mini-CEX encounters that there was a progressive increase in the skills acquisitions for the competencies, as is evident by the PGs group mean, median of the global ratings, and the ranks. The acquisition of clinical skills by the PGs was found to be statistically significant for all the four competencies focussed during the study [Table/Fig-4].
Improvements in clinical competencies of the Postgraduates (PGs) (n=16).
| Competencies | Parameters | Global rating scores of Mini-CEX encounters | Friedman value | p-value |
|---|
| E3 | E4 | E5 | E6 | E7 |
|---|
| Medical interviewing skills | Mean (SD) | 4.5 (0.6) | 4.8 (0.8) | 5.1 (1.02) | 5.6 (0.79) | 5.5 (0.72) | 24.9 | 0.001 |
| Median | 4 | 5 | 5 | 6 | 6 |
| IQR | 4-5 | 4-5.75 | 5-6 | 5.25-6.5 | 5-6 |
| Rank | 1.94 | 2.34 | 3 | 4 | 3.72 |
| Physical examination skills | Mean (SD) | 4.5 (0.81) | 5.1 (1.3) | 5.6 (0.88) | 5.7 (1.3) | 5.9 (1.06) | 24.31 | 0.001 |
| Median | 5 | 4.5 | 5.5 | 6 | 6 |
| IQR | 4-5 | 4-6 | 5-6 | 4.25-6 | 6-7 |
| Rank | 1.59 | 2.81 | 3.44 | 3.41 | 3.75 |
| Counseling skills | Mean (SD) | 3.9 (0.68) | 4.6 (1.0) | 5 (1.2) | 5.3 (1.3) | 5.1 (0.62) | 23.79 | 0.001 |
| Median | 4 | 5 | 5 | 6 | 5 |
| IQR | 3.25-4 | 4-5.75 | 4.25-6 | 4.25-6 | 5-5.75 |
| Rank | 1.63 | 2.75 | 3.44 | 3.72 | 3.47 |
| Professionalism | Mean (SD) | 3.9 (0.68) | 4.2 (1.0) | 4.4 (1.4) | 5 (1.06) | 4.9 (0.57) | 17.91 | 0.001 |
| Median | 4 | 4.5 | 5 | 5 | 5 |
| IQR | 3.25-4 | 3-5 | 3.25-5.75 | 4-6 | 5-5 |
| Rank | 2.06 | 2.56 | 2.84 | 3.88 | 3.66 |
p-value <0.05 was considered statistically significant; IQR-Interquartile range
Discussion
It was observed from present study that Mini-CEX was feasible and well acceptable in our settings, without any extra resources. Thirteen faculties from the Department of Surgery conducted a total of 124 Mini-CEX encounters with 16 PGs, with a range of 7-9 encounters for every Postgraduate during the study period. The majority of PGs were male (93%). A total of 30 (24.19%) encounters were conducted during the third year, 49 (39.52%) during the second year, and 45 (36.29%) during the 1st year postgraduate training period. An encounter completion rate of 86.11% was observed in present study. A completion rate of 64.4% was recorded in the study of Singh T and Sharma M, 93.3% by Khalil S et al., and 66.7% by Ramula M and Arivazagan N [Table/Fig-5] [5,10-12].
The completion rate of Mini-CEX in the research studies [5,10-12].
| Mini-CEX completion rate (%) | Research study |
|---|
| Singh T and Sharma M 2010 [5] | Khalil S et al., 2017 [10] | Gupta S et al., 2017 [11] | Ramula M and Arivazagan N 2018 [12] | Present study |
|---|
| 64.4 | 93.3 | 50 | 66.7 | 86.11 |
The encounters were conducted in different clinical settings in present study, 58 (46.77%) in the OPD, 49 (39.52%) in the surgical wards, and 17 (13.7%) in the emergency ward. In the study by Goel A and Singh T, Mini-CEX was conducted in different settings- OPD (38%), wards (22%), PICU (19%), NICU (15%) and casualty (6%) [13]. In the study of Singh T and Sharma M, most of the encounters were conducted in Outpatients Department [5].
The majority of the cases selected in present study were of moderate complexities (77.41%) followed by high complexity (14.51%) and low complexity (8.06%). In present study, the mean time taken for observation was 13.01±4.13 minutes. The mean feedback time was 6.6±4.26 minutes. The mean time for observation was 12.3 (8-30 min), and feedback was 4.2 (3-10 min) in the study of Joshi MK et al., [14] Norcini JJ et al., in their study recorded the mean time for observation was 15 minutes, and the time for feedback to the resident was 5 minutes [15]. In the study of Merseh E et al., the observation time was 14.24 minutes, and the feedback time was 9.71 minutes; whereas in the study of Deshpande S et al., the observation time was 15.25 minutes, and the feedback time was 11.20 minutes. In present study, the minimum time and maximum time taken for one Mini-CEX exercise was 5 minutes and 40 minutes, respectively [Table/Fig-6] [5,14-17].
The encounter time observed in the research studies [5,14-17].
| Mini-CEX time | Research study |
|---|
| Norcini JJ et al., 2003 [15] | Singh T and Sharma M 2010 [5] | Joshi MK et al., 2017 [14] | Merseh E et al., 2018 [16] | Deshpande S et al., 2018 [17] | Present study |
|---|
| Mean observation time (minute) | 15 | 22 | 12.3 | 14.24 | 15.25 | 13.01 |
| Mean feedback time (minute) | 5 | 10 | 4.2 | 9.71 | 11.20 | 6.6 |
Low motivation of faculty along with time constraints were the initial hurdles. A few of the faculty expressed difficulty in conducting and assessing the Mini-CEX encounters during the initial stages. Few of the faculty felt that practically it takes more time than what is planned for the encounters. We believe, maybe a little more on the faculty training could resolve such issues. Lack of proper training as one of the limitations in the acceptance of Mini-CEX has also been observed by other authors [5,18-20]. But, as the study progressed, it was observed that these difficulties were resolved and the faculty group expressed that Mini-CEX was feasible and acceptable in our settings in the day-to-day hospital activities like OPDs, ward rounds, thus generating a learning environment without compromising the hospital services.
The feasibility of Mini-CEX was better than the traditional assessment methods as it was being carried out during the course of routine clinical work. Though initially, it requires faculty training and extra time, there is no requirement for additional infrastructure, and hence is a cost-effective assessment tool [21]. Regular training of assessors through workshops on the provision of effective feedback is also required [22]. The UK General Medical Council in Good Medical Practice mentioned that the teachers essentially must-have the skills, attitudes, and practices of a competent teacher to discharge the responsibility for teaching [23]. We feel the important considerations and limitations of the rater-based judgments are issues of observer’s self-style, individual bias, and immediate environmental factors. The specific narratives and contextual comments are better informative which justify the rater judgments of the trainees [24].
Mini-CEX was entirely a novel experience for the students. In the beginning, almost 44% of them were anxious due to direct observation of their clinical skills by the faculty in real-time settings. As the study progressed, their anxiety levels reduced, with the realisation that the exercises provided immediate constructive feedback with contextual learning opportunities for the gaps identified in clinical skills as they were directly observed by faculty. Malhotra S et al., in their study also observed that the anxiety level reduced as the familiarity with the exercise increased [25]. The students’ anxiety was insignificant and the acceptance of Mini-CEX by students was good in the study of Goel A and Singh T [13]. In present study, the seniority of the assessor influenced the quality of feedback during Mini-CEX. The direct one to one interaction was perceived as an informal learning environment and was liked by them, and they further expressed that it strengthened the student-teacher professional relationship. Similar were the findings observed in the study of Kogan JR et al., [26]. In present study, all the PGs expressed that Mini-CEX exercises motivated them and the feedback provided was effective for learning clinical skills. Similar findings were observed in the study of Singh T and Sharma M [5,11,13].
The mean satisfaction scores with Mini-CEX exercise were 7.3±0.88 and 6±0.89 on a scale of 0-9, among the PGs and faculty groups. This is a good indicator of their positive feelings and its acceptability. Goel A and Singh T, recorded a mean rating score for satisfaction 7.4 out of 9 for the students and 7.1 for the teachers [Table/Fig-7] [13,16,17,22].
Mini-CEX Satisfaction rating in the research studies [13,16,17,22].
| Mini-CEX satisfaction rating (out of rating og 9) | Research study |
|---|
| Goel A and Singh T 2015 [13] | Merseh E et al., 2018 [16] | Deshpande S et al., 2018 [17] | Moudgil T et al., 2019 [22] | Present study |
|---|
| Mean satisfaction rating of the students | 7.4 | 7.26 | 8.34 | 7.7 | 7.3 |
| Mean satisfaction rating of the faculty | 7.1 | - | 7.23 | 8 | 6 |
The high satisfaction index among PGs and faculty in present study and in other Indian studies is a favourable sign in the light of the thrust of regulatory bodies towards competency-based medical education and WPBA [27].
We found a statistically significant improvement in the competencies of medical interviewing skills, physical examination skills, counseling skills and professionalism. Similar were the findings of Goel A and Singh T and Singhal A et al., [13,27]. Hejri SM and Jalili M concluded that mini-CEX has reasonable validity, reliability and educational impact [28].
Limitation(s)
Mini-CEX was introduced for the first time in the Surgery Department. Being a new experience for the faculty and PGs, it took some time to get used to it. The actual number of encounters held was less than the total number planned.
Conclusion(s)
Mini-CEX was found to be acceptable by the PGs and faculty in our setting. It was found to be an effective tool for formative assessment of PGs in identifying the gaps in knowledge and skills. The specific feedback after the Mini-CEX encounters improved PGs’ clinical skills and acquisition of desirable competencies. Mini-CEX may be practiced in the Outpatient Department, emergency room, inpatient wards, and Intensive Care Units. Mini-CEX is an effective workplace-based tool for the clinical skills learning of the PGs.
#4 PGs joined in First-year while 4 PGs passed out; encounters were recorded as per the year of PG trainingp-value <0.05 was considered statistically significant; IQR-Interquartile range