Academic Achievement of First-year Medical Students using Self-directed Learning with Formative Assessment vs Conventional Teaching Method: A Longitudinal Study

1. Associate Professor, Department of Anatomy, Medical College, Kolkata, West Bengal, India. 2. Associate Professor, Department of Anatomy, Medical College, Kolkata, West Bengal, India. 3. Junior Resident, Department of Anatomy, Medical College, Kolkata, West Bengal, India. 4. Junior Resident, Department of Anatomy, Medical College, Kolkata, West Bengal, India. 5. Junior Resident, Department of Anatomy, Medical College, Kolkata, West Bengal, India. 6. Senior Resident, Department of Anatomy, Medical College, Kolkata, West Bengal, India.

Abstract

Introduction: The Self-Directed Learning (SDL) approach is an important component of the Competency-Based Medical Education (CBME) curriculum. However, learning independently can be challenging, even for the brightest and most motivated students. Moreover, self-evaluation is quite difficult. If SDL is used in combination with formative assessment and constructive feedback for individual topics, it would facilitate self-modulation by learners and guide them towards higher academic achievement.

Aim: To compare the academic performance of first-year medical students in Anatomy Practical class using SDL with formative assessment and feedback versus conventional teaching methods.

Materials and Methods: A comparative evaluation was conducted using a longitudinal study design in the Department of Anatomy, Medical College Kolkata, West Bengal, India, from November 2022 to June 2023. During Anatomy Practical classes, 250 first-year medical students, (Batch 2022-2023), were divided into two equal groups using simple random sampling through a lottery method. One group (study group) was taught by implementing SDL followed by formative assessment with immediate feedback, and the other group was taught by the traditional method (control group). Following the completion of each of the first three consecutive regions (superior extremities, inferior extremities, and thorax) during Anatomy Practical classes, summative assessments were conducted. The collected data were calculated in an Excel sheet and computed using JAMOVI software (free version). Independent t-test statistics for the three summative assessments were evaluated, and the p-value of each assessment was determined. A p-value <0.05 was considered to be statistically significant. Additionally, students’ feedback was documented and analysed.

Results: It was observed that there was an improvement in academic scores in both groups. However, higher academic performance was progressively attained in the group which was provided SDL sessions and formative assessment with immediate feedback. A total of 55 (80%) students passed the third summative assessment in comparison to 40 (59.7%) students of the control group who passed the same exam. Differences between the means of the control and study group were found to be statistically significant (p-value ≤0.001 and 0.023 in two summative assessments).

Conclusion: The present study revealed that SDL with formative assessment followed by immediate feedback during Anatomy Practical classes had a remarkable positive impact on students’ academic performance compared to those students taught using conventional teaching methods and might be implemented in the future for better learning and improved academic achievement of students.

Competency-based medical education, Regional anatomy, Self-evaluation

The SDL is a learner-centric method of learning, where the learner has sole responsibility for the learning process (1),(2). In fact, deep learning and lifelong learning of students may be promoted through the process of SDL. SDL and formative assessments with feedback are important pillars of the CBME curriculum (3). Moreover, SDL plays a vital role in medical education as it gives importance to reflection (4).

Studies have revealed a positive correlation between SDL and higher academic achievement among medical students. Tekkol I?A and Demirel M observed in their study that a moderate and positive relationship existed between SDL skill and lifelong learning (5). They recommended the need to design instructional environments in such a way that it would lead to improved self-control skills and also advocated the need to read reflective journals to achieve upper cognitive learning strategies. Agrawal P and Verma N observed that small group discussion with SDL was helpful for students to achieve higher academic performance (6). In fact, in their study, Chaudhuri A et al., proved that SDL with interactive lecture classes was more effective than the conventional teaching method (7). Several similar studies (1),(8),(9) also revealed that SDL was more effective than the conventional teaching method for medical students to attain academic improvements. In addition, Srivastava TK et al., described in their study that formative assessment classroom techniques help to diagnose learning issues and facilitate timely feedback to improve learning (10).

However, studies conducted utilising SDL with formative assessment followed by immediate feedback in the setting of practical classes, followed by a record of its impact on students’ academic performance, are relatively scarce, especially in the eastern zone of India. Keeping this in mind, the present study was undertaken in order to bridge this gap and to provide useful and relevant information to medical educators for the implementation of SDL with formative assessment for better academic achievement of undergraduate medical students.

The aim of the study was to compare the academic performance of first-year medical students in Anatomy Practical class using SDL with formative assessment and feedback versus the conventional teaching method. The objectives of the study were to determine students’ academic performance in Anatomy Practical classes after incorporating SDL with formative assessment for:

a) Understanding and application of cognitive knowledge of demarcated learning areas.
b) Identification of the structures in dissection classes.

Material and Methods

A comparative evaluation of Anatomy Practical classes by means of SDL with formative assessment versus conventional teaching methods with a longitudinal study design was conducted on 250 first-year medical students (Batch 2022-2023) of Medical College, West Bengal, India from November 2022 to June 2023 following ethical clearance (MC/KOL/IEC/NON- SPON/1758/01/2023).

Inclusion criteria: First-year medical students of Batch 2022-2023 who were willing to participate in this study and gave their informed consent were included. Also, students who had at least 75% attendance in dissection classes during the given study period (accepted as a baseline level after discussion with other faculty members of the Anatomy department) were included.

Exclusion criteria: Students who had less than 75% attendance in dissection classes during the given study period and those who were unwilling to participate in this study were excluded.

Sample size calculation was done using the formula:

n=f(α/2, β)×{p1×(100-p1)+p2×(100-p2)}/(p2-p1)². Where p1 and p2 are the percentages of ‘success’ in the control and experimental groups respectively, and f(α,β)={F-1 (α)+F-1 (β)}2. A minimum sample size of 86 students is required to have a 90% chance of detecting, as significant at the 5% level, an increase in the primary outcome measure from 0% in the control group to 20% in the experimental group (11).

Procedure

The present study was conducted in the Department of Anatomy during practical classes. The first-year medical students were divided into two equal groups by simple random sampling using a lottery method, following which one group was taught by implementing SDL followed by formative assessment with immediate feedback and the other group was taught by the traditional method.

SDL topics were introduced by two approaches, namely: i) topic-wise (e.g., cubital fossa, back of thigh, intercostal spaces to promote understanding); and also ii) as problem-based sessions (e.g., case-based scenarios of claw hand, foot drop, thoracic inlet syndrome to promote reasoning skills and application of knowledge). Criteria for the selection of topics included the ease of understanding for students, the clinical aspect of the region, and importance for the purpose of their exams. Three SDL topics were chosen for each 6region. In this manner, nine SDL sessions were conducted for the topics of cubital fossa, claw hand, and back of the arm (superior extremity), adductor canal, back of thigh, and foot drop (inferior extremity), intercostal spaces, thoracic inlet syndrome, and lungs (thorax). The rest of the sessions of the study group were conducted using the conventional method.

All the practical sessions of the control group, on the other hand, were carried out using the conventional method. The elements of the teaching-learning methods used in the study group and control are shown in (Table/Fig 1).

(Table/Fig 1) highlighted the additional elements of the teaching-learning method used in the study group versus the control group.

The method of conduction of SDL was adapted from the study of Sachdeva K and Mahajan A (12). Prior to the conduction of SDL, faculties and students were sensitised regarding the SDL sessions. Students were provided with the topic of SDL, learning areas, and the references for study one week before the SDL sessions and were instructed to bring those study materials during their designated SDL sessions. At the start of the two-hour dissection classes, students of the study group were divided into 10 batches, with each batch consisting of 12-13 students. A total of five facilitators were involved during these sessions, with two groups allotted to each facilitator. SDL sessions were followed the next day by cadaveric demonstration of the corresponding region to consolidate students’ knowledge.

SDL sessions were followed by formative assessment with immediate feedback. Study investigators divided formative assessment into two sections, namely: a) assessment of cognitive knowledge (theory paper); b) assessment of psychomotor skills (oral/practical examination). The method used by Hafez SA was adapted (13). Formative assessments were followed by immediate feedback. Results of formative assessments with feedback are shown in (Table/Fig 2).

Summative assessments were taken for the entire batch to see their academic improvement after the completion of each region of Anatomy. It included both theory examination and practical examination to assess the cognitive and psychomotor domains. For the assessment of the psychomotor domain by Objective Structured Practical Examination (OSPE), the authors adopted the method used by Vishwakarma K et al., (14). Altogether three summative assessments (S1: first summative assessment; S2: second summative assessment; S3: third summative assessment) were conducted for the three regions of anatomy covered during the entire study period. Mean scores were calculated by adding all scores of students who appeared for each summative assessment and then dividing by the total number of students who appeared for that particular summative assessment. Students who did not appear for the summative assessment, however, were excluded during the computation of the mean score.

The results were collected and compared with the control group. In addition, students’ feedback was recorded at the end of the study. A total of 100 students’ feedbacks were recorded, and thematic analysis was done, as shown in (Table/Fig 3).

Statistical Analysis

The collected data was calculated in an Excel sheet and computed using JAMOVI software (free version). The difference between the scores of the control and study groups was evaluated by an independent student t-test, and p-values were calculated.

Results

(Table/Fig 2) revealed that formative assessment with immediate feedback reinforced cognitive knowledge and improved skills of identification of pinned structures. Mistakes were also immediately corrected.

Students’ feedback (n=100) was also recorded, as shown in (Table/Fig 3). A total of 79 (79%) students agreed that SDL with formative assessment increased their eagerness for learning, 81 (81%) of them agreed that they had a better understanding of their strengths and weaknesses as a learner. A total of 75 (75%) students agreed that they achieved their study target by means of SDL with formative assessment, while 18 (18%) did not know whether they met their study targets by this method. Additionally, 7 (7%) students opined that they did not attain their study targets by means of SDL with formative assessment.

In the present study, positive feedback was recorded from students regarding the current learning method in the study group. It was found that 79 (79%) students agreed that SDL with formative assessment increased their eagerness for learning, but 7 (7%) students opined that they did not attain their study targets by means of SDL with formative assessment (Table/Fig 3).

On the stipulated date of the three summative examinations, an average of 82 students in the study group {n(S1)=88, n(S2)=89, n(S3)=68 where n(S) denotes the number of students who appeared in the summative assessment}, as compared to an average of 78 students in the control group {n(S1)=86, n(S2)=82, n(S3)=67 where n(S) denotes the number of students who appeared in the summative assessment}.

It was observed that there was an improvement in academic scores in both groups. However, a greater number of students in the group which was provided SDL sessions and formative assessment with immediate feedback passed the examinations compared to the control group. This is highlighted in (Table/Fig 4), which shows that in the first summative assessment (S1), 35 (39.8%) students of the study group passed the test (i.e., obtained >50% marks) compared to 37 (43.02%) students of the control group. In contrast, records of the second (S2) and third (S3) summative assessments showed that a greater number of students in the study group passed the tests - S2: 63 (70.7%) and S3: 55 (80%) compared to students in the control group - S2: 41 (50%) and S3: 40 (59.7%). Thus, three consecutive assessments showed that performance was likely to improve with the repetition of SDL.

Statistical analysis of academic scores is displayed in (Table/Fig 5). It showed that the mean value of the score of S1 was 9.01 with a standard deviation of 3.16 in the study group and 9.38 with a standard deviation of 2.91 in the control group. In S2 and S3 examinations, the scores were as follows: mean value 11.9 with a standard deviation of 3.77 and 13.2 with a standard deviation of 4.17 in the study group, while in the control group, the scores of S2 and S3 had mean values of 10 with a standard deviation of 3.26 and 11.4 with a mean value of 4.92, respectively.

Additionally, the differences of means between the two groups were calculated by an independent student’s t-test and were found to be statistically significant in S2 (p≤0.001) and S3 (p=0.023) examinations, as shown in (Table/Fig 6).

Discussion

In the present study, the authors observed that academic performance was better in the study group compared to the control group. Academic improvement was recorded among 55 (80%) students in the study group at the time of the third summative assessment (Table/Fig 4). Agrawal P and Verma N conducted a comparative study to assess students’ academic performance before and after discussion in an SDL module in small group teaching learning method (6). They observed an overall mean performance improvement of students to be about 78.3%-87.7%.

Several previous studies (15),(16) were conducted to assess the academic impact of SDL sessions. Pryce-Miller M carried out a cohort study among 450 nursing students and observed that out of 450 students, 245 students had awareness about SDL. They concluded that SDL was an essential component in the learning process for nursing students to cope with continuously changing knowledge and handle the healthcare environment in a better manner (15). Abraham RR et al., also observed a positive impact of SDL sessions. They compared lecture examination scores with SDL exam scores (72±0.40 vs. 76±0.21) and proved that the lecture exam score was significantly low (16).

Similar to Sachdeva K and Mahajan A, the present study utilised MCQ and SAQ tests to evaluate the knowledge gained by incorporation of SDL sessions (12). They observed that the mean scores of most students increased remarkably after the SDL sessions. In the present study, the academic performance of students in the study group increased after incorporation of SDL, formative assessments, and feedback (Table/Fig 4).

In the present study, SDL sessions were incorporated as: i) topic-wise, and ii) Problem-Based Learning (PBL), supporting the view of Pawlina W et al., who stated in their study that the main objective of PBL sessions was to teach problem-solving techniques and to increase learners’ interest in practical sessions (17). In the current study, PBL was implemented by providing case-based scenarios to promote reasoning skills and the application of knowledge.

Nasri NM et al., suggested a few important elements regarding feedback and assessment (18). They mentioned that timely feedback and sustainable formative assessment were needed to promote SDL. Moreover, Patra S et al., collected feedback from 130 students out of 160 students in their study (19). They observed from recorded feedback that 67% of students were satisfied with the SDL module, while 70% of students mentioned the program was interesting, and 60% of students agreed to further study on the allotted topic. Sachdeva K and Mahajan A also found in their study that participating students showed high readiness towards SDL and were oriented to become lifelong learners (12). Charokar K and Dulloo P, mentioned in their article that SDL sessions helped students to build up leadership skills and time management (20). Palve S and Palve S observed in their study that SDL sessions with formative assessments were more beneficial than didactic lectures for understanding the subject and acquiring knowledge (21). The authors of the present study observed a similar result. Lu SY et al., also obtained positive results after implementing SDL in a blended teaching approach compared to the conventional didactic teaching method (22). The results of the present study supported the findings of Lu SY et al., (22).

Limitation(s)

The present study was conducted in only one medical college in Eastern India. It would be relevant to carry out a multi-centric study in the future with a larger sample size and a wider representation of students.

Conclusion

The present study revealed that SDL with formative assessment, followed by immediate feedback during Anatomy Practical classes, had a positive effect on students’ academic performance in comparison to those students taught by conventional teaching methods. It might be implemented in the future during dissection classes for better learning and improved academic achievement of students. In addition, the application of this method might guide students towards SDL and lifelong learning. Although the authors observed an academic improvement of students using SDL and formative assessment from this study, a 100% involvement of the students was lacking. From this observation, the authors recommended the Sensitisation program for students regarding SDL and Faculty development program regarding SDL.

Acknowledgement

The authors would like to extend their gratitude to the faculties of the Departments of Anatomy and Community Medicine, Medical College, Kolkata, West Bengal, India, whose support facilitated smooth implementation of the new module. Above all, this study is dedicated to our beloved students who are the driving forces of all our efforts as well as the very fruits of our labour.

DOI and Others

DOI: 10.7860/JCDR/2024/66565.19033

Date of Submission: Jul 18, 2023
Date of Peer Review: Oct 07, 2023
Date of Acceptance: Dec 28, 2023
Date of Publishing: Feb 01, 2024

AUTHOR DECLARATION:
• Financial or Other Competing Interests: None
• Was Ethics Committee Approval obtained for this study? Yes
• Was informed consent obtained from the subjects involved in the study? Yes
• For any images presented appropriate consent has been obtained from the subjects. NA

PLAGIARISM CHECKING METHODS:
• Plagiarism X-checker: Jul 19, 2023
• Manual Googling: Dec 18, 2023
• iThenticate Software: Dec 27, 2023 (4%)

ETYMOLOGY: Author Origin

EMENDATIONS: 7