Assessment is a systematic and methodical process that attempts to analyse the extent to which the learning objectives have been attained [1,2]. Assessment must be objective, reliable and valid in order to ensure learning [3]. Written examination is effective to assess the cognitive domain but the psychomotor and affective part remains obscure. Thus, practical examination becomes an integral part of the assessment in medical colleges [4]. Majority of medical schools conduct practical examination by traditional methods. Viva voce is a vital component in such a scenario that lacks objectivity, reliability and carries the risk of teacher bias. The marks given by such methods are generally based on overall performance of the student and not on the individual skills [5]. In addition, conventional method of examination is time consuming. Thus, there is a need to incorporate some innovative methods for practical examination in pre and para clinical subjects.
OSPE was derived from Objective Structured Clinical Examination (OSCE) in 1975 and later modified in 1979 to improve practical assessment in pre and para clinical subjects [6,7]. This method has proved to be quite effective in eliminating the drawbacks of conventional practical assessment methods. Various studies have demonstrated that the methods adopted for assessment influence student learning [8-10].
OSPE is a tool to assess various components of practical skills such as basic procedures, interpretation of laboratory values, communication and attitude [10]. A number of studies have been conducted earlier which demonstrate the effectiveness of OSPE as an assessment tool in pre and para clinical subjects. However, there is paucity of data and this study was planned to cover the lacunae observed in previous studies like assessing practical and communication skills as well as attitude of students towards OSPE.
The present study was conducted with the aim of introduction and evaluation of OSPE and its perception among second year medical students in Pharmacology so as to make necessary modifications in the conventional method of practical examination. The student feedback obtained from this study was used to compare OSPE with CPE. This study will serve as a preliminary step towards achievement of the goal of Competency Based Medical Education (CBME).
Materials and Methods
The present cross-sectional study was conducted on second professional MBBS students of 2018 batch from March 2020 to September 2020 in the Department of Pharmacology of a tertiary care teaching hospital. Ethical approval was obtained from the Institutional Ethics Committee (Letter number-No. MGMCH/IEC/JPR/2020/96 dated 12th June, 2020) before starting the study. Since OSPE was introduced for the first time, pattern of examination and demonstration of various procedures was done by trained faculty members to all the students four weeks prior to the date of assessment. The assessment was conducted by skilled faculty and staff of the Department of Pharmacology. The OSPE questions were designed for each station along with the answer key and checklist as applicable. Validity of the study was assessed through focus group discussions among faculty in the department to determine that the questionnaire measured what it was supposed to measure. A pilot study was done on a small group of students beforehand to assess the feasibility of the study and necessary modifications were made. The participants were explained about the entire procedure and an informed consent was obtained.
Inclusion and Exclusion criteria: All the students of second professional MBBS who were willing to participate were included in the study. The students who were enrolled in the pilot study and those not willing to participate in the study were excluded from the study.
A total of 122 students were included in the study. They were divided into four batches (batch I and II comprised of 31 students each while batch III and IV comprised of 30 students each) and the assessment was done for four consecutive days. The pattern of examination, number of stations, and time allocated for each station was explained to all the students. The topics included in the study were related to aspiration of drug from ampoule into syringe, demonstration of correct technique of drug administration by different routes on mannikins, identification of experimental animals and its uses, Latin abbreviations, labelling of drugs, adverse drug reactions and drugs of choice for poisoning and emergency conditions.
The OSPE comprised of 10 stations out of which five were procedure stations (observed) while the other five stations were unobserved in the form of spotters. The time allocated for each station was five minutes. The checklist was handed over to the faculty member (observer) present at each observed station. Observed stations consisted of aspiration of the drug from the ampoule into the syringe, demonstration of various techniques of drug administration by intravenous, intramuscular, subcutaneous, and inhalational routes on mannikins. The questions at unobserved stations were related to clinical pharmacy, clinical pharmacology, and experimental pharmacology.
At the end of the examination, student’s opinion towards OSPE was obtained using a feedback questionnaire. It comprised of questions related to cognitive, psychomotor, and affective domains. In addition, questions related to the overall perception of students towards OSPE as an assessment tool were also included in the feedback questionnaire.
A pilot study on 15 students was conducted twice before starting the study to assess the test-retest reliability and Pearson’s coefficient of correlation was calculated. The students were made to fill the questionnaire with the same set of questions twice within a gap of 30 minutes on the same day to avoid dilution of results of the pilot test. The mean and standard deviation of the scores from both the times was correlated to measure the consistency of test.
The scores obtained in feedback questionnaire were analysed to calculate Cronbach’s alpha. The mean score of the pilot test was 1.27±0.83 for the first time and 1.22±0.81 for the second time. Pearson’s coefficient of correlation in this study was 0.976 (p<0.001) which showed good reliability. Cronbach’s alpha value for the questionnaire was 0.79.
Statistical Analysis
The data was analysed using Statistical Package for Social Sciences (SPSS) version 21.0. The perception of students towards OSPE and CPE was compared using Student’s t-test and p-value was calculated.
Results
In the cognitive domain, 93 (76.23%) students asserted that it helped in better learning and reasoning. It was observed that 106 (86.89%) students were in favour that the syllabus was well covered. Majority {112 (91.82%)} of the students were affirmative that the questions were quite clear [Table/Fig-1].
Perception of students in cognitive domain (N=122).
| Parameters | Yes | Can’t say | No |
|---|
| Does it help in better learning and reasoning? | 93 (76.23%) | 21 (17.21%) | 8 (6.56%) |
| Was the syllabus well covered? | 106 (86.89%) | 11 (9.02%) | 5 (4.09%) |
| Were the questions clear? | 112 (91.82%) | 5 (4.09%) | 5 (4.09%) |
In psychomotor domain, it was observed that 76 (62.30%) students responded in favour of its usefulness in skill development. When asked about its ability to test a wide range of skills, 59 (48.36%) students were affirmative about it; 58 (47.54%) students agreed that it was helpful in learning the clinical application of the subject [Table/Fig-2].
Perception of students in psychomotor domain (N=122).
| Parameters | Yes | Can’t say | No |
|---|
| Was it useful in skill development? | 76 (62.30%) | 31 (25.41%) | 15 (12.29%) |
| Does it test wide range of skills? | 59 (48.36%) | 28 (22.95%) | 35 (28.69%) |
| Was it helpful in learning clinical application of the subject? | 58 (47.54%) | 24 (19.67%) | 40 (32.79%) |
In affective domain, 95 (77.87%) students said that it was not stressful. On asking if it helps in building confidence in the subject, 82 (67.21%) students were affirmative about it; 88 (72.13%) students opined that OSPE is not physically taxing [Table/Fig-3].
Perception of students in affective domain (N=122).
| Parameters | Yes | Can’t say | No |
|---|
| Was it stressful? | 15 (12.30%) | 12 (9.83%) | 95 (77.87%) |
| Does it help in building confidence in the subject? | 82 (67.21%) | 21 (17.21%) | 19 (15.58%) |
| Is it physically taxing? | 9 (7.38%) | 25 (20.49%) | 88 (72.13%) |
Various questions were asked regarding the overall perception of students regarding OSPE as an assessment method. It was observed that 114 (93.44%) students agreed that the instructions given in OSPE were clear and adequate [Table/Fig-4].
Perception of students towards OSPE as an assessment method (N=122).
| Parameters | Yes | Can’t say | No |
|---|
| Were the instructions clear and adequate? | 114 (93.44%) | 8 (6.56%) | 0 (0%) |
| Was sufficient time allotted at each station? | 106 (86.89%) | 6 (4.92%) | 10 (8.19%) |
| Does it eliminate bias? | 78 (63.93%) | 33 (27.05%) | 11 (9.02%) |
| Is it more scoring? | 85 (69.68%) | 25 (20.49%) | 12 (9.83%) |
| Is it a better assessment method? | 77 (63.11%) | 20 (16.40%) | 25 (20.49%) |
The student feedback was obtained using a set of questions and OSPE was compared with CPE by applying Student’s t-test. It was observed that a statistically significant difference (p<0.001, df=6, confidence interval=95%) was present in the perception of students between OSPE and CPE suggesting that OSPE was strongly favoured by the students [Table/Fig-5].
Perception of students towards OSPE and CPE (N=122).
| Parameters | OSPE | CPE | p-value |
|---|
| Which method helps in better learning? | 93 (76.23%) | 29 (23.77%) | <0.001 |
| Which method is more time consuming? | 73 (59.84%) | 49 (40.16%) |
| Which method covers wide range of topics? | 106 (86.89%) | 16 (13.11%) |
| Which method helps in skill development? | 76 (62.3%) | 46 (37.7%) |
| Which method is not stressful? | 104 (85.25%) | 18 (14.75%) |
| Which method eliminates bias? | 78 (63.93%) | 44 (36.07%) |
| Which method will you prefer for examinations in future? | 93 (76.23%) | 29 (23.77%) |
Student’s t-test was used
Discussion
The OSPE has been considered as the preferred mode of practical examination in para clinical subjects like pharmacology [9]. However, it has been observed that change in the methods of assessment fails to keep pace with the changes in curriculum [11]. As a result, there are just a handful of medical colleges which employ this pattern of assessment [10]. Introduction and evaluation of OSPE as an assessment tool for medical students in pharmacology in the study institute was commenced as a milestone towards refinement in the traditional methods of practical examination.
This study aimed to address the unmet needs of CPE in terms of application of theoretical knowledge to clinical practice. OSPE is a type of assessment method which is believed to fulfil the deficiencies seen in CPE [12].
Since, OSPE was introduced to second year medical students for the first time, this study was planned to assess their perception and obtain their valuable feedback so that necessary modifications in OSPE can be made before it is included in formative and summative assessment. Feedback is the most important tool for modifications and improvement in medical education [13]. With the recent changes in undergraduate curriculum by Medical Council of India towards incorporating CBME, OSPE as a type of assessment method becomes the need of the hour [14]. OSPE offers several advantages over CPE as it helps to circumvent examiners bias, brings objectivity in examination, and facilitates the student to express his/her knowledge [15].
In the present study, majority of students agreed that OSPE helps in better learning and reasoning (76.23%). Most of the students were in favour that OSPE included a wide range of topics, syllabus was well covered and the questions asked were quite clear. These findings are consistent with a similar study in which 96% of students agreed that OSPE has significantly contributed in improvement of their knowledge and learning [16]. Another study found that 72% of students believed that OSPE helped to achieve the learning objectives in a uniform manner [17].
Most of the students were in favour that OSPE helps in learning a wide range of practical and communication skills and clinical application of the subject. These results were consistent with another study conducted earlier where 100% of students agreed that OSPE helped in learning practical skills [18]. In another similar study, majority of students believed that OSPE helped them learn practical skills [19]. In contrast to this study, where practical and communication skills were assessed using various injection techniques on mannikins, previous studies suggested the need for incorporation of mannikins in OSPE [4,18].
Students usually suffer from stress and anxiety in the examination and studies conducted earlier have regarded OSPE to be quite stressful [20,21]. However, in the present study, majority of students agreed that OSPE was neither stressful nor physically taxing and it helped in building confidence in the subject. This could be due to the fact that every student faced similar questions thus eliminating bias and there was minimum interaction with teachers in OSPE. These results are in tandem with a similar study where most of the students disagreed that OSPE is stressful and appeared to be more comfortable with this pattern of examination [22].
A strongly positive feedback was obtained when students were asked about using OSPE as an assessment method in the future. Total 86.89% of students were satisfied with the time allocated to each station. The results are coherent with studies conducted earlier [17,18,23]. Vast majority of students opined that OSPE is a more scoring and better method of assessment than the traditional method as observed in previous studies [11,13,17].
Comparison of students’ perception between the two methods of assessment suggested that majority of the students preferred OSPE over CPE for future examination. Another similar study conducted earlier demonstrated that 93.16% of students opined that OSPE is better than traditional examination [18].
Limitation(s)
The present study primarily focuses on the perception of medical students towards OSPE and does not include teachers’ perception which is also equally important.
Conclusion(s)
OSPE is an effective and skill enhancing method of assessment for medical students in pharmacology. It ensures better student’s performance, removes bias, eliminates stress, and redundancy in the examination. However, it requires extensive planning and carries logistics issues.
Student’s t-test was used