Level of Cognitive Domains and Weightage of Various Topics in the Undergraduate Summative Examination Question Paper of Community Medicine: A Cross-sectional Study

1. Associate Professor, Department of Community Medicine, Pramukh Swami Medical College and Sri Krishna Hospital, Bhaikaka University, Karamsad, Anand, Gujarat, India. 2. Assistant Professor, Department of Community Medicine, Manipal TATA Medical College, Jamshedpur, Manipal Academy of Higher Education, Manipal, India. 3. Associate Professor, Department of Forensic Medicine, Pramukh Swami Medical College and Sri Krishna Hospital, Bhaikaka University, Karamsad, Anand, Gujarat, India.

Abstract

Introduction: Assessments direct students towards learning. There are three types of learning: cognitive, affective, and psychomotor. While all three are interlinked, the cognitive domain, which covers knowledge and intellectuality, is foundational and can be assessed through theory examinations. Theory exams are the best way to test the cognitive domain of a learner, while practical exams can assess the psychomotor and affective domains. Since the question paper is the most important tool in theory exams, it is essential that the question paper covers relevant topics and tests all levels of cognitive learning, which encompass knowledge, comprehension, application, analysis, synthesis, and evaluation.

Aim: To assess the level of cognitive domains assessed and the weightage allocated to various topics in the undergraduate summative examination question paper of community medicine.

Materials and Methods: A cross-sectional study was conducted at Department of Community Medicine, Pramukh Swami Medical College and Sri Krishna Hospital, Bhaikaka University, Karamsad, Anand, Gujarat, India in which the five-year question papers (2016-2020) from five Universities were analysed to assess the level of cognitive domains and the weightage assigned to various topics. Each question was categorised as recall, comprehension, or application based on the cognitive domain it assessed. Furthermore, each question was analysed to determine the topic it pertained to, and the marks allocated to that question were assigned to the corresponding topic.

Results: The majority of marks in Universities 1, 2, 3, 4, and 5 were, respectively allocated as follows: 324 (80.62%), 459 (84.66%), 453 (75.5%), 895 (89.5%), and 379 (63.16%) for questions testing the recall ability of the learners. For questions assessing comprehension ability, the marks allotted were 20 (4.87%), 0, 81 (13.5%), 65 (6.5%), and 51 (8.15%). The fewest marks were assigned to questions testing application, synthesis, or evaluation abilities. Epidemiology had the highest weightage in all five Universities, with marks of 73 (18.25%), 75 (13.89%), 93 (15.5%), 141 (14.1%), and 83 (13.83%).

Conclusion: The cognitive domains assessed in the subject across all five Universities were unsatisfactory. The weightage of marks allotted to topics varied, highlighting the need to develop a question paper plan that facilitates a systematic distribution based on different levels of cognitive domains and topics.

Blueprinting, Community medicine, Epidemiology, Theory examination

Theory examinations are an important and major part of summative assessment. It is well established that a good system of evaluation has a profound effect on learning (1),(2). The three domains of learning are cognitive, affective, and psychomotor. Cognitive pertains to knowledge, affective pertains to attitude, values, and beliefs, and psychomotor pertains to skills (3). Questions that assess knowledge on a particular topic through recall or test the understanding of the learner by asking them to paraphrase, contrast, differentiate, or apply that knowledge and understanding in a specific situation, or formulate a plan in a certain scenario, or ask critical points on a topic, test the cognitive domain (3). Any question where the learner is asked to demonstrate something with physical and intellectual skills will test the psychomotor domain, and questions where the learner is required to advocate, discuss, or negotiate with individuals will test the affective domain (3).

Bloom’s Taxonomy describes lower to higher levels of cognitive learning in six levels. The first level is Knowledge, which encompasses the ability to recall bits of information such as classifications, names, definitions, etc. The next level is comprehension, which involves understanding a particular concept and being able to describe it in one’s own words. Higher in the order is application, which combines knowledge attainment and understanding to implement it in a new and practical situation. Following that are analysis, synthesis, and evaluation. Analysis involves critical thinking on a topic, while synthesis and evaluation entail creating something new in a specific situation and providing feedback, respectively (4),(5).

Theory examinations assess the cognitive domain, with the question paper being the most common tool (6),(7),(8),(9). Therefore, it is essential to have a question paper with set standards that assesses each level of Bloom’s Taxonomy (8) and covers all the topics taught. With the change in the Bachelor in Medicine, Bachelor in Surgery (MBBS) curriculum in India, the Competency-based Medical Education (CBME) approach has been implemented since the academic year 2019-2020 (10),(11). However, any change will be futile if the assessment methods are not modified accordingly.

The evaluation of the quality of question papers is crucial. Evaluations of question papers have been conducted for various subjects such as Forensics, Pharmacology, Microbiology, and Biochemistry (6),(7),(8),(9),(12). While an evaluation of the question paper for community medicine was conducted for one University by Nagaraj K et al., it is necessary to evaluate question papers from multiple Universities to draw meaningful inferences (8). Therefore, taking this into consideration, the present study was planned to evaluate the existing theory question papers of community medicine from various Indian Medical Colleges, aiming to assess the level of cognitive domains in the questions asked and the weightage assigned to different topics within the subject.

Material and Methods

A cross-sectional study was conducted at Department of Community Medicine, Pramukh Swami Medical College and Sri Krishna Hospital, Bhaikaka University, Karamsad, Anand, Gujarat, India, from July to September 2023. A retrospective analysis of five-year question papers (2016-2020) of community medicine from five different Universities in India was performed. The identities of these Universities have been kept anonymous throughout the study. Since the data collection involved using question papers already available in public domain databases, a waiver of consent was obtained for data collection from the Institutional Ethics Committee (IEC/BU/2023/Ex.35/196/2023). The evaluation of this subject in summative exams is conducted in two papers, paper one and paper two. Hence, the question papers of paper one and paper two for each academic year were used in the present study.

Inclusion and Exclusion criteria: To maintain uniformity, Universities for which both paper 1 and paper 2 were available for all five years were included in the study. Universities for which either paper 1 or paper 2 was missing between 2016-2020 were excluded.

Study Procedure

A total of 50 question papers (ten from each university) were evaluated. Each question was categorised as a long question, short question, very short question, or Multiple choice Question (MCQ). Additionally, each question was assessed and categorised based on the level of learning it assessed for the learners. Questions that required learners to recall information and write were classified under the recall category, questions that assessed understanding and required learners to differentiate, explain, or summarise were classified under comprehension, and questions that required learners to apply knowledge in hypothetical situations were classified under application. The marks allotted to each question in each category were summed up (6),(7),(8),(9),(12). Only three domains were considered in the present study, as no higher-level cognitive domain questions were asked in the summative assessment.

Each question was also analysed to determine the topic it covered. The curriculum laid down by the National Medical Commission (NMC) provided a list of 20 topics for the study. Each question was assigned to the corresponding topic, and the marks allotted to that question were noted for the topic. The marks allotted to each topic in a specific question paper were summed up and recorded for all 20 topics. This categorisation was performed independently by two assessors (PT and AP), and conclusions were reached based on their agreement.

Statistical Analysis

All observations were collected in MS excel, and appropriate tables and graphs were used to present the data.

Results

Out of the 830 questions analysed in the 50 question papers, the total number of questions in each question paper was as follows: 19 in University 1, 11 in University 2 (excluding 6 marks for 12 MCQs), 13 in University 3, 22 in University 4, and 18 in University 5.

As depicted in (Table/Fig 1), it was found that the majority of the marks in almost all the Universities were allotted to short answer question types. In University 1, long answer questions were given a weightage of 50%. It was also found that only one University allocated 10% of the marks to MCQs.

As depicted in (Table/Fig 2), it was found that in all the Universities, the majority of the questions were in the recall domain. This ranged from 379 (63.16%) in University five to 895 (89.5%) in University four. Application-based questions ranged from as low as 40 (4%) in University 4 to a maximum of 170 (28.3%) in University 5. Comprehensive questions were almost negligible, ranging from 0 (0%) in University two to 81 (13.5%) in University 3.

(Table/Fig 3) illustrates the marks weightage assigned to each topic in all five Universities. It is evident that the distribution of marks was uneven across the question papers of all the Universities. The topic of epidemiology consistently received the highest marks weightage in all Universities.

Furthermore, it can be observed that there were no questions from topics like essential medicine in University 1, 2, 3, and 5. Similarly, University 2 and 5 did not include any questions from the topic of recent advances.

Discussion

One of the most crucial aspects of medical education is assessment. Assessment is defined as “any formal or informal action to gather information about the student’s performance and competence.” Validity, reliability, acceptability, and consequences of assessment are the four most important characteristics of an effective assessment procedure (13),(14).

The present study revealed that the majority of marks in almost all Universities were allocated to short answer and very short answer question types. Only one University allocated 10% of marks to MCQs. Since there was only one study (8) found on community medicine question papers, comparisons were made with studies conducted on question papers of other subjects such as Forensic Medicine, Microbiology, and Pharmacology.

Similar findings were observed by Dayanidhi VK et al., in the question paper of forensic medicine, where nearly 50% of the marks in all Universities were allotted to short essay or short answer question types, followed by long answer questions and very short answer questions (12). In the study by Mehta S et al., on microbiology question papers, 74% of the questions were short notes with four marks each, 21% were short answer type questions with two marks each, and 5% were long answer questions with 10 marks each (8). In the study by Nagaraj K et al., each paper of community medicine was worth 100 marks, with 50% of the marks allotted to short notes with five marks each, 20% to long essay questions of 10 marks, and 30% to very short answer questions of three marks each (8).

In the present study, it was found that a majority of the questions in all the Universities ranged from 60% to 89% in the recall domain. Application-based questions accounted for as low as 4% and up to a maximum of 28%. Comprehensive questions were almost negligible, ranging from 4.8% to 13.5% only. University 2 did not have any comprehension-based questions in either of the papers during the study period. Similar findings were also noticed in a study conducted by Nagaraj K et al., on community medicine question papers, where 57.6% of the questions were recall-based, 33.1% were comprehension-based, 9% were application-based, and 0.3% was analysis-based (8). In a study conducted by Dayanidhi VK et al., on forensic medicine question papers, the majority of the questions were allotted to the recall domain (80%), followed by comprehension (20-70%), and negligible or zero marks were allotted to application-based questions (12). Similarly, a study by Khuteta NK and Saurabh MK on pharmacology question papers found that 92.08% of the questions were recall-based, and the rest were reasoning-type questions (7). The results of the above studies showed that the question papers measured students’ knowledge and cognitive abilities. Restricting the questions to only the recall domain has an impact on the quality of the test. The medical profession mainly relies on prompt thinking and knowledge application, which varies from patient to patient. Thus, higher-order cognitive abilities must be tested to ensure the validity of the assessment.

The results showed that the distribution of marks for each topic varied across all the university’s question papers. The majority of the marks were allotted to topics such as epidemiology, communicable diseases, and reproductive and child health. However, other important topics like geriatrics health, occupational health, nutrition, and environment were given the least importance. Similar findings were seen in a study conducted by Nagaraj K et al., in which community medicine theory question papers from 2008 to 2016 were analysed (8). It was found that a few chapters like epidemiology, environment and health, nutrition, communicable diseases, and reproductive and child health were given more emphasis. The marks allotted to communicable diseases ranged from 40 to 3 marks in a 100 marks question paper. Less marks allotted to a topic give the impression of it being less important, even if it is a public health problem.

A blueprint should be developed to ensure that every facet and domain of the curriculum is assessed within the allotted time. Using a blueprint for setting questions in question papers will enable appropriate distribution of weightage to topics, as mentioned in references (15),(16). A systematic approach for framing question papers needs to be implemented across Universities. Since the curriculum for MBBS has been standardised, there is also a need for standardisation of the question paper pattern. The practical implications of a topic should be given emphasis when setting a question paper. The present study needs to be conducted on a larger scale for a better understanding and improvement of the curriculum.

Limitation(s)

The limitation of the present study was that authors could not include other Universities due to limited accessibility and feasibility.

Conclusion

In the present study, the majority of questions were based on recall. Therefore, it is necessary to include comprehension-based and application-based questions. Topics with practical implications, such as mental health, geriatrics, health education, disaster management, and occupational hazards, were neglected. The present study included question papers from before the adoption of the new curriculum. Therefore, further comparative studies can be conducted to observe any changes in the pattern of question papers after the curriculum change for all subjects.

DOI and Others

DOI: 10.7860/JCDR/2024/68509.18986

Date of Submission: Nov 06, 2023
Date of Peer Review: Nov 30, 2023
Date of Acceptance: Dec 22, 2023
Date of Publishing: Feb 01, 2024

AUTHOR DECLARATION:
• Financial or Other Competing Interests: None
• Was Ethics Committee Approval obtained for this study? Yes
• Was informed consent obtained from the subjects involved in the study? No
• For any images presented appropriate consent has been obtained from the subjects. Yes

PLAGIARISM CHECKING METHODS:
• Plagiarism X-checker: Nov 15, 2023
• Manual Googling: Dec 02, 2023
• iThenticate Software: Dec 20, 2023 (3%)

ETYMOLOGY: Author Origin

EMENDATIONS: 7