Currently, there are about 256 postgraduate courses offered in field of health professions by various colleges in India [1]. The curriculum for these courses are well defined with respect to their respective subject matter but training and assessment of soft skills i.e., interpersonal skill, communication skill and professionalism are seldom part of this defined curriculum. These skills are as important as clinical skills and knowledge to produce a good medical professional, therefore evaluation of these skills is essential [2].
Three-sixty degree feedback, also known as multisource assessment, is a process where an individual (recipient) is rated on their performance by multiple people who are closely associated with him and know something about his work – these persons can be peers, seniors, teachers, nurses, undergraduate students and patients/ patient’s relatives etc. This feedback method is mentioned as the first method for evaluating resident as per the toolbox assessment methods published by the American Board of Medical Specialities and Accreditation Council for Graduate Medical Education [3].
360 degree evaluation is used mostly for formative but can be used for summative assessment as well. In the former situation, evaluators after assessment can provide comments and suggestions for better performance. Accreditation Council for Graduate Medical Education in 1999 had identified six core competencies which include: medical knowledge, patient care, professionalism, practice- based learning and improvement, systems-based practice and interpersonal and communication skills. Three sixty degree evaluations can potentially assess all the six core competencies but may be especially useful in assessing interpersonal skills, communications skills, and professionalism [4]. These skills are important for physicians as these skills help them to gain and maintain the trust of patients and also work as a team with other healthcare professionals [3]. It is important to teach and assess these skills in medical education which has become an area of increasing interest among medical educators [3,5,6].
Performance management by balancing strengths and weakness of any healthcare provider is essential for maximizing the quality of care given to the patient. In 360 degree evaluation, postgraduate students are evaluated by different people from their working spheres [1]. This approach is better than single evaluator assessment as it will enhance their soft skills and thereby improve their overall performance if done diligently.
The aim of the study was to find feasibility of evaluation of resident’s professionalism, communication and interpersonal skills at a rural tertiary care teaching hospital in Western India through 360 degree evaluation method.
Materials and Methods
This descriptive observational study was conducted after due ethical approval at a tertiary care hospital affiliated with a medical college in the Western province of India to evaluate the postgraduate students’ personal attributes by 360 degree in November 2014. This was a pilot study implemented in four departments to start with. All residents of four departments viz., ‘Ophthalmology’ (Ophthal); ‘Otorhinolaryngology’ (ENT); ‘Dermatology, Venereology and Leprosy’ (Skin & VD); and ‘Tuberculosis and Chest Diseases’ (TBCD) were enrolled in the study after orienting them to the study objectives.
Each resident doctor was evaluated by two attending physicians, two nursing staff, two peers, two patient’s families and two undergraduate students for their attributes of professionalism, communication skill and interpersonal skill through a questionnaire prepared by the professors of the hospital as a part of student’s log book. All these assessors were selected randomly from the residents’ sphere of working. One male and one female relative of the patient, able to understand the study and voluntarily consenting to respond for each resident were enrolled. After proper orientation in their language of preference (Gujarati or English), the assessors were given separate evaluation forms for each resident with details of the attributes (professionalism, communication and interpersonal skills) and Likert scale rating for their response. Forms were collected on the same day or next day. Identity of the assessors and their responses were kept confidential and specifically was not disclosed to the residents at any given time.
A five point Likert-scale was used to rate the resident’s skill (interpersonal skill, communication skills, and professionalism) with ‘5’ meaning ‘excellent’; ‘4’ meaning ‘very good’; ‘3’ meaning ‘average’; 2 meaning fair; and ‘1’ implying ‘poor’.
Parameters to assess these skills (interpersonal skill and communication skills, and professionalism) were consensually developed by medical educationists of our institution based on their experience and wisdom. These included 10 parameters namely:
Humanism (respect, empathy and compassion) (is polite and considerate, treats the patient with respect, has respect for others, is helpful to the patient, listens attentively to patients and shows concern and compassion, is sensitive to emotional, social and cultural aspects of patient);
Maintains punctuality (in all duties, assignment, both departmental and institutional);
Discipline (available on duty, properly dressed, maintains code of conduct, follows standard protocols);
Commitment and sincerity (takes initiative, responds promptly to calls, hardworking, takes responsibility for the work and completes on time, efficiently and has a keen desire to learn, is accountable);
Honesty (admits mistakes, reflects on them, no manipulation of information and open to suggestions);
Loyalty to institution (optimum use of resources, has a sense of ownership);
Interpersonal skills (works in harmony with the peers, nurses, students and teachers, gives respect);
Has the ability to work effectively in teams;
Shows leadership skills with good decision making in challenging situations;
Communication skills (both verbal and non-verbal, communicates on time, clearly and effectively);
Ten response sheets for 26 residents, making a total of 260 were tabulated further to deduce the observations in a comprehensive manner.
Results
There were a total of 26 residents evaluated from the aforesaid four department’s viz., Ophthal, ENT, Skin & VD and TBCD. Skin & VD and Ophthal departments had five residents each while ENT and TBCD had eight each.
There was a visible trend of increase in mean score across years of residency noted in general i.e., the overall average scores of third year resident was better than first and second year residents [Table/Fig-1].
Distribution of scores of residents by department and years of residency.
| Years of residency | ENT n/mean | Ophthalmology n/mean | Skin n/mean | TB Chest n/mean | Total n/mean |
|---|
| First year | 3/3.58 | 2/3.80 | 2/4.11 | 3/3.99 | 10/3.85 |
| Second year | 3/3.82 | 2/4.11 | 2/4.10 | 3/4.42 | 10/4.00 |
| Third year | 2/4.19 | 1/4.18 | 1/4.42 | 2/4.01 | 6/4.17 |
| Total | 8/3.82 | 5/4.00 | 5/4.17 | 8/4.01 | 26/3.98 |
n= number of residents in each year. mean= average of mean scores in all traits of all residents in each year by all raters.
Upon classifying the parameters of questionnaires and rearranging score values for all resident doctors, the observations were deduced as per [Table/Fig-2]. This project being a pilot project to help the institution for continuous self improvement in long run in training the postgraduate students, the students were not only compared to the general bench mark of average score of ‘3’, but also were assessed on their performance in comparison to their own average score to find their weaker areas and give timely feedback for improvement (‘B’ part of the [Table/Fig-2]). It was helpful in checking if the resident doctor is falling below his own average in any trait.
Distribution of number of residents according to score cut off(s).
| Raters→Traits↓ | Patients | Nurses | Undergraduate students | Peers | Consultants |
|---|
| A | B | A | B | A | B | A | B | A | B |
| Humanism | 2 | 12 | 0 | 11 | 1 | 18 | 3 | 21 | 3 | 22 |
| Maintains punctuality | 1 | 11 | 0 | 16 | 1 | 17 | 5 | 23 | 5 | 26 |
| Discipline | 0 | 14 | 0 | 17 | 1 | 17 | 5 | 21 | 4 | 26 |
| Commitment and sincerity | 0 | 6 | 1 | 20 | 0 | 15 | 4 | 19 | 7 | 26 |
| Honesty | 0 | 9 | 3 | 21 | 3 | 17 | 5 | 20 | 3 | 24 |
| Loyalty to institution | 2 | 6 | 1 | 16 | 0 | 20 | 5 | 20 | 6 | 26 |
| Interpersonal skills | 1 | 11 | 0 | 17 | 2 | 17 | 4 | 16 | 5 | 26 |
| Team work | 2 | 7 | 0 | 17 | 0 | 16 | 3 | 17 | 8 | 26 |
| Leadership skills | 1 | 8 | 0 | 13 | 2 | 21 | 6 | 20 | 8 | 26 |
| Communication skill | 1 | 7 | 0 | 17 | 4 | 19 | 2 | 20 | 4 | 25 |
A: Number of residents who were given a score less than ‘3’ (as per Likert scale) by a rater. B: Number of residents whose score value was below their own average score per trait by different raters.
Out of 26 resident doctors rated by patients, none of them scored below ‘3’ with reference to discipline, commitment and sincerity and honesty. However, one or two residents were found below score ‘3’ with reference to remaining traits. In contrast to rating by patients, nursing staff and undergraduate students, number of residents scoring below ‘3’ was higher in consultant and peer category across different traits. Therefore, larger deviations were noted in the consultants and peers groups as compared to the other groups [Table/Fig-2].
However, there was overall good agreement in scores of various assessors. One point difference in the Likert scale was compared between different raters for their agreement. There was highest agreement noted i.e., 88.4% between ratings done by nursing staff and patients for the residents. As for example, if consultants were giving 77% points for all qualities to residents; in comparison the nursing staff, undergraduate students and patients were giving them more points respectively [Table/Fig-3].
Percentage of observation within unit difference i.e., one point difference on the Likert scale.
| Peer | Nursing staff | Undergraduate students | Patients |
|---|
| Consultants | 77.0 | 77.3 | 84.6 | 74 |
| Peer | | 87.3 | 81.5 | 72.4 |
| Nursing staff | | | 85.0 | 88.4 |
| Undergraduate students | | | | 81.6 |
The average scores were found to be maximum for the quality of punctuality in all duties assigned to a resident across all raters (mean of all raters: 4.05). Also, the highest ratings were done by the nursing staff and patient relatives among all the raters implying good impression and very good behaviour of the residents with nursing staff and patient relatives [Table/Fig-4].
Mean (SD) scores by various assessors for different traits.
| Trait | Peer | Nursing staff | Clinician | Patient relative | Undergraduate student |
|---|
| Humanism | 3.83 (0.54) | 4.24 (0.67) | 3.76 (0.45) | 4.13 (0.59) | 3.87 (0.56) |
| Maintains Punctuality | 3.87 (0.74) | 4.26 (0.42) | 3.69 (0.59) | 4.28 (0.52) | 4.17 (0.52) |
| Discipline | 3.89 (0.71) | 4.19 (0.59) | 3.72 (0.58) | 4.20 (0.59) | 4.15 (0.57) |
| Commitment and Sincerity | 3.80 (0.87) | 3.98 (0.55) | 3.48 (0.61) | 4.24 (0.51) | 4.13 (0.58) |
| Honesty | 3.76 (0.86) | 4.02 (0.63) | 3.52 (0.56) | 4.17 (0.53) | 3.87 (0.67) |
| Loyalty to institution | 3.81 (0.74) | 4.09 (0.54) | 3.46 (0.59) | 4.30 (0.62) | 3.96 (0.62) |
| Interpersonal skills | 4.07 (0.83) | 4.28 (0.58) | 3.61 (0.66) | 4.15 (0.65) | 4.07 (0.65) |
| Team work | 3.80 (0.70) | 4.06 (0.54) | 3.48 (0.69) | 4.24 (0.58) | 4.04 (0.44) |
| Leadership skills | 3.72(0.76) | 4.04(0.57) | 3.35(0.68) | 4.24(0.49) | 3.85(0.62) |
| Communication skill | 3.85(0.65) | 4.19(0.50) | 3.46(0.72) | 4.33(0.57) | 4.00(0.65) |
Discussion
The concept of 360 degree evaluation and feedback is quite well established and utilized in management related fields [7]. The concept is however gaining momentum slowly but steadily in medical training.
From the results of the current pilot study, based on small sample of residents, it is learnt that this feedback method is effective in shaping the residents’ subtle attributes on attitude and communication skills that are very important for making of a compassionate professional. Medical council of India has introduced Attitude and Communication (AT-COM) module recently for the undergraduate teaching but nothing has so far been recommended for postgraduate teaching [8]. Thus, 360 degree evaluation can be considered as a tool in producing an ideal postgraduate student that the institution strives to produce. Based on the positive and very encouraging findings from this study, this method of evaluation and feedback has now been implemented in all the departments across the institution for the holistic development of the resident doctors during their training with us.
From the current pilot study in our set-up, we not only found the students’ average score in various traits, we also found the traits of students that needed improvement when compared to their own average score. This was helpful by giving customized feedback to the students for their personal improvement (B part of [Table/Fig-2]).
A recent study by Hemalata R and Shakuntla BS concludes the assessment done by the nurses and patient’s relatives was inferior to peer and self evaluation [9]. Such findings were also consistent with earlier studies done by Chandler N et al., and Brinkman WB et al., [2,9]. To our surprise, ratings done by patients, nursing staff and undergraduate students was good; in fact better than rating by peers and consultants. This although is a positive finding for the institute, as patients seem to be satisfied with the health care professionals here, still, there is a need to explore further for reasons for lesser score by others. The probable explanation for this difference could be the regular sensitization of faculty and resident doctors for professionalism, communication skills, interpersonal skills etc., in this institution possibly resulting in higher expectations by the peers and consultants. Also, strong association between patients and colleague rating was noted in a study done by Lelliott P et al., [10].
Majority of the postgraduate institutions admit 50% of students on ‘All India’ Quota hence any postgraduate institution can have resident doctors from any other region/state. Thus, a resident doctor is all of a sudden exposed to a different society with linguistic and social diversity, mostly without any sensitization of local culture of that community to which he is expected to cater health services. In such a situation, the feedback offered by the 360 degree evaluation might prove to be vital for the progress of resident as this will give a holistic view of his behaviour from various people he associates to.
The apex statutory authority- Medical Council of India, in its official publication on “Vision Document 2015” has incorporated certain traits i.e., professionalism, communication skill and interpersonal skill etc., in addition to skill and competence of core areas [10]. However, no study has shown how to incorporate these skills in the medical curriculum so far. We understand that the perception of resident doctors by five different assessors is varied and hence assessors other than consultants have significant role in the evaluation of the resident doctor.
Limitation
Though, the current study has its limitations owing to a small sample size and difficulty in explaining the patient’s relatives importance of this exercise; we understood that the consultants and peers felt need for improvement in the residents’ performance in most of the traits.
Conclusion
Through this small pilot study we are able to demonstrate the need and usefulness of 360 degree evaluation of the residents in the current situation. Making a 360 degree evaluation as a norm in postgraduate teaching can be helpful in improving quality of medical education and health care. This will also help the resident doctors with potential to become stalwarts of healthcare system and get groomed in a complete way as per the need of the society.
n= number of residents in each year. mean= average of mean scores in all traits of all residents in each year by all raters.A: Number of residents who were given a score less than ‘3’ (as per Likert scale) by a rater. B: Number of residents whose score value was below their own average score per trait by different raters.