Improving the Quality of Paediatric Discharge Summaries at the University Teaching Hospital: A Retrospective Chart Review

1. Associate Professor, Department of Paediatrics, Pramukhswami Medical College, Bhaikaka University, Karamsad, Gujarat, India. 2. Professor, Department of Neonatology, Pramukhswami Medical College, Bhaikaka University, Karamsad, Gujarat, India. 3. Professor, Department of Central Research Services, Pramukhswami Medical College, Bhaikaka University, Karamsad, Gujarat, India. 4. Professor, Department of Paediatrics, Pramukhswami Medical College, Bhaikaka University, Karamsad, Gujarat, India. 5. Professor, Department of Neonatology, Pramukhswami Medical College, Bhaikaka University, Karamsad, Gujarat, India.

Abstract

Introduction: The Discharge Summary (DS) is a vital document for ongoing patient care. Poor quality of DS may lead to adverse events, medication errors, readmission, or unnecessary repetition of diagnostic work-ups. DSs provided by academic institutions may not meet expectations and, therefore, need to be evaluated.

Aim: To improve the quality of DSs issued from the paediatric ward of a tertiary-care teaching hospital.

Materials and Methods: The present retrospective chart review was conducted in the Department of Paediatrics, Shree Krishna Hospital (a rural tertiary care teaching hospital), Karamsad, Gujarat, India, from May 2018 to May 2020. Thirty-three summaries were randomly selected from the last quarter (October to December) of each calendar year 2017, 2018, and 2019 (a total of 99 summaries), representing the pre-intervention, post-intervention, and retention phases, respectively. In the absence of a contextual paediatric DS assessment tool, a novel scoring system was developed to evaluate 33 components of the DS under seven broad domains. This scoring system was consensually validated. Residents responsible for preparing the DS were empowered in small groups. Peer audits of the summaries prepared by each resident were conducted on multiple occasions by consultants, followed by feedback highlighting areas for improvement. Review of the summaries by senior residents or consultants were promoted. Discussions on discharge processes were emphasised during ward rounds including important DS documentation points. The percentage mean±Standard Deviation (SD) domain scores and the total final score were compared using Analysis of Variance (ANOVA) with post-hoc comparisons by Scheffe’s test using Statistics and Data (STATA) software version 14.2.

Results: Out of 33 summaries from 2017, one patient file was not traceable at the time of the study, and reviews of 30 and 32 DSs could be reviewed for the years 2018 and 2019, respectively. The scoring tool demonstrated good inter-rater reliability. The mean±SD total % score was 78.50±8.23 in 2017, 85.48±8.91 in 2018, and 88.48±9.05 in 2019. There was a significant improvement (p-value=0.012) from 2017 to 2018. Domains such as history, physical examination, and discharge plan showed significant improvement, while the hospital course domain remained resistant to improvement.

Conclusion: The DS score improved consecutively for both years. The quality of DSs prepared by residents in a teaching institute can be enhanced through education, audit feedback, and changes in the work system. These methods can be incorporated into postgraduate training to improve documentation.

Discharge planning, Documentation, Medical education, Quality improvement

The hospital DS is an important medicolegal document and imperative for the continuity of patient care, particularly for patients with chronic health conditions requiring multidisciplinary consultations (1),(2). The value of a good DS is more important in the Indian context, as the healthcare system is unstructured, and patients can frequently change their family practitioner or apex hospital. A DS can even serve as a patient or caregiver educational tool to enhance their disease understanding, discharge medication, home care, future follow-up and danger signs (2),(3). Inaccuracy and deficiencies in the DS affect the quality of ongoing patient care and increase the risk of adverse events, medication errors or readmission (4),(5). A well-documented DS will avoid repeating diagnostic tests and medications, thus saving constrained resources (6). However, determining what makes a good DS is not easy. The need for inclusion or omission of a particular summary component varies on a case-to-case basis. In an attempt, some healthcare organisations have outlined components of a quality DS (7),(8).

The essential components of a high-quality DS include admission diagnosis, pertinent physical examination findings, laboratory results, procedures and complications in the hospital, discharge diagnosis, discharge medications, active medical problems at discharge and follow-up (2),(9). Evidence suggests that discharge summaries do not meet expectations. There are considerable deficiencies reported in the completeness and efficacy of DSs in two tertiary teaching hospitals. DSs were grossly inadequate at documenting tests with pending results and appropriate follow-up providers (10),(11). Deficits in communication and information transfer are common at hospital discharge and may adversely affect patient care (12). The quality of the DS depends significantly on the person who prepares it, their clinical acumen, understanding of the disease process, and documentation skills. At educational institutes, summaries are prepared by junior residents and are not necessarily reviewed by the consultant. The summaries prepared by junior residents are likely to have more errors (13).

Quality was improved for the summaries prepared by the interns who received instruction on DS skills (14). Audit and feedback sessions were also associated with better outcomes (15). On the other hand, there is no formal education in the medical curriculum in this regard. Evidence supports that appropriate training can improve the quality of the DS. A brief, low-intensity educational intervention involving the first-year paediatric residents improved the quality of discharge communication and can be incorporated into the residency training (16). It was also observed that the quality of the DS can be improved by implementing a structured programme to teach chart documentation skills, ongoing improvement was evident in 1st and 2nd years of the program suggesting that continuing instruction in those skills could be beneficial (4). It was possible to instill soft skills like communication, documentation, etc., through an innovative curriculum (3).

Deficiencies were observed at the present study site in discharge documentation when the patients returned to the Outpatient Department (OPD) follow-up. When the OPD team was different from the treating team during the hospital stay, it was difficult for the OPD team to understand the exact sequence of events that patients had experienced based on the discharge documentation. A similar study to improve the quality of DS was previously conducted successfully in the Department of Internal Medicine at the study site (3). The study used an assessment tool developed by Talwalker JS et al., with minor modifications (4). However, the tool was not specific to paediatrics, and it did not provide the option of “Not Applicable (NA),” which affected the accuracy of scoring.

Therefore, a longitudinal mixed educational, audit-feedback, and improved working pattern interventions were implemented to enhance the quality of discharge summaries from the Paediatric Ward of the Institute. Additionally, a comprehensive assessment tool covering all important aspects of DS was developed, considering the lack of a relevant tool for paediatrics.

Material and Methods

The present retrospective chart review was conducted in the Department of Paediatrics, Shree Krishna Hospital (a rural tertiary care teaching hospital), Karamsad, Gujarat, India, from May 2018 to May 2020. The study was approved by the Institutional Ethics Committee (IEC/HMPCMCE/103/Faculty/14), and a waiver of informed consent was obtained due to the nature of the study.

Inclusion criteria: The DS of patients admitted for >48 hours and discharged from the Paediatric Ward were included in the study.

Exclusion criteria: Patients getting discharged from the Paediatric Intensive Care Unit (PICU), transferred, or taking Discharge Against Medical Advice (DAMA) were excluded from the study, to maintain uniformity.

Sample size calculation: The total percentage score was used as the basis for sample size calculations. A moderate effect size of 0.7 was considered significant. With this effect size, a sample size of 32 was required to achieve 80% power, allowing for a 5% Type-I (alpha) error. A random sample of 33 summaries from each year (2017, 2018, and 2019) meeting the mentioned criteria was selected.

Study Procedure

The ward operations were divided into three units, each with two or three consultants and two or three residents {one first-year resident (R1) and one or two second or third-year residents (R2 or R3)}. Eighteen residents from the department rotated in two-month postings among the wards, neonatal department, and PICU.

Consultants remained in the designated unit. A computerised DS template had been used in the ward long before this study began [Annexure-1]. The patient’s details were entered into a Microsoft Word copy of the template, relevant changes were made, and the summary was printed in two copies. One copy was issued to the patient at the time of discharge, and another was kept with the Medical Records Department (MRD) file. This DS was not reviewed by a consultant every time. All the unit consultants had teamed up to identify gaps and agreed upon improvement strategies.

Discharge Summary (DS) assessment tool: The authors devised a novel scoring tool for the quality assessment of a paediatric DS provided in the ward [Annexure-2]. The paediatric patient profile is quite diverse in many aspects. The communication value of each summary element varies widely depending on the patient and his diagnosis. For example, describing perinatal history or developmental history in 17 year old admitted for viral fever has limited value. whereas these components are crucial for a six-month-old admitted for convulsion and developmental delay. Considering these factors and guided by the previous literature, the authors decided on the components and their weightage for the scoring tool [2,4,7-9,17-21]. A total of 33 components were judged under seven broad headings: basic information, history, physical examination, courses during hospitalisation, investigational data, diagnosis and discharge plan. Most of the components were scored 0, 0.5, or 1 out of 1, but components of importance, like complaints leading to hospitalisation, physical examination at admission and discharge, hospital course, diagnosis, and discharge medication list, were given weightage by scoring them 0, 1, or 2 out of 2. History components like past, perinatal, and developmental history, and detailed anthropometric measurements could be scored out of 1 or 2 depending on their importance on a case-to-case basis, as exemplified above. Some components may not be applicable in a particular case, such as a cross-departmental reference note. If no cross-departmental reference was made for a particular child, that component was nullified by scoring it 0 out of 0. The total score was calculated by simply adding the component scores. However, the total score might be deceptive due to Not Applicable (NA) components. For example, three discharge summaries can have the same total score, let’s say 25, but one may have a score of 25 out of 33, while the other two may have a score of 25 out of 30. For any comparisons, these scores need to be brought to the same platform. This was achieved by simply calculating the percentage score, which accounts for “NA” components. In the above example, the percentage score for the first summary will be 75.76% (25/33×100), whereas the percentage score for the other two summaries will be 83.33% (25/30×100). The percentage score was compared rather than the raw total score. Thus, the overall scoring tool framework was fixed and defined, allowing descriptive freedom for a diverse patient profile. The scoring tool was repeatedly reviewed and re-evaluated, considering each author’s practical experience, before the final version was approved. The scoring rubric or descriptor of components [Annexure-3] was also developed to minimise inter-assessor variation. Pilot testing was conducted to check the consistency in scoring between the assessors for the total score based on ten summary assessments (other than the summaries included in the final assessment).

Interventions

Small group discussion: The unit head led a half-hour small group discussion with the residents, emphasising the importance of a good quality summary and its components, maintaining completeness with brevity, clarity of language, sentence framing, and documentation skills. These sessions were repeated every two months by each unit to ensure that every resident was exposed to these discussions atleast 2-3 times in an academic year.

Team-based audit and feedback: Following the small group discussion, one summary from the previously posted resident team was selected for the audit. The current resident team of the unit reviewed the summary to identify its weaknesses and strengths. The consultant critiqued the summary and provided debriefing with constructive feedback. Each resident team had atleast one such exposure during their bimonthly unit posting.

Improvised working practices: All consultants discussed discharge processes, including relevant points in summary documentation, during ward rounds. Second and third-year residents rechecked all DSs prepared by first-year residents. Consultants reviewed the summaries of patients who had a complicated or lengthy (≥7 days) stay.

All possible efforts were made to adhere to the identified interventions and incorporate these steps into our working habits.

Outcome measurement: The summaries were selected using a random number generator, but the equal representation of each unit was ensured through stratified balanced randomisation. The selected summaries were assessed and scored by one of the three authors. The details were cross-verified from the indoor file or the hospital software system. Each author was given summaries from units other than their own to avoid bias. It was decided by consensus that a DS that scores 80% or more will be termed as “Satisfactory”.

Statistical Analysis

The analysis was performed using STATA software version 14.2. The scoring consistency between the assessors was evaluated using Cronbach’s Alpha with assessors as domains. Descriptive statistics {Mean±SD, frequency (%)} were used to present the quality of DSs at different time points. The mean±SD domain scores and the total final score were compared using ANOVA with post-hoc comparisons by Scheffe’s test, as the summaries involved were different at different time points. A p-value less than 0.05 was considered statistically significant.

Results

The Cronbach’s alpha was good in pilot testing for the total score with assessors as domains (α=0.72). Out of 33 summaries from 2017, one patient file was not traceable at the time of the study. Similarly, reviews of 30 and 32 DSs could be reviewed for the years 2018 and 2019, respectively. Compliance to small group discussion and team-based feedback was more than 80% of all the available occasions after one year in May 2019. (Table/Fig 1) depicts the quality of DSs across the years. The mean±SD percentage score is given for each domain, along with the total percentage score in mean±SD at the end. Each component under all the domains is shown as the number (%) of precise observations. The term “precise” means the total score for the components, i.e., one out of one, or two out of two. (Table/Fig 2) shows the total mean±SD percentage scoring in the box plot diagram across the years.

The mean±SD total percentage score improved significantly (p-value <0.001). However, the post-hoc test showed that the improvement was significant (p-value=0.012) from 2017 to 2018, but the score was similar from 2018 to 2019 (p-value=0.44). Scores for the “basic information” domain were good to start with and remained so, although precision for the address component improved from 2017 to 2018. Scores for “history” improved significantly (p-value <0.001), but again the improvement was mainly from the year 2017 to 2018 (p-value=0.001), with almost no change between 2018 and 2019 (p-value=0.97). Precision was low for components like chief complaint, description of the complaint, negative history, treatment history and past history in 2017. These improved in 2018 and the improvement was sustained in 2019. Even after the interventions, dietary history presented challenges. The “physical examination” domain was found to be better to start with. However, a significant number of imprecise entries in all three components became more precise in 2018 and then further in 2019. Similarly, the “diagnosis” domain was better to start with and remained so. The “course during hospitalisation” appeared to be a difficult aspect with almost no improvement from 2017 to 2018 (p-value=0.97). There was an improvement from 2018 to 2019, but it was not statistically significant (p-value=0.13). The component of course and outcome during hospitalisation mainly contributed to the poor quality of this domain. The “investigational data” improved from 2017 to 2018, and the improvement remained in 2019. The “discharge plan” domain was also better to start with, and there was improvement as well. Notable improvement was evident in the components of education and counselling advice and danger signs explanation (Table/Fig 1),(Table/Fig 3).

The frequency (%) of “satisfactory” summaries improved marginally in 2018 compared to 2017 {15 (46.88%) vs. 20 (66.67%), p-value=0.12), but the improvement was not statistically significant. The frequency (%) of “satisfactory” summaries improved significantly in 2019 compared to 2018 {20 (66.67%) vs. 28 (87.5%), p-value=0.047}.

Discussion

The pre-intervention total mean±SD percentage scoring in 2017 was 78.50±8.23, which was good to start with, making further improvement challenging. Nevertheless, statistically significant improvement was demonstrated in 2018, and further improvement in 2019. When analysing domain performance, “basic information” was excellent to start with and continued to be so. As the template was in use, errors were limited to typo errors, which can further be rectified by semiautomation of such details from hospital electronic records. “History” and “physical examination” domains showed consistent improvement each year. Dietary history is an essential component in paediatrics, and its non improvement was worrisome. On root cause analysis, it was found that the existing template [Annexure-1] had not given separate space for dietary details. This is a limitation of a template system, where the user enters details only under the given headings and does not add necessary information or omit unnecessary information, even if liberty is given to do so. An error in the template gets replicated until corrected. Later on, the template was corrected with the inclusion of some instructions related to discharge advice and homecare in the local language [Annexure-4]. The component of course and outcome during hospitalisation had not shown the expected improvement. As this involves narrative description, some subjective variation was inevitable. The “investigational data” domain had shown due improvement. “Diagnosis” was better to start with, but being the most important DS component, a paucity of further improvement was disheartening. The secondary diagnosis was missed most of the time. The “discharge plan” domain demonstrated improvement. The earlier study had shown inadequacies in documenting tests and follow-up service, but the current study showed improvement in both components for all three years (11). Standardised paediatric DS assessment tools for the Indian context were unavailable when the current study was being planned. Earlier studies on paediatric discharge summaries assessed the presence or absence of pre-specified components before, during, or after the interventions (16),(17).

The authors were more concerned about improving the quality of documentation rather than just the presence or absence of specified components. The computerised DS template was already in place, which could ensure the presence of necessary components. It is challenging to mitigate subjectivity while assessing discharge documents, rather than any documentation for that sake.

The DS assessment tool was developed in the past for the adult population. Reinke CE et al., while comparing electronic vs dictated surgical summaries, scored 13 selected items as 0, 1, 2, 3, or NA based on the absence, inadequacy, presence, very thoroughness, or not applicable of components (18). The percentage of possible scoring was taken into account. Talwalker JS et al., classified 31 summary elements under seven broad headings: basic elements, history, physical exam, data, hospital course, discharge plan, and overall (4). They used a five-point Likert scale for scoring summary elements. Rao P et al., scored the summary based on the elements that needed inclusion, unnecessary elements that needed exclusion, and also gave weightage for subjective components like clarity and consistency (19). Coit MH et al., scored 22 elements as 0, 1, or 2 out of 1 or 2 and gave different weightage with a multiplier of 1 or 2 depending on their deemed importance in the summary (20). They also developed a rubric for scoring. The percentage of possible scores earned was taken into account. This scoring system was later used by Axon RN et al., with some modifications (21).

Skills like documentation and communication were traditionally not addressed in medical education and were expected to be gained with experience (3). Although “communication” is given its importance in the newly adopted medical education policy in India, “documentation” has not been addressed (22). Earlier, various strategies including educational intervention (didactic lecture or small group teaching), audit and feedback, use of a template, reminder, etc., were adopted to improve discharge documentation, but longitudinal effects were not assessed in most of these studies (14),(15),(16),(19). Talwalker JS et al., adopted a longitudinal monthly workshop that provided feedback and opportunity for peer assessment and demonstrated continued benefit for two consecutive years (4). Axon RN et al., implemented a multipronged approach including the DS curriculum with its online availability, template use, and individual and team-based feedback with academic detailing (21). Almidani E et al., focused entirely on work system changes by laying down a set of rules and quarterly record assessments for three years (17). Coit MH et al., concluded that reducing residents’ workload can improve the DS quality (20).

The current study tried a mixture of strategies suitable for the unit. It provided education in a small group, followed by an opportunity for peer assessment along with team-based feedback. Appropriate work system changes supported these two activities. Interventions were disseminated in the unit-based system, ensuring that each resident had exposure to education and peer assessment atleast twice a year. In an academic centre, each year, novices enter, and experienced residents leave. This makes it necessary to continue the tested interventions for continued quality improvement. However, when the standardised work system is established, a novice will not take long to demonstrate the desired output. The current study introduced a unique scoring system for paediatric DS assessment and tested its inter-rater reliability. Such a scoring system contextual to paediatric DS is not available in the literature. Though the scoring was aligned with the working documentation system, this can be explored and tested at other centres. One may relate the observed improvement to the Hawthorne Effect, a phenomenon of improved human behaviour and performance when aware of being observed (23). However, such an effect is generally short-lasting and cannot continue consecutively for two years. Moreover, neither the residents nor the unit consultants knew from which time frame summaries would be selected for testing.

Limitation(s)

By implementing a mixture of strategies simultaneously, the effect of individual strategies could not be tested. As the study aimed for overall improvement, a control group was not possible in this design. Different units delivered the educational intervention and peer assessment/ feedback activities, so the delivery style and preferences could be varied, albeit the content was agreed upon at the onset. The confidence and satisfaction of the involved participants before and after the programme were not assessed. However, it appeared that the program was well-received among the residents. In fact, the preparation of the improvised template [Annexure-4] was headed by the residents.

Conclusion

The total score improved significantly in 2018 compared to 2017, and the improvement sustained in 2019, as well. A similar trend was noted in the components of DS like history, physical examination and discharge plan. Considerable improvement in the quality of DSs prepared by residents in a teaching institute can be achieved with approaches like small group educational intervention, peer assessment with feedback and work system changes. Strategies addressing documentation skills could be incorporated into the undergraduate or postgraduate curriculum.

DOI and Others

DOI: 10.7860/JCDR/2024/63973.18916

Date of Submission: Mar 09, 2023
Date of Peer Review: Apr 28, 2023
Date of Acceptance: Oct 30, 2023
Date of Publishing: Jan 01, 2024

AUTHOR DECLARATION:
• Financial or Other Competing Interests: None
• Was Ethics Committee Approval obtained for this study? Yes
• Was informed consent obtained from the subjects involved in the study? No
• For any images presented appropriate consent has been obtained from the subjects. NA

PLAGIARISM CHECKING METHODS:
• Plagiarism X-checker: Mar 21, 2023
• Manual Googling: May 10, 2023
• iThenticate Software: Oct 16, 2023 (2%)

ETYMOLOGY: Author Origin

EMENDATIONS: 7