Year :
2023
| Month :
December
| Volume :
17
| Issue :
12
| Page :
PC09 - PC12
Full Version
Inter-observer Reliability in Reporting Complications of Transurethral Resection of the Prostate with Clavien-Dindo Classification: A Retrospective Observational Study
Published: December 1, 2023 | DOI: https://doi.org/10.7860/JCDR/2023/67033.18832
Pushpendra Kumar Shukla, Vivek Sharma, Avinash Pratap Singh Thakur, Ashish Ghanghoria, Varsha Shukla
1. Associate Professor, Department of Urology, Shyam Shah Medical College, Rewa, Madhya Pradesh, India.
2. Associate Professor, Department of Urology, Shyam Shah Medical College, Rewa, Madhya Pradesh, India.
3. Associate Professor, Department of Urology, NSCB Medical College, Jabalpur, Madhya Pradesh, India.
4. Assistant Professor, Department of Urology, Shyam Shah Medical College, Rewa, Madhya Pradesh, India.
5. Assistant Professor, Department of Surgery, Shyam Shah Medical College, Rewa, Madhya Pradesh, India.
Correspondence Address :
Dr. Pushpendra Kumar Shukla,
Associate Professor, Department of Urology, Shyam Shah Medical College, Rewa-486001, Madhya Pradesh, India.
E-mail: pushpendra2507@gmail.com
Abstract 
Introduction: Transurethral Resection of the Prostate (TURP) is a gold standard treatment for Benign Prostatic Hyperplasia (BPH). The Clavien classification system was proposed in 1992 to grade complications of general surgery, which was then modified in 2004 by Dindo. In 2017, the European Association of Urology (EAU) endorsed its use for reporting complications in urological procedures. However, the system relies on observers to grade complications, and different observers may assign different grades to the same negative outcome, highlighting the need for uniformity.
Aim: To assess the inter-observer reliability and uniformity among urologists in reporting complications of TURP using the modified Clavien-Dindo classification system.
Materials and Methods: A retrospective observational study was conducted at Department of Urology, Shyam Shah Medical College, Rewa, Madhya Pradesh, India, from January 2018 to May 2023. The study included symptomatic patients with BPH and prostate gland sizes greater than 40 cc. Negative outcomes were evaluated and complications were rated by nine different raters. Inter-observer reliability was tested using weighted kappa with Statistical Package for Social Sciences (SPSS) version 21.0 (Fleiss’s kappa test).
Results: A total of 523 men with BPH who underwent TURP were evaluated, with a mean age of 66.7±5.2 years and a mean prostate volume of 62 cc. Adverse events occurred in 96 patients (18.36%), and these complications were classified according to the Clavien-Dindo System (CDS) by nine different blinded observers. The rating of these complications was then assessed for concordance. A concordance rate using weighted kappa of 0.847 was observed, indicating high inter-observer reliability for TURP.
Conclusion: Clavien-Dindo’s system demonstrated good inter-observer concordance and reliability. Some modifications to the grading system are needed, taking into account the impact of pre-existing co-morbidities and auxiliary procedures. Large-scale prospective studies are recommended to further validate these findings.
Keywords
Benign prostatic hyperplasia, Clavien-Dindo system, Complication rating, Concordance
Introduction
The TURP is considered a gold standard treatment for BPH due to its clinical effectiveness and economic viability. However, despite being a relatively simple and safe procedure, it has certain drawbacks. The Clavien classification system was initially proposed in 1992 to grade complications in general surgery (1). Originally used for laparoscopic cholecystectomy, its applicability was expanded to other surgeries. In 2004, Dindo modified the system to encompass all surgical procedures after validation it in a study involving a substantial patient cohort undergoing elective general surgery (2). Subsequently, EAU endorsed the use of the Clavien-Dindo system for grading and reporting postoperative complications in urology (3). However, the existing literature lacks a standardised methodology for assessing the reliability of complication reporting. Conducting a study with multiple raters rating simultaneously, who are blind to each other and to the operating surgeon, can help evaluate the agreement among the raters. This, in turn, will contribute to achieving consistency in reporting complications among the surgeons. With this background, the present study aimed to assess the interobserver reliability and uniformity among urologists in reporting complications of a common procedure such as TURP using the Clavien-Dindo classification system.
Material and Methods
A retrospective observational study was conducted at Department of Urology, Shyam Shah Medical College, Rewa, Madhya Pradesh, India, from January 2018 until May 2023. Institutional approval was obtained for the study (SN 10/MRD/SSB/2023).
Inclusion criteria: Symptomatic patients with BPH and gland size greater than 40 cc were included in the study.
Exclusion criteria: Patients with multiple co-morbidities (such as known cases of ischaemic heart disease, chronic renal failure, American Society of Anaesthesiologists (ASA) category ≥4, known cases of prostate cancer, or those whose histopathology report revealed carcinoma of the prostate after TURP) were excluded f romthe study.
All surgeries were performed using the standard Mauer Mayer’s technique of TURP. Bladder irrigation was discontinued after 24 hours, and the catheter was removed on the second day following the procedure, unless otherwise indicated by institutional protocol. Patients were typically discharged on the day following catheter removal. Preoperative data, including patient age, gland size, and indication for surgery, were recorded. All complications occurring during the perioperative period were independently classified according to the modified Clavien-Dindo System (Table/Fig 1) (2) by nine observers who were not involved in the operative procedure. In cases where two or more complications were present, they were placed into multiple categories as indicated, and the patient was classified with the higher grade for comparison. The raters individually reviewed the patient notes and postoperative complications without access to patient particulars, and assigned the patient to the category they deemed appropriate. The classification made by one observer was unknown to the other observer, ensuring a double-blind procedure. The rating data was collected by a separate set of coordinators who were unaware of how a patient (and their complications) was categorised by a specific observer. The results are presented as complication rates per grade.
Statistical Analysis
Inter-observer reliability was assessed using weighted kappa with SPSS 21.0 (Fleiss’s kappa test). For the purpose of this study, dysuria was not considered as it was regarded as a part of the natural postoperative course.
Results
A total of 523 patients were included in the study, with a mean age of 66.7±5.2 years. The mean prostate volume was 62 cc (range: 45-77 cc). The average Qmax with range was 7 mL/sec (interquartile range: 6-9 mL/sec), and the average International Prostate Symptom Score (IPSS) was 24±5.1. Among the patients, 116 (22.18%) presented with acute urinary retention. The mean creatinine level was 0.98±0.24 mg/dL. Concurrent diabetes mellitus and hypertension were present in 32 (6.11%) and 47 (8.98%) cases, respectively (Table/Fig 2).
Complications were observed and recorded in 96 patients during the first postoperative month, resulting in an overall perioperative morbidity rate of 18.36%. Each observer independently graded the complications using the aforementioned classification system. The Fleiss kappa value for inter-rater reliability (with standard error) was 0.847±0.03 (p-value <0.0001), as shown in (Table/Fig 3). For each grade, the inter-rater reliability ranged from good to excellent across all categories, with a range of 0.68 in Grade-I to 1.0 in Grade-V. The value of 1.0 for Grade-V is expected, as it represents cases with mortality, which warrants absolute concordance.
Discussion
In 1992, Clavien PA et al., proposed a systematic classification system based on therapeutic outcomes to rate surgical complications, providing an example of its utility in cholecystectomy (1). Mamoulakis C et al., utilised the Clavien-Dindo System to report complications in their study involving 198 patients with BPH who underwent TURP. They concluded that the modified Clavien-Dindo System offers a validated system that has been effectively implemented in several urological centres for major surgeries (4). Bansal A et al., employed the modified Clavien-Dindo System to classify complications of TURBT retrospectively in 968 patients over an eight-year period from 2006 to 2014 (5).
The Clavien-Dindo classification system can be easily utilised by urologists. De Nunzio C et al., conducted a comparative study between monopolar TURP and bipolar TURP to validate the use of this assessment tool in a contemporary cohort of patients. They reported an overall morbidity rate of 19% in the monopolar TURP group. This system provides the advantage of objectively comparing two or more procedures (6). Cai F et al., utilised the modified Clavien-Dindo System to compare three modes of prostate resection, namely TURP, plasma kinetic resection of the prostate, and Holmium laser resection of the prostate. They assessed the clinical value of the Clavien-Dindo System in standardising the evaluation of complications (7). However, they noted that further modifications to the classification may be necessary.
The modified Clavien-Dindo System offers a validated method that has been successfully adopted by several urological centres for grading complications in major oncologic procedures. Yoon PD et al., observed that the Clavien-Dindo system has been increasingly utilised in reporting complications in published papers (8). To further refine and standardise the classification, the Japan Clinical Oncology Group (JCOG) commissioned a committee to optimise the system for surgical issues, ensuring homogeneity and uniformity in documenting postoperative complications across various surgical fields (9). To assess the applicability of the Clavien-Dindo classification system in the elderly population, Khan A et al., conducted a retrospective study on postoperative problems in patients over 75 years who underwent open or laparoscopic nephrectomy/nephroureterectomy. They categorised these problems according to the Clavien-Dindo classification (10). The researchers found that the system was easy to use and effectively classified postoperative issues in the elderly population.
In the present study men between the ages of 40 and 70 years were included to ensure representation of a wider age group undergoing TURP. Sometimes discrepancies can arise when different methods are used to assess the same outcome, and it is desirable to resolve these discrepancies by reaching a consensus.
Surgical Site Infection (SSI) rates and severity using the Clavien-Dindo classification were compared by Yamamoto T et al., (11) to investigate any association between treatment duration and the depth of SSI. Multivariate analysis was also conducted, revealing that patients with grade III SSI in the Clavien-Dindo classification required a longer treatment duration. Thus, the prognostic value of this system is evident. In this study, authors aimed to assess the use of a modified Clavien-Dindo system for complications of TURP and examine its applicability for day-to-day reporting of complications on a clinical basis. Endourological procedures are unique in the sense that they may involve complications not typically found in laparoscopic and open procedures, such as TUR syndrome and a high incidence of pulmonary embolism, as well as mild derangement of renal function tests. Sometimes these complications result from existing diseases that do not fit well within the modified CDS continuum. Simultaneously, it is important to evaluate observer variation and inter-observer reliability within a specific procedure. In this regard, present study determined TURP to be the most suitable procedure, as it is one of the most commonly performed procedures in Urology and has historically shown low mortality and morbidity rates, minimising bias among observers. inter-observer reliability was tested using weighted kappa, resulting in a high kappa value of 0.847, indicating strong agreement among the raters. To the best of our knowledge, no previous study has examined the concordance of Clavien-Dindo among observers using a single procedure. To assess the inter-observer variability of Clavien-Dindo scoring, a multicentre thematic survey was conducted by Poletajew S et al., (12). The responders were asked to grade nine typical urological complications according to the Clavien-Dindo classification, which ranged in severity. Inter-observer agreement, simulation of answers, and factors affecting variability were analysed. While there was some incongruity in grading local complications after transurethral surgery (39% accordance), the study demonstrated the simplicity, reproducibility, and logical scheme of the Clavien-Dindo classification. Various scenarios were presented for rating, aiming to evaluate the overall concordance of TURP, a prospectively evaluated urological procedure. Dodwell ER et al., (13) performed a similar study by presenting different surgical scenarios to surgeons at their centre to test internal reliability after training them in using this system. Additionally, they conducted a survey with 48 scenarios to test external reliability. The Clavien-Dindo system does not account for pre-existing diseases that can manifest as complications (14). Furthermore, different surgeons with varying techniques and learning curves can influence surgical outcomes. Complications related to the respiratory and cardiac systems may also result from anaesthetic procedures, requiring appropriate modifications to the system. However, it is reasonable to consider including intraoperative complications, as they ultimately impact postoperative outcomes.
Validation and standardisation of this classification for all urological procedures is also needed through large cohort studies. The same complications are graded differently in different hospitals according to their protocols. The type of anaesthesia affects the outcome, so the mode of anaesthesia administration can change the grade. This classification system does not take this into account. Singh AK et al., also pointed this out in their study on percutaneous nephrolithotomy (15). A procedure may be performed under general anaesthesia, local anaesthesia, or regional anaesthesia, resulting in varying outcomes. Consequently, patients grouped under Category 3a in one setting may find themselves placed in Category 3b at another hospital. The subdivision of Group-3 is based on the type of anaesthesia required, not the surgical procedure. The system lacks a mechanism to grade complications attributed to accompanying procedures. Overall, present study found that the CDS is a less arduous, time-saving, and easily applicable tool for grading perioperative TURP complications.
Limitation(s)
It was performed at a small centre for a single procedure. The entry of minor complications may not have been marked as negative outcomes due to incongruity in reporting. TURP is a surgery with a limited duration. Longer resection times are associated with complications such as haematuria and TUR syndrome, which were not studied. The type of anaesthesia was not included as a variable in present study. Different complications manifest as a spectrum, such as haematuria, which can range from mild to severe depending on the observer’s individual decision. Some long-term complications, like urethral stricture, were not included in this study, as only the early postoperative period was covered.
Conclusion
The CDS demonstrated good inter-observer concordance and reliability in present study. However, certain modifications in the grading system are needed, taking into consideration the impact of pre-existing co-morbidities and auxiliary procedures. Large-scale prospective studies are recommended to further investigate these aspects.
Reference
|
| | 1. | Clavien PA, Sanabria JR, Strasberg SM. Proposed classification of complications of surgery with examples of utility in cholecystectomy. Surgery. 1992;111(5):518-26.
| | 2. | Dindo D, Demartines N, Clavien PA. Classification of surgical complications: A new proposal with evaluation in a cohort of 6336 patients and results of a survey. Ann Surg. 2004;240(2):205-13.
[ CrossRef] [ PubMed] | | 3. | Mitropoulos D, Artibani W, Biyani CS, Bjerggaard Jensen J, Rouprêt M, Truss M. Validation of the Clavien-Dindo Grading System in Urology by the European Association of Urology Guidelines Ad Hoc Panel. Eur Urol Focus. 2018;4(4):608-13.
[ CrossRef] [ PubMed] | | 4. | Mamoulakis C, Efthimiou I, Kazoulis S, Christoulakis I, Sofras F. The modified Clavien classification system: A standardised platform for reporting complications in transurethral resection of the prostate. World J Urol. 2011;29(2):205-10.
[ CrossRef] [ PubMed] | | 5. | Bansal A, Sanchar S, Goel A, Kumar M, Purkait B, Aeron R. Grading of complications of transurethral resection of bladder tumor using Clavien–Dindo classification system. Indian J Urol. 2016;32(3):232-37.
[ CrossRef] [ PubMed] | | 6. | De Nunzio C, Franco G, Candoli L, Autorino R, Cicione A, Perdona S, et al. Transurethral Resection of the Bladder (TURB): Analysis of complications using a modified Clavien system in an Italian real-life cohort. Eury J Surg Oncol. 2014;40(1):90-95.
[ CrossRef] [ PubMed] | | 7. | Cai F, Chen C, Zhang J. Applicability of Clavien Dindo classification for comparing complications of three endoscopic procedures for benign prostatic hyperplasia. Nan Fang Yi Ke Da Xue Bao. 2015;35(9):1344-48.
| | 8. | Yoon PD, Chalasani V, Woo HH. Use of Clavien-Dindo classification in reporting and grading complications after urological surgical procedures: Analysis of 2010 to 2012. J Urol. 2013;190(4):1271-74.
[ CrossRef] [ PubMed] | | 9. | Katayama H, Kurokawa Y, Nakamura K, Ito H, Kanemitsu Y, Masuda N, et al. Extended Clavien-Dindo classification of surgical complications: Japan Clinical Oncology Group postoperative complications criteria. Surg Today. 2016;46(6):668-85.
[ CrossRef] [ PubMed] | | 10. | Khan A, Palit V, Myatt A, Cartledge JJ, Browning AJ, Joyce AD, et al. Assessment of Clavien-Dindo classification in patients >75 years undergoing nephrectomy/nephroureterectomy. Urol Ann. 2013;5(1):18-22.
[ CrossRef] [ PubMed] | | 11. | Yamamoto T, Takahashi S, Ichihara K, Hiyama Y, Uehara T, Hashimoto J, et al. How do we understand the disagreement in the frequency of surgical site infection between the CDC and Clavien-Dindo classifications? J Infect Chemother. 2015;21(2):130-33.
[ CrossRef] [ PubMed] | | 12. | Poletajew S, Zapala L, Piotrowicz S, Wolyniec P, Sochaj M, Buraczyn´ ski P, et al. Interobserver variability of Clavien-Dindo scoring in urology. Int J Urol. 2014;21(12):1274-78.
[ CrossRef] [ PubMed] | | 13. | Dodwell ER, Pathy R, Widmann RF, Green DW, Scher DM, Blanco JS, et al. Reliability of the modified clavien-dindo-sink complication classification system in pediatric orthopaedic surgery. JB JS Open Access. 2018;3(4):e0020.
[ CrossRef] [ PubMed] | | 14. | Constantinides CA, Tyritzis SI, Skolarikos A, Liatsikos E, Zervas A, Deliveliotis C. Short- and long-term complications of open radical prostatectomy according to the Clavien classification system. BJU Int. 2009;103(3):336-40.
[ CrossRef] [ PubMed] | | 15. | Singh AK, Shukla PK, Khan SW, Rathee VS, Dwivedi US, Trivedi S. Using the Modified Clavien Grading System to classify complications of percutaneous nephrolithotomy. Curr Urol. 2018;11(2):79-84. Doi: 10.1159/000447198. Epub 2017 Dec 30. PMID: 29593466; PMCID: PMC5836180. [ CrossRef] [ PubMed] |
DOI: 10.7860/JCDR/2023/67033.18832
Date of Submission: Sep 01, 2023
Date of Peer Review: Sep 23, 2023
Date of Acceptance: Nov 24, 2023
Date of Publishing: Dec 01, 2023
AUTHOR DECLARATION:
• Financial or Other Competing Interests: None
• Was Ethics Committee Approval obtained for this study? Yes
• Was informed consent obtained from the subjects involved in the study? Yes
• For any images presented appropriate consent has been obtained from the subjects. NA
PLAGIARISM CHECKING METHODS:
• Plagiarism X-checker: Sep 07, 2023
• Manual Googling: 000 00, 0000
• iThenticate Software: Nov 18, 2023 (5%)
ETYMOLOGY: Author Origin
EMENDATIONS: 6
|