Laboratory diagnostics plays a crucial role in healthcare setting for accurate diagnosis and patient management. A clinical laboratory should always strive to achieve highest quality goals and provide accurate and reliable test results. Laboratory testing processes can be divided in three phases i.e., preanalytical, analytical and postanalytical. Quality improvement measures can effectively prevent the errors in each phase of the laboratory processes [1]. Quality Control (QC) in the analytical phase is done by using Internal Quality Control (IQC) and External Quality Control (EQC) in order to maintain accuracy and precision of the laboratory results [2]. But, QC cannot detect the number of errors that occur in the laboratory during analytical phase. This justifies the need for adoption of quality management strategies like Six-Sigma for quality assurance. Six-Sigma is a quality management system which was developed by Bill Smith (an engineer in Motorola) in 1980’s to make improvements by identifying errors and mistakes. It is being widely implemented in business, industry and healthcare sector [3].
Six-Sigma is the ultimate benchmark of all processes that can fit in six Standard Deviations (SD) on either side of mean. It is uniform way of describing quality in terms of defects per million opportunities. Six-Sigma performances represent 3.4 defects per million operations [4]. Sigma metrics estimation can effectively evaluate existing QC processes in a laboratory for any shortcomings, and help in selecting optimum QC frequency and multi-rules for each analyte. Thus, it helps in improving accuracy and error detection rate of analytical tests and reducing the false rejection rate [5]. Quality Goal Index (QGI) indicates the problems of inaccuracy and imprecision for the analytes with respect to their quality goals [6]. Various Studies [7-11] have been done to estimate sigma metrics in clinical laboratories to quantitatively evaluate errors and improvise QC strategies. The present study was carried out to estimate Sigma Metrics of various biochemical analytes and to estimate the performance of QC in the Central Clinical Biochemistry Laboratory (CCBL). Further, Quality Goal Index (QGI) was also calculated to identify the problems of inaccuracy and imprecision for analytes having lower sigma values.
Materials and Methods
The present observational study was conducted in the year 2020 at the CCBL of Seth GS Medical College and KEM Hospital in Mumbai after obtaining Institutional Ethics Committee approval (EC/OA-113/2020).
Data was collected retrospectively for a period of six months (July 2019 to December 2019). Sigma Metrics was calculated for biochemical analytes run on two automated AU 680 clinical chemistry analysers (Beckman Coulter, Inc. Brea, California, United States) and on Rx Imola clinical chemistry analyser (Randox laboratories, Crumlin, United Kingdom).
The biochemical analytes assayed on AU 680 were-Bilirubin Direct, Bilirubin Total, Calcium, Chloride, Glucose, Potassium, Total protein, Sodium, Urea, Albumin, Aspartate Transaminase (AST), Alanine Transaminase (ALT), Creatinine, Inorganic phosphate. The biochemical analytes assayed on Rx Imola were: Triglyceride (TAG), Total Cholesterol (TC), High Density Lipoprotein (HDL), Low Density Lipoprotein (LDL), Lactate Dehydrogenase (LDH) and Creatine Phosphokinase (CPK).
Internal Quality Control was performed daily before sample processing on both AU 680 and Rx Imola analysers using QC material (Normal-L1 and Pathological-L2) provided by Beckman Coulter for AU 680 instruments and Randox Laboratories for Rx Imola. The QC practices such as control material storage, reconstitution and analysis were done as per the manufacturer’s instructions. In the laboratory, IQC data was interpreted daily by using Levy Jennings’ charts and Westgard rules. The samples of patients were analysed only when the IQC results were within control limits. The Westgard’s rules 13s, 22s, were considered as rejection rules, and 12s as a warning rule for each analyte [1]. Maintenance and calibration of instruments were done regularly. EQC data was obtained by participating in monthly Randox International Quality Assessment Scheme (RIQAS) and biweekly Lipid cycles.
Statistical Analysis
Statistical analysis was performed using Microsoft office excel 2010 software. Mean, SD for biochemical analytes were calculated using IQC data. Coefficient of variation (CV%) was calculated by using laboratory mean and SD.
CV%=(SD×100)/Laboratory mean
Bias% was calculated by using RIQAS data, for each analyte. Peer group mean is the mean of all QC values of laboratories enrolled in the RIQAS program using the same instrument and method.
Bias%=(Peer group mean-Laboratory mean)/Peer group mean
Total Allowable Error% (TEa%) values of all biochemical analytes were adopted from Clinical Laboratories Improvement Act (CLIA) guidelines [12].
The sigma metrics for analytes was calculated by the following equation:
Sigma=(TEa%-Bias %)/CV%
TEa (observed) was calculated using the following formula:
TEa (Observed)=Bias%+(CV%×2)
Observed TEa was compared with TEa adopted from CLIA.
QGI was calculated by the following formula:
QGI=Bias/(1.5×CV)
QGI was estimated for analytes having less than 4 sigma. If QGI is <0.8, it indicates imprecision. QGI of 0.8 to 1.2 indicates both imprecision and inaccuracy and QGI >1.2 indicates inaccuracy [6].
Results
Sigma values, CV%, Bias%, TEa (observed) of 14 analytes at the QC material L1 and L2 were calculated for AU 680-1 and AU680-2 clinical chemistry analysers and are summarised in [Table/Fig-1,2], respectively. Sigma Metrics of 6 biochemical analytes at the QC material L1 and L2 was also calculated for Rx Imola clinical chemistry analyser and is summarised in [Table/Fig-3].
Sigma values, CV%, bias, and TEa of 14 biochemical analytes obtained using clinical chemistry analyser module AU 680-1.
| Sr. No. | Name of parameter | CV % | Bias % | TEa (CLIA) | Sigma | TEa (Observed) |
|---|
| L1 | L2 | L1 | L2 | L1 | L2 |
|---|
| 1 | Albumin | 1.2 | 1.6 | 1.4 | 8 | 5.5 | 4.1 | 3.8 | 4.6 |
| 2 | ALT | 2.4 | 2.9 | 2.3 | 15 | 5.3 | 4.4 | 7.1 | 8.1 |
| 3 | AST | 2.5 | 2.7 | 2.5 | 15 | 5.0 | 4.6 | 7.5 | 7.9 |
| 4 | Bilirubin direct | 3.6 | 3.2 | 2.8 | 20 | 4.8 | 5.4 | 10 | 9.2 |
| 5 | Bilirubin total | 3.3 | 3.3 | 3.1 | 20 | 5.1 | 5.3 | 9.7 | 9.5 |
| 6 | Calcium | 3.1 | 3.5 | 1.8 | 10 | 2.6 | 2.3 | 8.0 | 8.8 |
| 7 | Chloride | 1.7 | 2.0 | 0.6 | 5 | 2.6 | 2.2 | 4.0 | 4.6 |
| 8 | Creatinine | 2.5 | 2.6 | 1.6 | 10 | 3.4 | 3.2 | 6.6 | 6.8 |
| 9 | Glucose | 1.2 | 1.4 | 0.7 | 8 | 6.1 | 6.6 | 3.1 | 2.9 |
| 10 | Phosphate inorganic | 3.6 | 3.5 | 2.0 | 10 | 2.2 | 2.3 | 9.2 | 9.0 |
| 11 | Potassium | 1.2 | 1.4 | 0.8 | 5 | 3.5 | 3.0 | 3.2 | 3.6 |
| 12 | Total protein | 1.0 | 1.2 | 0.7 | 8 | 7.3 | 6.1 | 2.7 | 3.1 |
| 13 | Sodium | 0.9 | 1.1 | 0.6 | 4 | 3.8 | 3.1 | 2.4 | 2.8 |
| 14 | Urea | 1.3 | 1.1 | 1.0 | 9 | 6.2 | 7.3 | 3.6 | 3.2 |
AST: Aspartate transaminase; ALT: Alanine transaminase; CV: Coefficient of variation; CLIA: Clinical laboratories improvement act; TEa: Total allowable error
Sigma values, CV%, bias, and TEa, of 14 biochemical analytes obtained using clinical chemistry analyser module AU 680-2.
| Sr. No. | Name of analytes | CV % | Bias % | TEa (CLIA) | Sigma | TEa (Observed) |
|---|
| L1 | L2 | L1 | L2 | L1 | L2 |
|---|
| 1 | Albumin | 1.3 | 1.7 | 1.2 | 8 | 5.2 | 4.0 | 3.8 | 4.6 |
| 2 | ALT | 2.6 | 3.1 | 2.1 | 15 | 5.0 | 4.2 | 7.3 | 8.3 |
| 3 | AST | 2.4 | 2.6 | 2.4 | 15 | 5.3 | 4.8 | 7.2 | 7.6 |
| 4 | Bilirubin direct | 3.8 | 3.3 | 2.4 | 20 | 4.6 | 5.3 | 10 | 9.0 |
| 5 | Bilirubin total | 3.3 | 3.1 | 2.9 | 20 | 5.2 | 5.5 | 9.5 | 9.1 |
| 6 | Calcium | 3.1 | 3.7 | 2.5 | 10 | 2.4 | 2.0 | 8.7 | 9.9 |
| 7 | Chloride | 1.6 | 2.1 | 0.5 | 5 | 2.8 | 2.1 | 3.7 | 4.7 |
| 8 | Creatinine | 2.7 | 2.9 | 1.3 | 10 | 3.2 | 3.0 | 6.7 | 7.1 |
| 9 | Glucose | 1.2 | 1.1 | 0.6 | 8 | 6.2 | 6.7 | 3.0 | 2.8 |
| 10 | Phosphate inorganic | 3.9 | 3.7 | 1.8 | 10 | 2.1 | 2.2 | 9.6 | 9.1 |
| 11 | Potassium | 1.1 | 1.3 | 0.9 | 5 | 3.7 | 3.2 | 3.1 | 3.5 |
| 12 | Total protein | 0.9 | 1.1 | 1.0 | 8 | 7.8 | 6.4 | 2.8 | 3.2 |
| 13 | Sodium | 0.9 | 1.1 | 0.5 | 4 | 3.9 | 3.2 | 2.3 | 2.7 |
| 14 | Urea | 1.3 | 1.2 | 0.8 | 9 | 6.3 | 6.3 | 3.4 | 3.2 |
AST: Aspartate transaminase; ALT: Alanine transaminase; CV: Coefficient of variation; CLIA: Clinical laboratories improvement act; TEa: Total allowable error
Sigma values, CV%, bias, and TEa, of 6 biochemical analytes obtained using clinical chemistry analyser module RX Imola.
| Sr. No. | Analytes | CV% | Bias % | TEa (CLIA) | Sigma | TEa (Observed) |
|---|
| L1 | L2 | L1 | L2 | L1 | L2 |
|---|
| 1 | TAG | 2.0 | 1.8 | 1.0 | 15 | 7.0 | 7.8 | 5.0 | 4.6 |
| 2 | Total cholesterol | 1.6 | 1.8 | 0.9 | 10 | 5.7 | 5.1 | 4.1 | 4.5 |
| 3 | CPK | 3.5 | 3.2 | 3.9 | 20 | 4.6 | 5.0 | 10.9 | 10.3 |
| 4 | LDH | 3.0 | 2.8 | 1.5 | 15 | 4.5 | 4.8 | 7.5 | 7.1 |
| 5 | HDL | 2.9 | 3.1 | 1.2 | 20 | 6.5 | 6.1 | 7.0 | 7.4 |
| 6 | LDL | 3.1 | 2.8 | 1.1 | 20 | 6.1 | 6.8 | 7.3 | 6.7 |
CV: Coefficient of variation; CLIA: Clinical laboratories improvement Act; TEa: Total allowable error; TAG: Triglyceride; CPK: Creatine phosphokinase; LDH: Lactate dehydrogenase; HDL: High density lipoprotein; LDL: Low density lipoprotein
According to the sigma value, biochemical analytes were divided into 5 categories- excellent, (sigma ≥6), good (sigma 4-6), fair (sigma 3-4), marginal (sigma 2-3), unacceptable (sigma <2) as shown in [Table/Fig-4] [13]. The analytes with sigma value <3 were considered as poor performers.
Sigma metrics performance of various analytes.
| Performance | Unacceptable | Marginal | Fair | Good | Excellent |
|---|
| Sigma | <2 | 2-3 | 3-4 | 4-6 | >6 |
|---|
| AU 680 1 | L-1 | Nil | Calcium, Chloride, Inorganic phosphorus | Creatinine, Sodium, Potassium | Albumin, ALT, AST, Bilirubin (Total, Direct) | Total protein Glucose, Urea |
| L-2 | Nil | Calcium, Chloride, Inorganic phosphorus | Creatinine, Sodium, Potassium | Albumin, ALT, AST, Bilirubin (Total, Direct) | Total protein, Glucose, Urea |
| AU 680-2 | L-1 | Nil | Calcium, Chloride, Inorganic phosphorus | Creatinine, Sodium, Potassium | Albumin, ALT, AST, Bilirubin (Total, Direct) | Total protein, Glucose, Urea |
| L-2 | Nil | Calcium, Chloride, Inorganic phosphorus | Creatinine, Sodium, Potassium | Albumin, ALT, AST, Bilirubin (Total, Direct) | Total protein, Glucose, Urea |
| Rx Imola | L-1 | Nil | Nil | Nil | Total Cholesterol, CPK, LDH | TAG, HDL, LDL |
| L-2 | Nil | Nil | Nil | Total Cholesterol, CPK, LDH | TAG, HDL, LDL |
AST: Aspartate transaminase; ALT: Alanine transaminase; TAG: Triglyceride; CPK: Creatine phosphokinase; LDH: Lactate dehydrogenase; HDL: High density lipoprotein; LDL: Low density lipoprotein
Total protein, Glucose, Urea on AU 680, TAG, HDL, LDL on Rx Imola for L1 and L2 were the analytes which showed excellent world class performance while the analytes with poor performance (sigma <3) were Calcium, Chloride and Inorganic phosphate.
QGI ratio of biochemical analytes with sigma value <4 (Creatinine, Sodium, Potassium, Chloride, Inorganic Phosphorus, Calcium) for both QC material L1 and L2 was less than 0.8; hence the performance was attributed to problem of imprecision [Table/Fig-5].
Quality goal index analysis for analytes with sigma value <4.
| Instrument | Analyte | Level 1 | Level 2 | Problem |
|---|
| AU 680 1 | Creatinine | 0.4 | 0.4 | Imprecision |
| Sodium | 0.4 | 0.4 |
| Potassium | 0.4 | 0.4 |
| Calcium | 0.4 | 0.3 |
| Chloride | 0.2 | 0.2 |
| Inorganic phosphorus | 0.4 | 0.4 |
| AU 680 2 | Creatinine | 0.3 | 0.3 | Imprecision |
| Sodium | 0.4 | 0.3 |
| Potassium | 0.5 | 0.5 |
| Calcium | 0.5 | 0.5 |
| Chloride | 0.2 | 0.2 |
| Inorganic phosphorus | 0.3 | 0.3 |
Discussion
Sigma Metrics is reliable tool used by clinical laboratories which ensures quality by identifying defects in term of precision and accuracy quantitatively. Sigma performance of 6 is considered as excellent performance. Sigma value of 3 is considered as minimum acceptable level of quality. Sigma performance of an analyte can be used to select appropriate control rules and frequency of runs for a method [8]. In this study, Sigma Metrics of 20 analytes (14 analytes on two AU680 clinical chemistry analysers and 6 analytes on Rx Imola clinical chemistry analyser) was analysed. In the CCBL, Westgard rules followed daily as a part of Statistical QC procedures (SQC) are- 12s/22s/13s with two control measurements (L1 and L2) once daily. These rules are used for all the analytes measured in the laboratory. QC is repeated only when any out of control value is observed for particular analyte after taking corrective actions.
Westgard recently described SQC procedures based on Sigma Metrics-Run Size Matrix and Westgard Sigma Rules with Run Size which includes three parameters: 1) selection of appropriate Westgard Sigma Rules; 2) total number of control measurements per SQC event (N); and 3) frequency of SQC events (Run size (R) of patient samples between SQC events [10,14]. Same was implemented in the biochemistry laboratory to achieve <5% probability of false rejection (Pfr) and ≥90% probability of error detection (Ped) for various biochemical analytes as per new Clinical and Laboratory Standards Institute (CLSI) C24-Ed4, guideline [10].
For analytes with sigma value of >6 (Total protein, Glucose, Urea on both AU 680 analyser, TAG, HDL, LDL on Rx Imola for both L1 and L2),13s rule with two control measurements (N2) per QC event and a run size of 1000 patient samples between QC events (R1000) was adopted [10,14]. However, controls should be analysed every day [14]. Albumin, AST, ALT, Bilirubin (Total, Direct) on both AU 680 for L1, L2 and Total Cholesterol, CPK, LDH for L1 and L2 on Rx Imola were good performers having sigma value of 4-6. Good performers with sigma 5-6 require 13s, 22s, R4s multirules with N2, R 450, and analytes with sigma 4-5 require 13s, 22s, R4s, 41s with N2, R200. For the analytes with sigma value <4 (Creatinine, Sodium, Potassium, Chloride, Calcium, Inorganic phosphorus for one or both QC material levels, multi-rules (13s/22s/R4s/41s/6x) with N4 and R45, were adopted [14]. The SQC procedures for various biochemical analytes according to their sigma value are detailed in [Table/Fig-6].
Statistical Quality Control (QC) procedures selected for biochemical analytes [14].
| Sigma of analytes | Westgard sigma rules | Number of control levels (N), run size patient samples between QC events (R) |
|---|
| >6 | 13S | N2 and R1000 |
| 5-6 | 13S/22s/R4s | N2 and R450 |
| 4-5 | 13s/22s/R4s/41s | N4 and R200 |
| <4 | 13s/22s/R4s/41s/6x | N4 and R45 |
As revealed by QGI, the main reason of poor performance of 6 analytes (creatinine, sodium, potassium, calcium, chloride, and inorganic phosphorus) was imprecision. Staff training and review of Standard Operating Procedures (SOP) was done for these analytes to improve method performance. The quality of test results is dependent on various factors such as reagents quality, type and quality of QC materials, types of analysers, methodology followed, environmental conditions, training and personal competency of laboratory staff performing the tests. Hence, during root causes analysis, various aspects associated with methodology, materials, personnel, equipment, working conditions were investigated.
Laboratory staff training (for reagent preparation and control material reconstitution, instrument maintenance, reagent handling, storage), periodical competency assessment program was introduced to improve their attitude and knowledge in order to improve precision for biochemical analytes having low sigma values. Further, SOPs were also reviewed for those analytes having low sigma values and rewritten in simpler and user-friendly manner. In present study, on both AU-680 instruments, sigma performance of 14 biochemical analytes was comparable and analytes which showed sigma value <4 were same on both instruments. It indicates that there was no inter-instrument variability in term of instrument performance. The variation in sigma metrics by various studies can be due to difference in methods of analysis, instruments, IQC material, difference in bias calculated due to different EQC testing bodies [8-11].
Within analyte variation of sigma values of L1 and L2 was seen for some analytes as depicted in [Table/Fig-1,2 and 3]. This can be due to variation in method performance at normal and higher concentration level for particular analyte [10]. Strict SQC procedures should be followed to abolish such variation. The effective QC design based on sigma metrics ensures quality performance by quickly detecting medically significant errors. Fewest number of control measurements for analytes with sigma >6, save cost associated with QC materials, reagents and consumables [13]. Hence, quality improvement and cost reduction can go hand in hand by adopting quality management strategies like Six-Sigma. It serves the quality goal of laboratory to produce accurate results for correct and timely diagnosis which in turn can help in improving health of society by reducing morbidity and mortality.
Limitation(s)
Data to show effectiveness of new SQC procedures designed for all analytes is not included in this paper. A study of sigma metrics analysis with revised and modified QC strategy for analytes having low sigma value is under process.
Conclusion(s)
Six-Sigma is a data driven process improvement system which relies on measuring processes and making improvements by reducing errors and defects. Sigma Metrics helps to identify the shortcomings in the existing QC strategy, determine sources of variation and factors influencing laboratory processes to achieve benchmark quality goals of a laboratory. Six-Sigma metrics is easily adaptable to clinical laboratory and moves the laboratory towards proactivity, quality performance and continuous improvement.
Before this study, common QC strategy was being used for all analytes. Use of optimum QC procedures reduced unnecessary QC runs and associated cost, for analytes with the high sigma metrics result. Focused and effective QC strategy for analytes having low sigma improved the performance of those analytes.
AST: Aspartate transaminase; ALT: Alanine transaminase; CV: Coefficient of variation; CLIA: Clinical laboratories improvement act; TEa: Total allowable errorAST: Aspartate transaminase; ALT: Alanine transaminase; CV: Coefficient of variation; CLIA: Clinical laboratories improvement act; TEa: Total allowable errorCV: Coefficient of variation; CLIA: Clinical laboratories improvement Act; TEa: Total allowable error; TAG: Triglyceride; CPK: Creatine phosphokinase; LDH: Lactate dehydrogenase; HDL: High density lipoprotein; LDL: Low density lipoproteinAST: Aspartate transaminase; ALT: Alanine transaminase; TAG: Triglyceride; CPK: Creatine phosphokinase; LDH: Lactate dehydrogenase; HDL: High density lipoprotein; LDL: Low density lipoprotein