Internal Medicine Section DOI : 10.7860/JCDR/2019/39902.12561
Year : 2019 | Month : Feb | Volume : 13 | Issue : 02 Page : OC05 - SC08

Hyponatremia as a Mortality Predictor of Severe Malaria: A Hospital Based Cross-sectional Study

Manoj Parida1, Pravat Kumar Thatoi2, Anurag Choudhury3, Subhas Bhuin4, Sarita Behera5, Rina Mohanty6

1 Senior Resident, Department of Medicine, SCB Medical College, Cuttack, Odisha, India.
2 Associate Professor, Department of Medicine, PRM Medical College, Baripada, Odisha, India.
3 Junior Resident, Department of Medicine, SCB Medical College, Cuttack, Odisha, India.
4 Postgraduate Student, Department of Medicine, SCB Medical College, Cuttack, Odisha, India.
5 Assistant Professor, Department of Medicine, SCB Medical College, Cuttack, Odisha, India.
6 Associate Professor, Department of Medicine, SCB Medical College, Cuttack, Odisha, India.


NAME, ADDRESS, E-MAIL ID OF THE CORRESPONDING AUTHOR: Dr. Pravat Kumar Thatoi, Flat-104, Aryabhat Complex, College Square, Cuttack-753003, Odisha, India.
E-mail: drpravatthatoi@yahoo.co.in
Abstract

Introduction

It is well known that hyponatremia is common in severe malaria. However, there is scanty and conflicting literature regarding hyponatremia as a predictor of mortality in severe malaria.

Aim

To determine the prevalence of hyponatremia in severe malaria and its association with mortality.

Materials and Methods

It was a hospital based cross-divtional study conducted in a tertiary care referral hospital in the state of Odisha in India. Sample size was calculated to be 99 by using the prevalence of hyponatremia in severe malaria 55% with absolute precision of 10%. Taking into account of 10% drop out rates, the final sample size was determined to 109. Considering the prevalence of mixed plasmodium infection to be 13% and the proportion of falciparum to vivax infection to be 49:51, samples ≥45 in each group was adequate for comparison. Condivutive sampling was done over a period of two years. Severe malaria included either infection with Plasmodium falciparum alone or both Plasmodium falciparum and Plasmodium vivax along with any one feature of WHO criteria for severity. Patients aged more than 15 years with both smear positive and rapid card test positive were included. Vital parameters and electrolyte levels were measured for each patient. The data was analysed using SPSS and significance level was set at 95%.

Results

Out of the total number of 110 cases of severe malaria, isolated falciparum malaria was 57.3% (n=63) vs. mixed falciparum-vivax infection of 42.7% (n=47); overall mortality was 6.3% (n=4) vs. 14.8% (n=7) respectively. Hyponatremia was observed in 63.6% (n=70) of the total cases. The difference in incidence of hyponatremia in both the groups was 133±6 mEq/L vs. 127±6.3 mEq/L respectively and was statistically significant. Among the hyponatremic group of patients, the difference in level of sodium in both the group was (128±3.5 vs. 124±3.2) mEq/L and was statistically significant. Overall mortality rate was 10% (n=11/110), however mortality rate among the hyponatremic patients was 15.7% (n=11/70; 12.1% vs. 18.9% in both the groups respectively). Hyponatremia at a cut-off of 126 mEq/L predicted mortality with a sensitivity of 81.8%, specificity of 78.8%, and negative predictive value of 97.5%.

Conclusion

Hyponatremia was highly prevalent among the severe malarial patients. Higher degree of hyponatremia was observed in mixed plasmodium infection. Severe hyponatremia predicted mortality with high sensitivity and specificity.

Introduction

Severe malaria is a life threatening condition mostly caused by Plasmodium falciparum infection and is the major health problem in India particularly in the eastern coastal state like Odisha [1], where the environment is suitable for the vector. Unawareness about malaria and delay in starting treatment leads to life threatening complications like cerebral malaria, acute kidney injury, anaemia, acidosis, ARDS (Acute Respiratory Distress Syndrome), hepatopathy, and dyselectrolytemia [2]. Hyponatremia has been found as a complication in severe malaria [2-5]. In eastern coastal parts of India we find mainly two plasmodium species i.e., falciparum and vivax, causing life threatening malarial infection. Severe malaria is generally caused by Plasmodium falciparum infection or by mixed infection of both Plasmodium falciparum and Plasmodium vivax.

There are studies showing hyponatremia associated with severe Plasmodium falciparum infection in children [2,5,6]. But few studies focus its association with disease severity and outcome in adults with severe malaria. Studies suggest that increased secretion of vasopressin, repeated vomiting and improper action of anti-diuretic hormone secretion play an important role in the pathophysiology of hyponatremia in severe malaria [4,7,8].

Hyponatremia is a state where plasma sodium concentration is less than 135 mEq/L. Conventionally hyponatremia is categorised into mild, moderate and severe according to plasma sodium level. Mild hyponatremia is when plasma sodium is between 130-134 mEq/L, moderate hyponatremia is between125-129 mEq/L and severe hyponatremia is <125 mEq/L [9,10].

The aim of this study is to determine the prevalence of hyponatremia in severe malaria and its association with mortality in a referral hospital.

Materials and Methods

Study Setting and Design

It was a hospital based cross-sectional study carried out in the Department of General Medicine of a tertiary care teaching and referral hospital in the state of Odisha in India. After getting approval from Institutional Ethics Committee (IEC/IRB No. 23/ 9.10.2013; S.C.B. Medical College, Cuttack, Odisha), the study was conducted by involving the patients admitted to the medical wards and ICU over a period of two years, from May 2014 to April 2016.

Sample Size and Sampling

Sample size was calculated to be 99 by using the prevalence of hyponatremia in severe malaria 55% with absolute precision of 10% [11,12]. Taking into account of 10% drop out rates, the final sample size was determined to 109. Considering the prevalence of mixed plasmodium infection in India to be 13% [13], the sample size in the mixed plasmodium infection group was calculated to 45 [11-13]. As the proportion of Plasmodium falciparum to Plasmodiumvivax in India is 49:51 [13], a sample size of ≥ 45 in the Plasmodium falciparum group was considered adequate to be comparable. Consecutive sampling of 110 patients was done over a period of two years to meet the adequate sample.

Subjects

Inclusion criteria:

All smear positive or rapid card test positive patients of severe malaria above 15 years of age.

Willingness to participate by themselves or by relatives.

Informed consent provided by the subjects (written or verbal) or by their attendants (written only) after being explained in details regarding the study in local Odia language.

Exclusion criteria:

Patients of age less than 15 years of age.

Patients on diuretics and any other medication or disease which causes low sodium.

Case Definition of Severe Malaria

Both smear positive and rapid card test positive malaria with features of severity as per existing WHO criteria, were taken as cases [2,8].

A Glasgow Coma Scale (GCS) score <11 (indicating cerebral malaria) or

Anaemia (haematocrit <0.20 L/L with parasite count >100.000/μL) or

Jaundice (serum bilirubin >50 μmol/L with parasite count >100.000 μL) or

Renal impairment (urine output <400 mL/24 h and serum creatinine >250 μmol/L) or

Hypoglycaemia (blood glucose <2.2 mmol/L) or

Hyperparasitaemia (>10% parasitaemia) or

Shock (systolic blood pressure <80 mm Hg with cold extremities).

A careful history was obtained and detailed clinical examination was performed in each patient. Patients were recruited into the study as per the case definition. These cases were divided into two groups according to isolated Plasmodium falciparum infection or mixed plasmodium infection i.e., both Plasmodium falciparum and Plasmodium vivax infection.

Sample Collection

About three mL of blood was collected at the time of admission and sent for following laboratory investigations. Complete Blood Counts (CBC), blood glucose, renal and liver function tests were done in every patient including thick and thin smears and rapid card test for malaria parasite detection.

CBC was done by Pentra ES 60 (HORIBA MEDICAL) automated haematology analyser

Malaria parasite peripheral smears (thick and thin films) by staining with Giemsa stain

Malaria antigen detection test by Plasmodium falciparum Histidine-Rich-Protein 2 and Plasmodium vivax lactate dehydrogenase screening (ICT Malaria, Binax).

Evaluation of serum electrolytes (Na, K) by 9180 electrolyte analyser (Roche)

Evaluation of serum urea, creatinine and LFT by CobasIntegra-400 (Roche) biochemical analyser.

Statistical Analysis

The observed data was entered in Microsoft Excel and was statistically analysed by using Statistical Package for Social Sciences version 16 (PASW statistics for Windows, Chicago: SPSS Inc.). All the data were checked for homogeneity. The qualitative data were expressed as frequencies and percentages and comparisons of categorical variables between the two groups were performed with Chi-square test. The continuous variables were expressed as mean and standard deviation and compared by using student’s t-test. ROC curve was plotted to determine the cut-off level of serum sodium in predicting the mortality in severe malaria patients. A p-value of <0.05 was considered significant and p-value of <0.001 was considered highly significant.

Results

Out of the total number of 110 cases, males were 64 (58%) and females were 46 (41.8%). Only P.falciparum infection were 63 (57.3%) vs. mixed infection were 47 (42.7%). Among the 63 isolated falciparum malaria cases, 38 cases were observed in male and among the 47 mixed malaria cases, 26 cases were observed in male. Mortality among males and females were 7.8% vs. 13% [Table/Fig-1].

Gender wise distribution of data (N=110).

ParametersMale (N=64; 58.2%)Female (N=46; 41.8%)
Age (years)39.8±7.639.9±7.3
15-3012 (18.7)8 (17.4)
31-4017 (26.6)14 (30.4)
41-5031 (48.4)22 (47.8)
51-604 (6.3)2 (4.34)
Plasmodium species
P.falciparum38 (59.4)25 (54.3)
Mixed infection26 (40.6)21 (45.7)
Outcome
Improved59 (92.2)40 (87)
Death5 (7.8)6 (13)

The physical vital signs at presentation, serum electrolytes and outcome in two groups are provided in [Table/Fig-2]. Mortality in isolated falciparum group was 6.3% vs. mortality in mixed malarial infection was 14.8% as presented in [Table/Fig-2]. [Table/Fig-3] shows that the distribution of hyponatremia among the severe malaria cases was normally distributed with mean serum sodium of 130.6 mEq/L. The level of sodium in isolated falciparum malaria cases vs. mixed malaria cases were (133±6 vs. 127±6.3) mEq/L. This difference was statistically significant [Table/Fig-2].

Analysis of clinical parameters of patients with severe malaria (N=110).

ParametersP. falciparum (n=63) Mean±SD; n (%)Mixed infection (n=47) Mean±SD; n (%)p-value
Demographics
Age (years)39.5±7.840.7±7.10.549
Sex (male, female)38 (60.3), 25(39.6)26 (55.3), 21 (44.6)0.59
Vital signs
Pulse rate (bpm)100.7±8.2105.7±10.20.006
Systolic BP (mmHg)109.1±10.5106.9±11.60.3
Respiratory rate (rpm)24.7±3.325.2±4.30.496
Temperature (°F)100.7±0.9101.2±1.10.054
GCS <119 (14.2)7 (14.8)-
Laboratory parameters
Sr.Na (mEq/L)133±6127.3±6.3<0.001
Sr.K (mEq/L)3.2±0.63.4±0.80.336
Outcome
Improved, Death59 (93.6), 4 (6.3)40 (85.1), 7 (14.8)0.139

Distribution of serum sodium levels among severe malaria patients (n=110).

Amongst the 110 severe malaria cases, 70 patients (63.6%) had different degrees of hyponatremia. Hyponatremia was more common among the patients with mixed infection than those affected by isolated falciparum malaria and the difference was significant [Table/Fig-4]. Only among the hyponatremic group of patients the level of sodium in isolated falicarum malaria cases vs. mixed malaria cases were (128±3.5 vs. 124±3.2) mEq/L. This difference was highly significant [Table/Fig-4]. All the 11 cases of death occurred in patients with hyponatremia. Among the non-selected malaria cases the death rate was 11 out of 110 cases i.e., 10% (isolated falciparum vs. mixed malaria death rates were 4/63=6.3% vs. 7/47=14.8%). Analysing the data only among the population of hyponatremic patients, the total death rate was 11/70=15.7%. On further analysis of the data of hyponatremic patients the death rate among isolated falciparum vs. mixed malaria cases was 4/33=12.1% vs. 7/37=18.9%. This difference in mortality was not statistically significant. However, comparing the mortality prognosis (improvement vs. death) among the hyponatremic group of patients, the difference was highly significant. Serum sodium less than 126 mEq/L can predict mortality with a sensitivity of 81.8% and specificity of 78.8% [Table/Fig-5].

Analysis among hyponatremic patients (n=70).

ParametersSerum sodium (mEq/L) Mean±SDp-value
P. falciparum (n=33)Mixed infection (n=37)128±3.5124.5±3.2<0.001
Improved (n=59)Death (n=11)126.7±3.6122.8±2.8<0.001

Hyponatremia (cut-off <126 mEq/L) predicating mortality in severe malaria cases.

Area under curve: 0.876 (95% CI: 0.8-0.95; p< 0.001)

Discussion

Hyponatremia is a highly prevalent electrolyte disorder among the patients diagnosed with severe malaria in a tertiary care referral hospital. In the current study, patients infected with either P.falciparum or both P.falciparum and P.vivax were considered as severe malaria upon fulfilling the criteria for severity as per WHO. The present study was carried out in the light of filling the gaps in knowledge regarding the degree of hyponatremia and mortality associated with severe malaria, which often leads to improper fluid management in these patients [14]. Since the coastal belts of India are suitable for the growth of vectors, malaria is a major public health problem in the state of Odisha [1].

Ibinda F et al., reported malaria is the most common cause for hyponatremia in children with a prevalence of 44% [14]. Electrolyte disturbances are common in severe malaria [15,16]. Severe hyponatremia is associated with poor outcomes, prolonged hospital stay, neurological deficits, and death [14]. Though the pathophysiology of hyponatremia in severe malaria remains unclear, increased secretion of vasopressin (ADH) either appropriately or inappropriately along with repeated vomiting plays an important role [2,7,8,15]. In the present study, hyponatremia was found to be more severe and more fatal in severe mixed malarial infection than severe falciparum mono infection. As the patients with mixed malarial infection had more vomiting and dehydration, we assume severe vomiting as an important cause for hyponatremia among these patients.

Jasani HJ et al., have reported significant degree of hyponatremia in patients with severe malaria in comparison to P.vivax mono infection and isolated uncomplicated P.falciparum infection [17]. Hanson J et al., reported a prevalence of 57% hyponatremia among 181 in-patients with severe malaria [18]. Van Wolfswinkel ME et al., found hyponatremia in 77% of severe falciparum malaria which correlated with disease severity [19]. The high prevalence of hyponatremia among the severe malarial patients (63.6%) as reported from the current study is in accordance with previously reported studies from Uttarakhand (India), Bangladesh, Netherlands, and Kenya [4,14,18,19]. The mortality among the hyponatremia cases was higher than patients without having hyponatremia, which is in accordance with other studies [14]. Hyponatremia was associated with a statistically significant mortality. Hyponatremia at a cut-off of 126 mEq/L predicted mortality with a sensitivity of 81.8%, specificity 78.8%, and negative predictive value 97.5%. This association is never reported before and hence cannot be compared. The very high negative predictive value seems to be highly helpful in negative prediction of mortality in patients of severe falciparum malaria cases.

Hyponatremia in severe malaria can be taken a severity indicator. However, hyponatremia per se is present in many other diseases in hospitalised patients [20,21]. Hansan J et al., in a large cohort of 1000 adult severe malaria patients have found only base deficits and cerebral malaria but not sodium values as the predictors of outcome [22]. Similarly Phillips A et al., in their study including 482 persons with imported falciparum malaria have not found sodium as an independent risk factor for severe malaria [23]. It is prudent to say that the parameters which show impaired tissue perfusion like acidosis [24,25], hyperlactatemia [24,26] and increased base deficit [22] have been accepted as important risk factors for severe malaria and mortality than low serum sodium value. However, studies have not considered hyponatremia at cut-off level of <126 mEq/L as a mortality predictor as that of ours. The area under curve is 0.876 (95% CI: 0.8-0.95; p<0.001). Area under curve of more than 0.9 is excellent diagnostic test while more than 0.8 is considered good and more than 0.7 is considered as a fair diagnostic test [27].

Considering, the paucity of literature regarding prevalence of hyponatremia among severe malaria patients, this study is first of its kind to report prevalence and associated mortality from hyponatremia in severe malaria patients in the state of Odisha, India. Therefore, the results from this study could not be compared and correlated with other studies from Odisha.

Limitation

As the study was conducted in a tertiary care referral hospital, the prevalence and associated mortality rates could be underestimated owing to prior fluid management in peripheral hospitals. Further cross-sectional studies with larger sample size are recommended with similar study settings in order to generalise the results of the current study.

Conclusion

Hyponatremia is a common electrolyte disturbance in severe malaria. Hyponatremia is associated more commonly and more severely with mixed malarial infection rather than with isolated falciparum malaria. Hyponatremia at a cut-off of 126 mEq/L predicted mortality with a sensitivity of 81.8%, specificity 78.8% and negative predictive value 97.5%. Further similar studies are recommended.

References

[1]Pradhan A, Anasuya A, Pradhan MM, Kavitha AK, Kar P, Sahoo KC, Trends in malaria in Odisha, India—an analysis of the 2003-2013 time-series data from the National Vector Borne Disease Control Program PloS one 2016 11(2):e014912610.1371/journal.pone.014912626866696   [Google Scholar]  [CrossRef]  [PubMed]

[2]Karnad DR, Nor MB, Richards GA, Baker T, Amin P, Intensive care in severe malaria: Report from the task force on tropical diseases by the World Federation of Societies of Intensive and Critical Care Medicine J Crit Care 2018 43:356-60.10.1016/j.jcrc.2017.11.00729132978  [Google Scholar]  [CrossRef]  [PubMed]

[3]Yalçin M, Sevim E, Duran A, Treated with artemether-lumefantrine five evaluation of p. falciparum malaria cases in terms of hyponatremia and thrombocytopenia Türkiye Parazitolojii Dergisi 2015 39(2):15510.5152/tpd.2015.352026081891  [Google Scholar]  [CrossRef]  [PubMed]

[4]Jain A, Kaushik R, Kaushik RM, Malarial hepatopathy: clinical profile and association with other malarial complications Actatropica 2016 159:95-105.10.1016/j.actatropica.2016.03.03127019056  [Google Scholar]  [CrossRef]  [PubMed]

[5]Bodi JM, Nsibu CN, Aloni MN, Lukute GN, Kunuanuna TS, Tshibassu PM, Black water fever associated with acute renal failure among Congolese children in Kinshasa Saudi Journal of Kidney Diseases and Transplantation 2014 25(6):135210.4103/1319-2442.14432625394465  [Google Scholar]  [CrossRef]  [PubMed]

[6]Ladhani S, Patel VS, El Bashir H, Shingadia D, Changes in laboratory features of 192 children with imported falciparum malaria treated with quinine The Pediatric Infectious Disease Journal 2005 24(11):1017-20.10.1097/01.inf.0000183774.22593.7c16282945  [Google Scholar]  [CrossRef]  [PubMed]

[7]Hoorn EJ, van Wolfswinkel ME, Hesselink DA, de Rijke YB, Koelewijn R, van Hellemond JJ, Hyponatraemia in imported malaria: The pathophysiological role of vasopressin Malaria Journal 2012 11(1):2610.1186/1475-2875-11-2622280539  [Google Scholar]  [CrossRef]  [PubMed]

[8]Shah MP, Briggs-Hagen M, Chinkhumba J, Bauleni A, Chalira A, Moyo D, Adherence to national guidelines for the diagnosis and management of severe malaria: A nationwide, cross-sectional survey in Malawi, 2012 Malaria Journal 2016 15(1):36910.1186/s12936-016-1423-227430311   [Google Scholar]  [CrossRef]  [PubMed]

[9]Nagler EV, Vanmassenhove J, van der Veer SN, Nistor I, Van Biesen W, Webster AC, Diagnosis and treatment of hyponatremia: A systematic review of clinical practice guidelines and consensus statements BMC Medicine 2014 12(1):23110.1186/s12916-014-0231-125539784  [Google Scholar]  [CrossRef]  [PubMed]

[10]Hoorn EJ, Zietse R, Diagnosis and treatment of hyponatremia: Compilation of the guidelines Journal of the American Society of Nephrology 2017 28(5):1340-49.10.1681/ASN.201610113928174217   [Google Scholar]  [CrossRef]  [PubMed]

[11]Charan J, Biswas T, How to calculate sample size for different study designs in medical research? Indian Journal of Psychological Medicine 2013 35(2):12110.4103/0253-7176.11623224049221  [Google Scholar]  [CrossRef]  [PubMed]

[12]Naing L, Winn T, Rusli BN, Practical issues in calculating the sample size for prevalence studies Archives of Orofacial Sciences 2006 1:9-14.  [Google Scholar]

[13]Siwal N, Singh US, Dash M, Kar S, Rani S, Rawal C, Malaria diagnosis by PCR revealed differential distribution of mono and mixed species infections by Plasmodium falciparum and P. vivax in India PloS one 2018 13(3):e019304610.1371/journal.pone.019304629565981  [Google Scholar]  [CrossRef]  [PubMed]

[14]Ibinda F, Zarnack HC, Newton CR, Sodium disturbances in children admitted to a Kenyan hospital: Magnitude, outcome and associated factors PloS One 2016 11(9):e016132010.1371/journal.pone.016132027603309   [Google Scholar]  [CrossRef]  [PubMed]

[15]Asima RA, Akhtar S, Nawaz SK, Irfan S, Sadia AZ, Arshad M, Electrolyte disturbance and the type of malarial infection Iranian Journal of Public Health 2015 44(11):1492  [Google Scholar]

[16]World Health Organization. Management of severe malaria: a practical handbook. World Health Organization; 2000  [Google Scholar]

[17]Jasani JH, Sancheti SM, Gheewala BS, Bhuva KV, Doctor VS, Vacchani AB, Association of the electrolyte disturbances (Na+, K+) with the type and severity of the malarial parasitic infection J Clin Diagn Res 2012 6:678-81.  [Google Scholar]

[18]Hanson J, Hossain A, Charunwatthana P, Hassan MU, Davis TM, Lam SW, Hyponatremia in severe malaria: evidence for an appropriate anti-diuretic hormone response to hypovolemia The American Journal of Tropical Medicine and Hygiene 2009 80(1):141-45.10.4269/ajtmh.2009.80.14119141852  [Google Scholar]  [CrossRef]  [PubMed]

[19]vanWolfswinkel ME, Hesselink DA, Zietse R, Hoorn EJ, van Genderen PJ, Hyponatraemia in imported malaria is common and associated with disease severity Malaria Journal 2010 9(1):14010.1186/1475-2875-9-14020497587   [Google Scholar]  [CrossRef]  [PubMed]

[20]Beukhof CM, Hoorn EJ, Lindemans J, Zietse R, Novel risk factors for hospital-acquired hyponatraemia: a matched case–control study Clinical Endocrinology 2007 66(3):367-72.10.1111/j.1365-2265.2007.02741.x17302870  [Google Scholar]  [CrossRef]  [PubMed]

[21]Waikar SS, Mount DB, Curhan GC, Mortality after hospitalization with mild, moderate, and severe hyponatremia The American Journal of Medicine 2009 122(9):857-65.10.1016/j.amjmed.2009.01.02719699382   [Google Scholar]  [CrossRef]  [PubMed]

[22]Hanson J, Lee SJ, Mohanty S, Faiz MA, Anstey NM, Charunwatthana PK, A simple score to predict the outcome of severe malaria in adults Clinical infectious diseases 2010 50(5):679-85.10.1086/64992820105074  [Google Scholar]  [CrossRef]  [PubMed]

[23]Phillips A, Bassett P, Szeki S, Newman S, Pasvol G, Risk factors for severe disease in adults with falciparum malaria Clinical Infectious Diseases 2009 48(7):871-78.10.1086/59725819243243  [Google Scholar]  [CrossRef]  [PubMed]

[24]Brand NR, Opoka RO, Hamre KE, John CC, Differing causes of lactic acidosis and deep breathing in cerebral malaria and severe malarial anaemia may explain differences in acidosis-related mortality PloS One 2016 11(9):e016372810.1371/journal.pone.016372827684745  [Google Scholar]  [CrossRef]  [PubMed]

[25]Maitland K, Pamba A, Fegan G, Njuguna P, Nadel S, Newton CR, Perturbations in electrolyte levels in Kenyan children with severe malaria complicated by acidosis Clinical Infectious Diseases 2005 40(1):9-16.10.1086/42602215614686  [Google Scholar]  [CrossRef]  [PubMed]

[26]McCarthy AE, Morgan C, Prematunge C, Geduld J, Severe malaria in Canada, 2001-2013 Malaria Journal 2015 14(1):15110.1186/s12936-015-0638-y25890126  [Google Scholar]  [CrossRef]  [PubMed]

[27]Mandrekar JN, Receiver operating characteristic curve in diagnostic test assessment Journal of Thoracic Oncology 2010 5(9):1315-16.10.1097/JTO.0b013e3181ec173d20736804  [Google Scholar]  [CrossRef]  [PubMed]