Neonatology Section DOI : 10.7860/JCDR/2017/27660.10852
Year : 2017 | Month : Nov | Volume : 11 | Issue : 11 Page : SC10 - SC15

Implementation and Evaluation of NICU Registry in IRAN

Kayvan Mirnia1, Taha Samad-Soltani2, Peyman Rezaei-Hachesu3

1 Neonatologist, Paediatrics Health Research Center, Tabriz University of Medical Sciences, Tabriz, Iran.
2 Assistant professor, Department of Health Information Technology, School of Management and Medical Informatics, Tabriz University of Medical Sciences, Tabriz, Iran.
3 Associate Professor, Department of Health Information Technology, Tabriz University of Medical Sciences, Tabriz, Iran.


NAME, ADDRESS, E-MAIL ID OF THE CORRESPONDING AUTHOR: Dr. Peyman Rezaei-Hachesu, Associate Professor, Road Traffic Injury Research Centre, Tabriz University of Medical Sciences, Tabriz, East Azerbaijan, Tabriz, Iran.
E-mail: rezaeip@tbzmed.ac.ir
Abstract

Introduction

Specialized registry in the NICU (neonatal intensive care unit) is necessary to gather process and report data in an accurate and timely manner. An electronic registry could enable the mother and infant to be monitored and traced through complete and comprehensive data.

Aim

Registry has been developed on the basis of a literature study and view point of participants about effectiveness and efficiency was evaluated.

Materials and Methods

A situational analysis was performed to achieve a clear view from current status of NICU data collecting and reporting, then a customized three tier architecture was proposed to design and implement a specialized registry. User satisfaction was evaluated by system usability scale among 30 participants to assess user’s attitude about system.

Results

A comprehensive data model with seven categories was defined by 65 entities, which actually retains part of the infants information. System usability scale was in a high range about 70%. The highest agreement is on reduced registration error. The advantages of system were perceived by users and they had positive attitude about it.

Conclusion

The use of a suitable architecture and standard templates lead to enhanced acceptability, effectiveness, performance and functionality. Quality of data and health promotion has a direct relationship with system usability. These systems have a great potential to increase safety of patients by providing fast and easy access to information, monitoring and decision support operations.

How to cite this article

Kayvan Mirnia, Taha Samad-Soltani, Peyman Rezaei-Hachesu, .IMPLEMENTATION AND EVALUATION OF NICU REGISTRY IN IRAN.Journal of Clinical and Diagnostic Research [serial online]2017 Nov[cited:2019 May 23] 11 SC10 - SC15
 Available from
http://www.jcdr.net//back_issues.asp

Introduction

The maintenance and improvement of the health of newborns, categorized as vulnerable groups, has a special position in the healthcare system of most developing countries [1]. Neonatal Mortality Rate (NMR) is a valuable indicator of socio-economic development [2]. To reduce infant mortality, neonatal and perinatal factors in the first 28 days after birth should be considered. Each of these factors, depending on the circumstances of time and place has different effects on health and mortality [3]. Therefore, for a good patient care, these data should be collected, stored and analysed at all levels of patient’s management [4].

According to the importance of the documentation of medical records and its educational imperativeness, treatment, research and the corresponding statistical information, the correct, complete and timely registration of data can play a key role in satisfying health care system requirements [5]. Medical records of patients are the main source of care in the healthcare organizations [6]. In order to ensure data quality, documentation process should be modified because a good documentation requires an efficient information system. Studies have shown a significant improvement in the quality and safety of this domain which have been achieved by means of Health Information Technology (HIT) [7]. Computer information systems are more capable than paper-based systems by which more data can be stored and retrieved faster. The methods of storing are more reliable in these systems allowing us to have an efficient data management system [5].

In the past, patient’s information system was often paper-based, but now the trend is changing [8]. Although data entry in the paper chart is easy, it is difficult to retrieve, as they are often illegible [9]. Transferring experiences based on the paper records will face numerous problems, including illegibility, lack of concentration and time consumption or even loss of some events [10]. Data quality does not improve unless we replace the existing paper system by an appropriate electronic system [11]. Therefore, the development of information systems can enhance the quality of performance [12]. Despite the dramatic growth of clinical knowledge and information technology, many health care facilities are lacking computer systems to provide clinical data [4]. Regarding the main goals of electronic medical records (EMR) defined by the Institute of Medicine (IOM), such as supporting and improving the quality of patient care and clinical researches, it is necessary to apply electronic registry system and adopt this technology for neonates by increasing its utilization in the health care services [13].

Specialized registry system for managing clinical data have several advantages in comparison with paper records such as fast access to a patient information, legibility and accuracy of the information, retrieving the information easily, saving nurses’ time, avoiding repetitive procedures, possibility of quick reporting, reducing input errors and finally improving patient safety [14]. Complete and timely information is essential to inform public health decision makers for child health which is lacking in resource limited countries. In this situation, electronic registries are increasingly adopted to support maternal and child healthcare and surveillance. An electronic registry could enable the mother and infant to be monitored and traced through complete and comprehensive data. A number of problems including data management and patient confidentiality must be considered in designing and implementing such systems [15].

A study in 2006 by in New York City hospital showed that the use of a centralized registry system in the Neonatal Intensive Care Unit (NICU) improves the quality of care greatly, increase safety of infants and improves the efficiency of health services [16] and facilitate readability, availability and data quality [11]. Specialized registries store data on a particular type of disease. Maintaining a registry guarantees health authorities to have accurate and timely information, with ensuring the accessibility of data for care, research, and educational purposes [17]. To sum up, design and implementation of specialized registry system in the hospitals NICU is necessary to gather and present accurate and timely data. Data collected from analysis will be useful to the effectiveness of interventions in the NICU hospitals. In the present study, NICU registry system for hospitals with neonatal unit was designed, implemented and evaluated in Tabriz University of Medical Sciences in Iran.

Materials and Methods

Design and Development

This research was an applied study and conducted in the first six-month period of 2014 and registry software has been designed and executed on the central server of Tabriz University of Medical Sciences in winter 2014. All authorized users (members of pediatrics research center) have access to defined functions based on their access level. This system helps neonatal units of Alzahra, Koodakan and Taleghani hospitals to store and retrieve records through the university communication network.

At first, current status of recording data (hardware, software and process) was assessed by visiting NICU unit and weak points were discussed through consultation with neonatologists. In this assessment, the drawbacks of paper-based process were investigated and discussed comprehensively, i.e. unnecessary and redundant procedures in recording data were removed. The following steps were taken in the developing system.

System Properties and Tools used for Development

Three-tier architecture is the most widely used client-server architecture which is composed of three distinct components namely presentation layer or user interface which is viewed by an Internet browser, middle or business layer that includes a set of classes and operations and data access or database layer which consists of a multitude of data [18].

The databases operations and connections were defined in the data access layer. So, the components such as minimum data set and data model were defined for storing necessary data within the architecture.

Minimum demographic, diagnosis and treatment data were extracted by assessing requirements of system owners and stakeholders. Two 60-minute focus groups were conducted with four health information management experts and six neonatologists, separately. We also used the approved Minimum Data Set (MDS) of Canadian neonatal network to satisfy data interoperability requirments across multiple systems and softwares [19].

The final MDSs were accomplished after interviewing with some neonatologists and NICU nurses, then observing their task flows and also studying other registries. The database of electronic registry system was designed based on results of these focus groups and final MDSs. Data diagrams and related tables were designed in the SQL Server 2008, according to the user access control level and flexibility. All the processes will become operational codes by using C++ language. This processes formed system used cases. Selectable controls were used such as combo box or checkbox to prevent errors and improve data quality.

The appropriate and user-friendly user interface with multiple levels of authentication was designed. The interface was designed and implemented in web-based and shared forms between several hospitals using Asp.net and Hypertext Markup Language (HTML). User authentication and security protocols were developed. Also, the system user access levels were defined to protect data from unauthorized access. Some common standards were used to satisfy exchange and report requirements. International Statistical Classification of Diseases and Related Health Problems (ICD-10) codes were used for searching and selecting diagnosis. Health level-7 (HL7) standards were used to exchange data with generating XML (Extensible Markup Language) messages using NHapi toolbox. This toolbox was used to apply the HL7’s Version 2.x object model [20].

Evaluation of System

Study population consisted of all authorized staffs in three selected hospitals (Koodakan, Al-Zahra, and Taleghani hospitals) that have completed paper forms. Authorized staffs were nurses, medical record technicians and neonatologists. All staffs were included in study and there were no exclusion criteria. Samples consisted of 30 adults; 20 females, 10 males, median 34 years, out of which 13 nurses, 11 medical record staff and six were neonatologists. The respondents answered to System Usability Scale (SUS) questionnaire after using the system. SUS includes ten questions with a five point Likert scale from strongly disagree to strongly agree [21]. Finally, another checklist was presented to the users of registry to assess the impact of system on health promotion. This validated checklist consisted of 12 fields including cost reductions, reduced registration errors, increased information storage and retrieval speed, avoiding duplicate data recording, increased information recording, increased accuracy and privacy of information, enhanced information sharing, reduced physical space, facilitated reporting and ease of access [22]. Checklist was presented to participant users and was analysed using descriptive statistics.

Results

Implementation of NICU Registry was performed in the main data center of Tabriz University of medical sciences and was shared across three hospitals which are members of the university network. External systems did not have access to this network. All three layers were placed in a primary server and a direct domain name was defined as system URL. This system was approved by hospital managers for admission, registration and reporting of patients data. Data entry was performed in point of care and physicians, nurses and educated medical records staff was accessed to registry software. They were the main stakeholders and will enter data in next years. Nearly 3,000 infant records have been collected in accordance with the proposed minimum dataset and this set of records were available online to research centers of university with access levels and security considerations. This system also enables the creation and execution of customized query [23]. The designed registry covers retrospective and prospective methods. Present records were stored and the new ones were filled based on MDS that was very detailed. In first step, records of the past were entered retrospectively. Some fields were specified that “is not required”. According to FDA audit trial guidelines, where systems do not provide the facility to change data, also where the data is stored securely, in that case an audit trail should not be necessary [24]. Validation checks were provided in user interface and database layers. It was ensured that only authorized users had access to the system. Access controls between operations were separated and that only authorized persons got access to the registry. All changes were logged to database. In the various layers of this architecture several functions were performed which were discussed as follows:

The minimum data elements after reviews and the opinions of NICU physicians were defined in seven categories given in [Table/Fig-1]. A detailed list of data elements was attached as [Appendix 1].

Existing data elements at NICU registry.

Data Elements (Main Categories)
Demographics
Hospitalization info (Admission)
Risk factors(Foetal, Maternal, Placental, Others)
Diagnostic assessment and Actions(Reason, Severity, Action)
Disease(Short-term, Hypothyroid, Other)
Diagnosis(Diagnosis, Associated anomalies, Others)
Final data(Discharge)

A comprehensive and detailed documentation of system design, architecture and implementation challenges was provided in a previous study [23]. In this paper data model, used cases and other software analysis and design axis were discussed. The emphasis during software development had been on accessibility and user friendliness. Where possible, check boxes and radio buttons were used and free text fields were avoided. Usability evaluation showed that the usability score is above average and is in high range. The results of evaluation of user satisfaction using SUS are described in [Table/Fig-2]. The range of SUS is between 0 and 100. [Table/Fig-2] shows that the total mean of system usability is 70.16.

Result of system usability scale for NICU electronic registry.

VariableFrequencyMean of SUS result (0-100)
Gender
Male1068.5
Female2071
Education
Nurses1371
Medical record staff1169.27
Neonatologist670
Total mean of SUS value70.16

As shown in [Table/Fig-3]: User interface of NICU registry system which was consisted of seven tabs in multilingual labels as follow: Admission, Risk factors, Diagnostic assessment, Actions, Diseases, Tests, and Discharge data elements. The image shows the tests section and its sub-sections including blood test results, urine test, and susceptibility tests.

Shows the user interface of designed registry system.

[Table/Fig-4], the average value of agree and totally agree columns was 51.66% and the highest agreement was on reduced registration error, which was 73.33%. These values were calculated by averaging each column. Increased information communication between the users and employees at different levels, Reduced time and cost, Increased information storage speed, Increased information retrieval and Facilitated reporting to different health levels have an agreement less than 50%. Totally, the number of agreement results (Count of (agree OR totally agree greater than 50%)/ number of items = 7/12 = 58%) show that relatively, the advantages of system were percepted by users and they have positive attitude about specialized registry. At the login form, access control policies were seen. Only authenticated users could add new records and only admins can generate reports due to privacy concerns.

Evaluation of user’s attitudes around specialized NICU registry.

NoEvaluation fieldsTotally Disagree%(n)Disagree%(n)No comment% (n)Agree% (n)Totally agree% (n)
1Reduced registration errors3.33(1)6.6(2)16.6(5)13.3(4)60(18)
2Increased information recording speed20(6)3.33(1)16.6(5)33.3(10)26.6(8)
3Increased information recording accuracy13.3(4)20(6)0(0)43.3(13)23.3(7)
4Increased information security and privacy10(3)10(3)6.6(2)23.3(7)50(15)
5Increased information communication between the users and employees at different levels26.6(8)16.6(5)33.3(10)13.3(4)10(3)
6Avoiding duplicate data recording0(0)10(3)33.3(10)26.6(8)30(9)
7Reduced time and cost20(6)10(3)36.6(11)16.6(5)16.6(5)
8Reduced storage space0(0)30(9)6.6(2)20(6)43.3(13)
9Increased information storage speed10(3)43.3(13)20(6)16.6(5)10(3)
10Increased information retrieval16.6(5)0(0)36.6(11)16.6(5)30(9)
11Facilitated reporting to different health levels26.6(8)16.6(5)23.3(7)23.3(7)10(3)
12Easy access to the health care units’ documents10(3)13.3(4)13.3(4)30(9)33.3(10)

Discussion

The use of electronic system for recording and storing information in the form of web-based system leads to promotion of data quality, quality improvement and accurate evaluation which consequently can improve documentation and analysis of data in the NICU. The setting-up process in clinical information systems as a process of development has many advantages [27]. To achieve this, we have created an approachable and accessible registry. We will discuss some properties of registry.

Data management is one of the main goals of the department of health. Cook-Moine C and Cramer L described the advantages of a successfully implemented centralized Electronic Medical Record (EMR). They reported that the EMR provides instant access to updated patient information with no wastage of time [26]. Another study described that the performance of specialized registry system was better than standard and general systems [27]. The present study focussed on implementing a specialized electronic registry system of information in newborn patients. It provides a way to access the full scope of information and information management. Older processes, especially paper-based process could not generate cumulative reports and also storing and retrieving functions was very cumbersome and time consuming. Furthermore, the quality of paper records was very poor due to missed data and lack of precision. For these reasons, validation checks and required fields was used to improve quality of data entry in NICU system. Fakhrzad M et al., considered the capabilities of electronic systems for improving quality of life, easy access to information and clinical records and ultimately improving the health of populations [28]. In our study, data were shared among related hospitals and access to records was performed in real time. Nearly, all data entry process is coded using combo box, checkbox, radio buttons etc. Mamlin BW et al., mentioned that the use of unstructured data (narrative reports, diagnosis, etc.,) converts ambiguous paper fields into non-ambiguous concepts. They believed that the best chance of pleasing everyone was through the collection of highly coded data [29].

Advances in applying information technology to reach the goal of high-quality health care, are prevented by the lack of coordinated data standards. To sharpen quality improvement in neonatal care, healthcare experts must collaborate actively in the compilation of health care data standards. Data standards such as minimum data set leads to efficiency, timeliness, effectiveness, equity, patient-centeredness and some additional benefits [30]. When creating electronic platforms for medical settings it is important to understand and include data items which is deemed fit to be the most clinically useful [31]. Minimum Data Set (MDS) is a conceptual framework which consists of information related to healthcare effectiveness and is considered as a factor to achieve effectiveness indicators. Using standard data elements and determining the criteria for the management of patient care leads to a specific framework [32]. An essential data element which must be collected consists of demographic and clinical data sets about an infant admitted to NICU. In the present study, MDS was designed based on the needs and viewpoints of physicians since, there is no complete and standard minimum data set in the neonatal intensive care. Variables in different MDS are usually the same and only some variables will be given priority in clinical assessment according to needs and importance. Also, the neonatal variables must be updated annually according to the needs. The designed MDS in comparison with database existing in Vermont Oxford networks; Canadian Neonate Network and Schulman have no particular superiority [33], but must have been designed in accordance with experts views and the needs of the NICU hospital in Iran.

Data model was the core of NICU system. Providing scalability and flexibility of systems are the strong points of designing data model [34]. In our study, registered data on the database have semantic relationships with each other. Tables are accessed through a semantic model to maintain data integrity and prevent loss of data.

Researchers have demonstrated that usability challenges are among the significant problems to use and the acceptance of health information systems, and they can negatively affect user’s decisions, time management, and productivity. This can lead to user harassment and weakness and subsequently to abandon or denial of the softwares and systems [34]. Hence, evaluation of such systems was very essential in order to make human computer interaction more efficient. In current study, NICU registry system was evaluated by experts and users to achieve a quantitative outcome from system. The most complex and important aspect in usability testing is user interface evaluation [36]. We used SUS questionnaire to evaluate user interface and results showed that designed UI has appropriate quality in user computer interaction. In the next step, a validated check list was used to evaluate user perception and attitude about system.

Several studies demonstrated that implementing a specialized system can yield real advantages in terms of promotion of care delivery, enhanced monitoring activities, reduction of data entry errors, and decreased rates of utilization for redundant care [37]. Moreover, electronic information systems have a great potential to increase safety of patients by improving communications between healthcare providers, providing fast and easy access to information, monitoring and decision support operations [38]. All those mentioned above, about quality of data and health promotion, have a direct relationship with system usability. System usability is becoming accepted as a necessary component of design to ensure that new systems are used effectively [39]. In current study, we examined both discussed aspects to achieve a detailed viewpoint of the neonatal specialized registry system. Results emphasized that usability of system has a high range and users have a positive attitude about NICU registry system. An important reason for these results was the observance of the software architecture unified modelling language and interoperability standards in design and implementation phases [40,41].

Limitation

The main limitation of study was the lack of appropriate university network infrastructure that prevents pervasive coverage of province hospitals. Other challenges were the immaturity of Iranian National Electronic Health Record System (INEHRS). Another issue is the increased demands on data entry staffs which promoted incomplete data capture over the time. To handle this challenge, regional offices of health information technology in Iran planned an educational program to provide registry technicians at the point of care. These specialized trained personnel handled work load in clinical environment.

Conclusion

Designing specialized registry systems allows healthcare providers manage information effectively, access to information in high demand, improve costs and reduce errors, and increase communication between healthcare providers. The positive attitudes of the users towards these systems can lead to many clinical, research and educational benefits. The developed system currently supports research activities in a variety of areas including pattern of care, SEPAS (It’s in Persian and denotes national electronic health record system) linked database, and knowledge discovery projects. When needed, the raw data can be passed to national data warehouse. It needs the collaboration of Ministry of Health and Medical Education (MOHME) IT center to develop accessibility of web services. Designing a standardized information system in all levels of healthcare system will make an integrated infrastructure to facilitate data access. Furthermore, we can assume that by implementing the registry clinicians attitude towards IT capabilities, it has changed positively, that can contribute to an actual patient safety increase. We recommend applying business intelligence and dashboards to monitor trends in these specialized registries.

References

References[1]Adam T, Lim SS, Mehta S, Bhutta ZA, Fogstad H, Mathai M, Cost effectiveness analysis of strategies for maternal and neonatal health in developing countries BMJ 2005 331(7525):1107

[2]Black RE, Cousens S, Johnson HL, Lawn JE, Rudan I, Bassani DG, Global, regional, and national causes of child mortality in 2008: a systematic analysis The Lancet 2010 375(9730):1969-87.

[3]Welaga P, Moyer CA, Aborigo R, Adongo P, Williams J, Hodgson A, Why are babies dying in the first month after birth? a 7-year study of neonatal mortality in northern ghana PLoS ONE 2013 8(3):e58924

[4]Schulman J, Managing Your Patients’ Data in the Neonatal and Pediatric ICU: An Introduction to Databases and Statistical Analysis. First, editor 2008 USAJohn Wiley & Sons

[5]Dick RS, Steen EB, The Computer-based Patient Record: An Essential Technology for Health Care 1991 National Academy Press

[6]Moghaddasi H, Hosseini A, Sheikhtaheri A, A new model for the organizational structure of medical record departments in hospitals in Iran Perspectives in health information management/AHIMA, American Health Information Management Association 2006 3(4):1-24.

[7]Peyman RH, Ahmadi M, Aziz R, Zahra S, Farahnaz S, Nader M, Clinical care improvement with use of health information technology focusing on evidence based medicine Healthcare Informatics Research 2012 18(3):164-70.

[8]Eddy DM, Evidence-based medicine: a unified approach Health affairs 2005 24(1):9-17.

[9]Natarajan S, Hospital Supportive Service: Excel Books India 2010 [Available from: https://books.google.com/books?isbn=9350621150]

[10]Mirnia K ST, Rezaei M, Heidarzadeh M, Piri Z, Design and evaluation of electronic briefs of neonatal intensive care unit in taleghani hospital, tabriz, iran Global journal of Health Science 2014 6(5):125-31.

[11]Roukema J, Los RK, Bleeker SE, van Ginneken AM, van der Lei J, Moll HA, Paper versus computer: feasibility of an electronic medical record in general pediatrics Pediatrics 2006 117(1):15-21.

[12]Bloom BS, Crossing the quality chasm: A new health system for the 21st century JAMA: The Journal of the American Medical Association 2002 287(5):646-647.

[13]Dick RS, Steen EB, Detmer DE, The Computer-Based Patient Record: An Essential Technology for Health Care 1997 National Academies Press

[14]Beatty JD, Adachi M, Bonham C, Atwood M, Potts MS, Hafterson JL, Utilization of cancer registry data for monitoring quality of care The American Journal of Surgery 2011 201(5):645-49.

[15]Haskew J, G, Saito K, Turner K, Odhiambo G, Wamae A, Implementation of a cloud-based electronic medical record for maternal and child health in rural Kenya International Journal of Medical Informatics 2015 84(5):349-54.

[16]Naghavi M, Transition in Health Status in the Islamic Republic of Iran Iranian Journal of Epidemiology 2006 2(1):45-57.[Available from: http://irje.tums.ac.ir/article-1-185-en.pdf]

[17]Idowu PA, Williams KO, Alonge B, Sarumi OA, Online Spatial Cancer Registry System for Nigeria Journal of Biomedical Engineering and Medical Imaging 2015 2(5):10

[18]Patterns M, Team P, Microsoft Application Architecture Guide 2009 Microsoft Press

[19]Valls-i-Soler A, Halliday HL, Hummler H, International Perspectives Neonatal Networking: A European Perspective 2007 8(7):e275-e81.[Available from: http://neoreviews.aappublications.org/content/neoreviews/8/7/e275.full.pdf]

[20]Mabotuwana T, Hall CS, Ommering Rv, Tellis R, Sevenster M, An HL7 Data Pseudonymization Pipeline Healthcare Informatics (ICHI), 2015 International Conference on; 2015 21-23 Oct. 2015

[21]Sauro J, MeasuringU: Measuring Usability with the System Usability Scale (SUS) 2011[cited 12 Feb 2016] Available from: https://measuringu.com/sus/

[22]Taha K, Peyman R-H, Leila G, Designing and evaluating the web-based information system of primary health care in accordance with the electronic health records of iran Acta Medica Mediterranea 2016 32:2051-4.

[23]Rezaei-hachesu P, Samad-Soltani T, Mirnia K, Designing an Electronic Medical Record System of Infants in Hospitals of Tabriz University of Medical Sciences Journal of Health and Biomedical Informatics 2016 2(4):229-39.Available from: http://jhbmi.ir/article-1-126-en.pdf [cited 21 June 2017]

[24]FDA. Computer Systems Validation: Audit Trails; 2016[cited 7 March 2017]. Available from: http://www.csv-qa.com/articles/electronic-records/audit-trail-1

[25]Dargahi H, Ghazi Saeedi M, Safdari R, Hamedan M, A survey of clinical information system process in general hospitals of tehran university of medical sciences Payavard Salamat 2010 4(2):31-43.

[26]Cook-Moine C, Cramer L, Financial Savings From an Electronic Medical Record in a Small Group Practice 2005 Eden Park ROIEden Park ROI:1-8.

[27]Pickering BW, Dong Y, Ahmed A, Giri J, Kilickaya O, Gupta A, The implementation of clinician designed, human-centered electronic medical record viewer in the intensive care unit: A pilot step-wedge cluster randomized trial International Journal of Medical Informatics 2015 84(5):299-307.

[28]Fakhrzad M, Fakhrzad N, Dehghani M, The Role of Electronic Health Records in Presenting Health Information Interdiciplinary Journal of Virtual Learning in Medical Sciences 2012 2(4):31-40.

[29]Mamlin BW, Biondich PG, Wolfe BA, Fraser H, Jazayeri D, Allen C, Cooking up an open i emr for developing countries: open MRS – a recipe for successful collaboration AMIA Annual Symposium Proceedings 2006 2006:529-33.

[30]Spooner SA, Classen DC, Data standards and improvement of quality and safety in child health care Pediatrics 2009 123(Supplement 2):S74-S9.

[31]Herasevich V, Ellsworth MA, Hebl JR, Brown MJ, Pickering BW, Information needs for the OR and PACU electronic medical record Applied Clinical Informatics 2014 5(3):630-41.

[32]Castle NG, Ferguson JC, What Is Nursing Home Quality and How Is It Measured? The Gerontologist 2010 50(4):426-42.

[33]Horbar JD, The Vermont Oxford Network: evidence-based quality improvement for neonatology Pediatrics 1999 103(1 Suppl E):350-59.

[34]Marcinko DE, Hetico HR, The Business of Medical Practice: Transformational Health 2.0 Skills for Doctors 2011 Springer Publishing Company

[35]Khajouei R, Zahiri Esfahani M, Jahani Y, Comparison of heuristic and cognitive walk through usability evaluation methods for evaluating health information systems Journal of the American Medical Informatics Association: JAMIA 2017 24(e1):e55-e60.

[36]Armijo D, McDonnell C, Werner K, Electronic health record usability: Interface design considerations AHRQ Publication 2009(09):10-21.

[37]Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, Systematic review: impact of health information technology on quality, efficiency, and costs of medical care Annals of Internal Medicine 2006 144(10):742-52.

[38]Bates DW, Gawande AA, Improving safety with information technology New England Journal of Medicine 2003 348(25):2526-34.

[39]Karsh B, Beyond usability: designing effective technology implementation systems to promote patient safety Quality and Safety in Health Care 2004 13(5):388-94.

[40]Bass L, John BE, Linking usability to software architecture patterns through general scenarios Journal of Systems and Software 2003 66(3):187-97.

[41]Bass L, John BE, Kates J, Achieving usability through software architecture Software Engineering Institute 2001