Development and Validation of Health Education Tools and Evaluation Questionnaires for Improving Patient Care in Lifestyle Related Diseases

Lifestyle related diseases continue to be a significant burden on the health care system. Health education is a combination of educational strategies that promote voluntary adoption of healthy lifestyle choices and dietary behaviour. The use of simple and validated education and evaluation tools is now increasing in routine clinical practice to aid health status evaluation and communication between the patient, dietitian and the health care provider. Development of effective health education materials is a systematic process which starts with setting up the goals for education, followed by literature review and focus group discussion, content selection, designing the rough draft, seeking expert comments and validation. Questionnaire development should follow a logical and structured approach. Item generation should be based on extensive literature search and target group participation. Validation by the experts makes the questionnaire more meaningful, trustworthy and applicable. Considerable effort goes into designing and testing of these tools in order to ensure that they are effective. For enhancing clinical, dietetic and educational practice, it is pertinent to learn the process of developing these tools scientifically.

Lifestyle related diseases such as diabetes; hypertension, obesity, Non Alcoholic Fatty Liver Disease (NAFLD) and coronary artery disease are on the rise worldwide and have now become the most important cause of disease related morbidity. The increasing prevalence of these diseases is attributed mainly to lack of physical activity, sedentary lifestyle and faulty dietary pattern including the consumption of fast food and sugar sweetened beverages [1]. Compliance to medical health practitioner’s prescription and dietitian’s advice for long lasting adoption of healthy lifestyle plays a central role in the management of lifestyle related diseases [2]. However, the patients often fail to follow the advice to make the necessary changes in their habits, which impedes the achievement of treatment goals [3]. To effectively impart such education, medical health practitioners and dietitians must understand and address the existing gaps in knowledge, attitudes, and practices of patients. Well designed and validated questionnaires can help them in this task. The purpose of this article is to give a comprehensive description of the process of construction and validation of health education tools as well as evaluation questionnaires. This article would be useful to general practitioners, dietitians and auxiliary health workers in improving compliance to and follow up of lifestyle related advice.

Health education is a combination of educational strategies which promote voluntary adoption of healthy lifestyle choices and behaviour. Healthy lifestyle as defined by WHO is a lifestyle that involves eating lots of fruits and vegetables, reducing fat, sugar and salt intake and exercising [4]. Handbooks [4,5], guidelines [6], information leaflets/pamphlets [7,8] and booklets [9-11] are some of the traditional tools that provide health education to the masses. Websites [12], smart phone applications [13], blogs [14] and podcasts [15,16] are examples of newer methods which are popular in increasing health awareness among the youth.

These tools are useful in improving the overall knowledge of patients and their families about healthy lifestyle and their adherence to treatment. In clinical setting, these tools can be used as reinforcement by the physicians, dietitians and other auxillary health care workers during verbal communication with the patients [17]. Health education tools not only help in addressing the doubts and apprehensions of the patients [10] but also contribute immensely in teaching and research.

Education tool development is done in three steps [Table/Fig-1]. First step is to define the purpose and goal of developing the education material. This can be done by conducting surveys, interviews and focus group discussions with a small group from the target population to identify the relevant items acceptable to the population [18]. All such sessions should be tape recorded and transcribed verbatim. The data so collected will guide in generation of the topics to be included in the tool. Literature review is important to justify the selection of the topics.

[Table/Fig-1]:

A schematic diagram showing stages of development of health education tool.

In second step, a preliminary draft is prepared by incorporating all essential items identified. The content of the draft should be easy to read and simple to understand. Interesting and relevant illustrations should be developed with the help of specialists to enhance the presentation and effectiveness [19]. This draft can be printed before sending it for expert opinion and validation.

The third step includes evaluation by independent experts from the selected field. The health education material needs proper validation before it is implemented on patients. Experts selected should have previous experience in health promotion activities and validation of education tools. These experts should comment on the adequacy and appropriateness of information and its presentation style, from the reader´s perspective. While evaluating the tool, they should also assess the flow and ease of understanding as well as simplicity of language. The Readability Index (RI), related to the level of schooling necessary to understand a given text, should be calculated using the Flesch reading ease formula [20]. If the score comes below 50, the text should be rewritten by reducing sentence size and replacing words. After this process the text should be proof read by a specialist. Conceptual mistakes (if any) should also be pointed out. Regarding illustrations, it is important to evaluate the appropriateness of the picture, graphs and charts and their placement with reference to the text. The recommendations given by experts at the end of validation process should be integrally accepted and incorporated in the education tool. The new version of the education material after corrections should be subjected to another edition, revision and layout process [17].

Besides, few subjects (usually 10-20) from the target group should be invited for pilot testing the tool and analysing it. They should be asked to review the vocabulary indicating the difficult terms, as well as the appropriateness of illustrations [21].

It is important to frequently revise the health education material according to the latest scientific innovations and new knowledge in that field [22].

Evaluation is an important step in health care. It should be an integral part of the assessment process as it leads to quality improvement of clinical services. Questionnaire is a commonly used health evaluation instrument that helps in the measurement of a patient’s progress or lack of progress toward achievement of specified goals.

The use of questionnaires as a method of data collection in health-care research both at national and international level has increased in the past few years [23]. A questionnaire consists of a series of questions that help in gathering information from respondents. It has advantages over other methods of data collection as it is relatively quick to complete, economical and usually easy to analyse [24]. Self administered questionnaires can cover large sample size in a wide geographical area, access population which is sometimes difficult to reach and provide a good way to deal with sensitive topics, which are uncomfortable for many people to discuss face to face. A major drawback with questionnaires is that they are developed on the assumption that the researcher and respondents share similar understanding about language and interpret similar meaning from similar statements. Also, the response rate in a questionnaire may be low and they may only provide a snapshot of the situation rather than in-depth picture of the area of concern. Besides, questionnaires generate socially desirable responses at times [25].

Well-designed questionnaires can measure knowledge, attitudes, emotion, cognition, intention and behaviour. They capture the self-reported observations of the individual and are commonly used to measure patient perceptions of many aspects of health care. In a health care setup, questionnaires can be used in many ways such as describing a patient’s experiences with routine or new procedures, examining progress with the implementation of medical and dietary intervention and identifying areas for improvement by conducting periodic evaluations and identifying barriers to compliance to medical and dietary advice and prescriptions [26].

Questionnaires are commonly used in medical and nutrition education. Still there is not much clarity on how to design and construct these questionnaires for use in medical field [27]. As a result, many of these poorly designed questionnaires fail to achieve the purpose that they are designed for. Development of a questionnaire is a systematic process that involves following steps:

Step 1: Review of literature: One should start making the questionnaire only after reviewing all the existing literature on the selected topic. This will help to identify any already made questionnaire in that area, which can be modified or adapted for the desired purpose. This will also help in determining the construct of the questionnaire [27].

Step 2: Conducting interviews and focus group discussions: Knowing how well the target population understands and conceptualizes the disease and its treatment is instrumental in the development of a questionnaire. Therefore, it is important to understand the perspectives of the target group about the disease and its treatment. Interviews and focus group discussions are conducted for the purpose where active participation and interaction of the subjects is encouraged. The procedure should be repeated until no new ideas are available from the groups. All discussions should be recorded. The results of literature review and focus groups should be merged [27].

Step 3: Item generation: This step comprises of creating a list of items that adequately represent the construct of the questionnaire in a simple and lucid language. A lot of pilot work goes into creation of a list of items during a questionnaire development. Sequencing of words and content development should be given maximum attention. Questions should be formulated keeping in mind the characteristics of a good questionnaire [Table/Fig-2]. Consideration should be given to the order in which items are presented. Begin the questionnaire with simple, non threatening questions to help capture the interest of the respondent [28]. This should be followed by transition questions that are used to make different areas flow well together. Skips include questions similar to “If yes, then answer question (A) If no, then continue to question (B).” Too many jumps will confuse the respondent and may discourage them from continuing with the questionnaire. Difficult questions should be put towards the end. Personal questions should be at the closing of the questionnaire as they may make the respondents uncomfortable and may deter them from finishing the questionnaire [29]. Deciding the number of items is also important. The ideal number of items is determined by several factors, depending on the complexity of the questionnaire design. The most crucial element in item generation is to revisit the research questions again and again, to ensure that items reflect these and remain relevant [30].

[Table/Fig-2]:

Characteristics of a good questionnaire.

Scales and ranges in a questionnaire: While developing a new item/question, it is important to establish which scale and response format is to be used. Different scales and response styles produce different types of data which influence the analysis options [23].

Frequency scales may be used when it is important to establish how often a target behaviour or event has occurred. The example of a food frequency questionnaire explains the frequency scale where the subject needs to answer his behaviour regarding food consumption in terms of frequencies such as daily, weekly, fortnightly, monthly, rarely etc. Knowledge questionnaires may be helpful when evaluating the outcome of a patient education programme. For example, the change in knowledge of subjects about ways to control blood pressure before and after a lifestyle counselling session can be analysed by administering a knowledge questionnaire.

Some questionnaires measure separate variables such as questions on preferences (food preferences), behaviours (exercise behaviours), and facts (knowledge about symptoms of a disease). Other questionnaires have questions that are aggregated into either a scale or index, include questions that measure traits, attitudes (attitude towards alcohol), etc.,

Four types of response scales [31] for closed-ended questions exist:

In open-ended questions, the respondent’s answer is coded into a response scale afterwards. In clinical practice and research, Likert-type/frequency scales are most commonly used scales to measure attitudes and opinions. They offer fixed choice responses to the respondents. Agreement/disagreement can be measured using these ordinal scales. Generally the respondents have to choose between five, seven or nine already coded responses. These odd numbered scales provide a center point that is neutral, which means neither agree nor disagree [32].

Step 4: Demonstrating validity and reliability of a questionnaire: Validity refers to whether a questionnaire is measuring what it is supposed to measure [33]. Validity is of two types. Internal validity measures the extent to which questions within an instrument agree with each other. External validity measures the ability of the questionnaire to extrapolate the findings of the sample tested, to a large population. Internal validity confirms that a subject will respond to similar questions in a similar way and also affects the likelihood of producing false positives and false negatives.

Validation of a questionnaire is an important step before it can be used in population. It not only ensures its reliability but also improves it further in terms of usability and credibility. Usually, a validated questionnaire is better in terms of simplicity and precision. Besides, it is adequate for the problem intended to measure and capable of measuring the change [34].

In a questionnaire, certain aspects of validity should be ensured before its application. First is content validity (or face validity) i.e., whether the items or questions cover the full range of issues and topics relevant to the subject area. It ensures balanced coverage of different topics within the questionnaire. Second is criterion validity i.e., the extent to which a measure is related to an outcome. Another important one is construct validity i.e., extent to which a questionnaire can correctly measure the cause and effect link between a measure and some other factor [34] [Table/Fig-3].

[Table/Fig-3]:

A schematic diagram to show types of validity.

To make sure that the content is valid, many different sources are utilized for item generation. First of all extensive literature review is done. This is followed by consultation with experts in that subject area. Also, a few respondents are involved in the whole process. Content Validity Ratio (CVR) is a method that is specifically designed to check the content validity of health based questionnaires [35,36]. This method helps to identify which items should remain in the questionnaire and which should be discarded. Content validation forms should be developed and given to the experts for this phase [37].

Criterion validity is often divided into concurrent and predictive validity. Concurrent validity refers to the ability of the questionnaire to measure current performance. This can be established by correlating the performance of the questionnaire with concurrent behaviour. Predictive validity refers to the ability of the questionnaire to measure future performance. To establish predictive validity of the tool, it is needed to correlate performance of the tool, with behaviour in future [38].

Construct validity, on the other hand, refers to expert opinion concerning whether the scale items represent the proposed domains and concepts the questionnaire is designed to measure. To establish construct validity, it is required to correlate the performance of the tool with the performance of a pre-existing established tool.

Every questionnaire may not have all kinds of validity. Therefore, no questionnaire can ever be fully “validated.” It can only be validated for a certain patient population, under certain conditions. Therefore, it is important to validate each questionnaire according to the target population and purpose of data collection [39].

Reliability is the degree to which a questionnaire produces similar results each time it is administered. It is essential that the reliability of a developing questionnaire can be demonstrated. Cronbach’s α statistic is the most common method used to demonstrate reliability. Explaining Cronbach’s α statistic is beyond the scope of this article. Authors advice readers to consult statistician for in-depth knowledge.

Broadly, the questionnaires can be administered in the following ways:

To conclude, effective health education tools and evaluation questionnaires can aid the medical and dietetic practitioners in developing a patient centered plan to implement and maintain management plan. These tools help in improving patient outcomes in terms of compliance to medical and dietetic advice. Development of individualized and tailor made health education tools and evaluation questionnaires can be a boon not only to improve patient care but also to promote voluntary adoption of healthy lifestyle choices and dietary behaviour.

[1]. WHODiet, nutrition and the prevention of chronic diseases World Health Organ Tech Rep Ser 2003 916:i-viii.  [Google Scholar]

[2]. Memon KN, Shaikh NZ, Soomro RA, Shaikh SR, Khwaja AM, Non-compliance to doctors’ advices among patients suffering from various diseases: patients’ perspectives: a neglected issue Journal of Medicine 2017 18(1):10  [Google Scholar]

[3]. Stonerock GL, Blumenthal JA. Role of counseling to promote adherence in healthy lifestyle medicine: Strategies to improve exercise adherence and enhance physical activity. To be published in Progress in Cardiovascular Disease. [Preprint] 2016. Available from: https://www.ncbi.nlm.nih.gov/pubmed/27640186 [Accessed on 16^th March 2017]  [Google Scholar]

[4]. WHO. A healthy lifestyle. Available from: http://www.euro.who.int/en/healthtopics/disease-prevention/nutrition/a-healthy-lifestyle [Accessed on 8th March 2017]  [Google Scholar]

[5]. Muhlhausler BS, Leghi GE, Watson RR, De Meester F, Polyunsaturated fatty acids, cholesterol, triglycerides and cardiometabolic disease Handbook of Cholesterol 2016 NetherlandsWageningen Academic Publishers:133-138.  [Google Scholar]

[6]. Wildman RE, Nutraceuticals and functional foods 2016 2nd edUSACRC press  [Google Scholar]

[7]. Mani I, Kurpad AV, Fats and fatty acids in Indian diets: Time for serious introspection Indian Journal of Medical Research 2016 144(4):507  [Google Scholar]

[8]. Ajmal M, Community health education through non-formal education in Pakistan Jurnal Pendidikan 2016 16(2):150-57.  [Google Scholar]

[9]. Khurana S, Rao BK, Lewis LE, Bhat R, Purkayastha J, Kamath A, Development and validation of educational leaflet for caregivers of preterm infants Journal of Clinical and Diagnostic Research 2016 10(7):YC01-YC04.  [Google Scholar]

[10]. Moura DD, Moura ND, Menezes LC, Barros AA, Guedes MV, Development of a booklet on insulin therapy for children with diabetes mellitus type 1 Revista Brasileira de Enfermagem 2017 70(1):07-14.  [Google Scholar]

[11]. da Silva Jacob LM, Mont’Alverne DG, Caetano JÁ, Diógenes MA, Shimo AK, Pereira AM, Creation and validation of an educational booklet on the hypertensive pregnancy syndrome International Archives of Medicine 2016 9(42):01-08.  [Google Scholar]

[12]. Butcher KR, Jameson JM, Computer-assisted and web-based innovations in psychology, special education, and health 2016 Academic Press  [Google Scholar]

[13]. Luxton DD, McCann RA, Bush NE, Mishkind MC, Reger GM, mHealth for mental health: Integrating smartphone technology in behavioral healthcare Professional Psychology: Research and Practice 2011 42(6):505  [Google Scholar]

[14]. de Araújo AF, Guimarães JM, de Souza EC, Rebouças TO, Ferreira HS, Pequeno AM, The blog as educational tool for health education: possibility of joint-service education International Archives of Medicine 2016 9(1):01-08.  [Google Scholar]

[15]. Boulos MN, Maramba I, Wheeler S, Wikis, blogs and podcasts: a new generation of Web-based tools for virtual collaborative clinical practice and education BMC Medical Education 2006 6(1):41  [Google Scholar]

[16]. Johnson L, Grayden S, Podcasts--an emerging form of digital publishing International Journal of Computerized Dentistry 2006 9(3):205-18.  [Google Scholar]

[17]. Reberte LM, Hoga LA, Gomes AL, Process of construction of an educational booklet for health promotion of pregnant women Revista Latino-Americana de enfermagem 2012 20(1):101-08.  [Google Scholar]

[18]. Benevides JL, Coutinho JF, Pascoal LC, Joventino ES, Martins MC, Gubert FD, Development and validation of educational technology for venous ulcer care Revista da Escola de Enfermagem da USP 2016 50(2):309-16.  [Google Scholar]

[19]. Houts PS, Doak CC, Doak LG, Loscalzo MJ, The role of pictures in improving health communication: a review of research on attention, comprehension, recall, and adherence Patient Educ Couns 2006 61(2):173-90.  [Google Scholar]

[20]. Flesch R. Let’s Start with the Formula. In: How to write plain English. Available from: http://www.mang.canterbury.ac.nz/writing_guide/writing/flesch.shtml (Accessed on 16^th April, 2017)  [Google Scholar]

[21]. Rubio DM, Berg-Weger M, Tebb SS, Lee ES, Rauch S, Objectifying content validity: Conducting a content validity study in social work research Social Work Research 2003 27(2):94-104.  [Google Scholar]

[22]. Teles LM, Oliveira AS, Campos FC, Lima TM, Costa CC, Gomes LF, Development and validating an educational booklet for childbirth companions Revista da Escola de Enfermagem da USP 2014 48(6):977-84.  [Google Scholar]

[23]. Rattray J, Jones MC, Essential elements of questionnaire design and development Journal of Clinical Nursing 2007 16(2):234-43.  [Google Scholar]

[24]. Bowling A, Research methods in health: investigating health and health services 2014 UKMcGraw-Hill Education  [Google Scholar]

[25]. Patten ML. Questionnaire Research: A Practical Guide. London: Routledge Publishers;2016. Available from https://books.google.co.in/books?hl=en&lr=&id=SCkxDQAAQBAJ&oi=fnd&pg=PT6&dq=Patten+ML.+Questionnaire+Research:+A+Practical+Guide.+London:+Routledge+Publishers%3B+2016.&ots=LTvJIkxxkX&sig=B5JBC2VXvX1raTC-LvgCtdLqA8U (Accesssed on 23rd April 2016)  [Google Scholar]

[26]. Jack B, Clarke AM, The purpose and use of questionnaires in research Professional Nurse 1998 14(3):176  [Google Scholar]

[27]. Gehlbach H, Artino AR, JrDurning SJ, AM last page: survey development guidance for medical education researchers Academic Medicine 2010 85(5):925  [Google Scholar]

[28]. Diem KG, A Step-by-step guide to developing effective questionnaires and survey procedures for program evaluation and research 2002 New JerseyDesktop publishing by Rutgers-Cook College Rei Center  [Google Scholar]

[29]. Burns AC, Bush RF, Marketing Research 2010 7th edUpper Saddle River, NJPearson Education  [Google Scholar]

[30]. Oppenheim AN, Questionnaire design, interviewing and attitude measurement 1992 LondonPinter Publishers  [Google Scholar]

[31]. Mellenbergh GJ, Adèr HJ, Mellenbergh GJ, Hand DJ, Tests and questionnaires: construction and administration Advising on Research Methods: A consultant’s companion 2008 Huizen, The NetherlandsJohannes van Kessel Publishing:188-189.  [Google Scholar]

[32]. Burns N, Grove SK, The practice of nursing research conduct, critique, & utilization 1997 PhiladelphiaWB Saunders and Co  [Google Scholar]

[33]. Bryman A, Cramer D, Quantitative data analysis with SPSS for windows 1997 LondonRoutledge  [Google Scholar]

[34]. Kestrel Consultants Inc. Validating Questionnaires. Available from: http://kestrelconsultants.com/reference_files/Validating_Questionnaires.pdf (Accessed on 17th April 2017)  [Google Scholar]

[35]. Veneziano L, Hooper J, A method for quantifying content validity of health related questionnaires American Journal of Health Behavior 1997 21(1):67-72.  [Google Scholar]

[36]. Lawshe CH, A quantitative approach to content validity Personnel Psy 1975 28:563-75.  [Google Scholar]

[37]. La Artino AR, JrRochelle JS, Dezee KJ, Gehlbach H, Developing questionnaires for educational research: AMEE Guide No 87. Medical Teacher 2014 36(6):463-74.  [Google Scholar]

[38]. Trochim W. The research methods knowledge base. 2nd ed. Cincinnati, Atomic Dog Publishing, 2006. Available from https://www.socialresearchmethods.net/kb/measval.php (Accessed on 17^th April 2017)  [Google Scholar]

[39]. Damato S, Bonatti C, Frigo V, Pappagallo S, Raccanelli R, Rampoldi C, Validation of the Clinical COPD questionnaire in Italian language Health and Quality of Life Outcomes 2005 3(1):9  [Google Scholar]