The Translation Process, Validity and Reliability Study in Occupational Health Psychology amongst Healthcare Professionals by Multitrait-Multimethod Matrix: A Multimethod Study

1. Professor, Department of Community Medicine, Pramukhswami Medical College, Bhaikaka University, Anand, Gujarat, India. 2. Professor, Department of Psychiatry, Pramukhswami Medical College, Bhaikaka University, Anand, Gujarat, India.

Abstract

Introduction: Questionnaires are designed to measure specific constructs. Occupational Health Psychology (OHP) includes various domains/areas like work place well-being, work related stress, flourishing, resilient coping, coping mechanisms, compassion satisfaction, burnout and secondary traumatic stress. At times, specific questionnaire may not be available and if available, may not be in the language intended to be used by the respondents. A new questionnaire needs to be developed or translation of an available questionnaire into the respondents’ language is required in such case.

Aim: To detail out the translation process of the study tools (questionnaires/scales) and to assess the validity and reliability of these study tools used in OHP.

Materials and Methods: The present cross-sectional, Quasi-experimental study was conducted in the Department of Community Medicine, Pramukhswami Medical College, Anand District, Gujarat, India, from February 2021 to January 2023 using multimethod study design amongst the healthcare professionals. For translation, standard World Health Organisation (WHO) translation guide was followed. Nine scales/questionnaires along with a socio-demographic and a qualitative proforma were finalised to achieve the objectives. The questionnaires availability in public domain was taken into consideration while selecting the questionnaires. Reliability analysis and validity assessment were done. Reliability measure was checked for internal consistency by calculating Cronbach’s alpha. Construct validity was assessed by Multitrait-Multimethod Matrix. Translation was done in Hindi and Gujarati languages.

Results: Flourishing scale (FS), Brief resilient coping scale, compassion satisfaction and work satisfaction were positively correlated and all these were negatively correlated with all the stress measures, Depression, anxiety measures of Depression, Anxiety and Stress Scale (DASS), burn out and secondary traumatic stress. Cronbach’s alpha for internal consistency measure was calculated for different sub domains of the nine questionnaires. Construct validity was analysed by calculating correlation coefficients between different identified constructs/ sub constructs. Variables measuring similar construct were found to be positively correlated and significant.

Conclusion: Variables/questionnaires measuring a similar construct were found to be positively correlated and significant, while opposite constructs were negatively correlated. The questionnaires need to be translated in the language in which the participants can understand. The better linguistic comprehension will increase the internal validity of the study.

Convergent validity, Divergent validity, Transation guide

Occupational Health Psychology (OHP) is the application of the principles and practices of psychology to occupational health issues (1). OHP centres directly around the health of employees, further to their families with the overall purpose to develop, maintain and promote health (2). There are aspects of work which are difficult to describe. This includes identifying their effects on psychological and physical health. The recognition of such aspects of work started emerging in nineteenth century after the industrial revolution (3). European perspective of OHP centered around contribution of applied psychology to occupational health and was given by Tom Cox CBE and associates. North American representative body viewed OHP as interdisciplinary partnerships of psychological and occupational health science professionals to improve the quality of working life, safety, health and well-being of workers. Interventions to promote health, organisational research methods, design of the psycho-social work environment, stress theory, and stress interventions were among the five core topics in OHP curriculum (2). Various interactions and their complexities at work places, organisational values and practices, policy decisions, managerial aspects, work satisfaction, burnout, coping and resilience all circumscribe the employees’ health and well-being and impedes on a multiple interactions. There may not be a direct causation, but it is more of a two-way interaction between multiple variables or constructs. Questionnaires to measure different constructs like burnout, work place well-being, flourishing, stress, resilient coping, compassion satisfaction and other constructs are available. Based on the objectives of the study, the specific questionnaires can be decided. Multiple questionnaires measuring a similar construct may be used to prove convergent validity. Questionnaires are the important study tool to collect information from the participants. Data collection can range from collection of quantitative information through questionnaires and experiments to qualitative methods of observations, discussion and interviews. A specific questionnaire that measures the construct of interest may not be available and it becomes more important for some specific context (4). If such a questionnaire is available, it may not be in the desired language and with differing socio-demographic contexts. A new questionnaire needs to be developed or translation of an available questionnaire into the respondents’ language is required in such case, so it requires translation. The questionnaire can be used in the original form, if language and other socio-demographic contexts are not an issue. But still, it should undergo the process of validation and reliability analysis. So, this study was done with the objective to detail out the translation process with the assessment of validity and reliability of the study tools (questionnaires) used in the current study in OHP.

Material and Methods

The current cross-sectional, Quasi-experimental study was conducted in the Department of Community Medicine, Pramukhswami Medical College, Anand District, Gujarat, India, from February 2021 to January 2023 using multimethod study design amongst the healthcare professionals.

Two study designs were used in this multimethod study design (multiple approach design) (5): These were:

1. Cross-sectional study design: base pool of participants
2. Quasi-experimental Solomon four non equivalent control group study design for intervention [6,7]

Sample size calculation: Sample size for cross-sectional study was calculated using OpenEpi Version 2.3.1 and by the formula={DEFF*Np(1-p)}/{(d2/Z21-a/2*(N-1)+p*(1-p)} (8).

In the formula, N is the population size. Hypothesised % frequency of outcome factor in the population (p)- 50% to keep the maximum sample size for the given set of particulars in sample size calculation. Absolute precision %-7%, confidence limits as % of 100-/+7%,

Z21-a/2=Standard normal variate (at 5% type I error p=0.05, it is 1.96)

DEFF=Design effect-1, Confidence level- 95%

Based on the “p” and at 95% confidence limit, the calculated sample size was 196. Considering 10% non response rate, the final sample size came to 216. The data collection was done for 231 participants and during data entry three proformas with incomplete information for the nine scales were rejected. So, the final base pool of participants was 228.

Sample size for Solomon four-group design was calculated by using G Power 3.1 for F tests: ANOVA: Repeated measures, within-between interaction

Effect size f=0.25, α err (prob)=0.05, Power=0.90, Number of groups=4

Keeping the intervention group participants in the overall sample size, the sample increased by 32 participants over the base sample from group III and group IV of Solomon four-group design.

Cross-sectional study of 228 participants in Phase I was done followed by Solomon four-group design with a total 64 participants. A total of 32 participants were from original frame as group I and group II participants. So, 260 was the overall sample size with 32 participants additional to 228.

The current validity and reliability study was based on the analysis of 98 participants who were from the base pool of participants. This included those participants who filled the Gujarati translated questionnaires and those who were in the pilot testing of the Gujarati translation process. This study was approved by the Institutional Ethics committee (IEC) and the clearance number was IEC/HMPCMCE/122/Faculty/4/. All the participants have filled the informed consent form.

Inclusion criteria: Those who agreed to participate in the study and had been working for atleast minimum of one year at the same place were included in the study. Different healthcare professionals viz., allopathic doctors, physiotherapists, nursing professionals and Community Health Officers (CHOs) were included as the study participants.

Exclusion criteria: Participants who did not agree to participate in the study were excluded.

Questionnaire

The questionnaire was decided to be self-administered, as it was intended to measure many issues which included individual thinking, perceptions and personal habits and some past experiences. Self-administered questionnaire also increased the likelihood of responding truthfully.

Sampling was done as convenient sampling in two stages:

Stage-1: Primary units (Institutions and Government Public health facilities) (Table/Fig 1).
Stage-2: Participants from the institutions: Those who agreed to participate and were in the inclusion criteria.

Selection of questionnaires: To meet the desired objectives of the overall project, nine scales/questionnaires along with a socio-demographic proforma including occupational attributes and a qualitative proforma were finalised. All the nine questionnaires/scales were translated in both Hindi and Gujarati language (Table/Fig 2) (9),(10),(11),(12),(13),(14),(15),(16),(17).

In all the scales, permission to use and in some permission to translate was available on web page. In spite of the availability and written statement for public use and translation, authors got the permissions through mail for both use and translation in all. So, all the authors of the scales were communicated regarding the study and permission was taken from them regarding the use and the translation in two languages. Positive replies were received for use and translation. After translation, both the translated versions were sent to the respective authors. Different constructs were identified based on the objectives of the study (Table/Fig 3) (6),(7),(18),(19),(20),(21).

All the questionnaires used were in Likert scale whereas brief cope and OHS vulnerability measure questionnaire: Workplace health and safety survey were Likert type scale.

Statistical Analysis

In this study, Multitrait- Multimethod Matrix was prepared based on the correlation coefficient. Cronbach’s alpha was calculated as a measure for internal consistency.

Results

Translation process: This was done following the standard World Health Organisation (WHO) guidelines for translation (22),(23). The described translation process is for both Hindi and Gujarati translations. All the nine questionnaires/scales were translated in both Hindi and Gujarati language.

Forward translation: For Gujarati translation, the translation was first done by the author and checked through four other independent translators in a pair of two. To one group, the objectives of the current study were made clear and the other group did the translation without the study background [24,25]. For Hindi translation, the translation was done primarily by the first author and then checked through one other translator for content similarity and correctness. The translated versions were kept simple, clear and concise.

Backward translation: The translated versions were back translated into the original language i.e., English. The translated versions and the original questionnaire were seen for any dubious and altered meaning. Due consideration was given to each word, modals and the tense in which the questions were framed. Accuracy was checked. The translated versions and the original study tools (questionnaires) were reviewed for conceptual equivalence. This was done by the same translators (for both the languages) as the translators are equipped with the sound knowledge of both the languages. It took two revisions before the final print was ready to be given to the participants for a pilot study. The final version of both the translations were read and approved by both the authors.

After completion of the translation process, the respondents were given the translated versions and asked for any difficulty in understanding the questions. So, the testing and the revisions were done in small groups when the questionnaires were given and this was done two times and no more further queries pertaining to questions came later. The respondents were asked the meaning of different statements and it was matched with the original study tools (questionnaires). This was done in person. The respondents query pertaining to any confusion for the questions was resolved with incorporation of all the necessary and important changes and suggestions thereafter in the respective questionnaires. Further new prints were taken with the incorporated changes. So, with all the suggestions in these two sittings, the study tools (questionnaires) were finalised. Faculties from medical colleges and physiotherapists were given the original set of English questionnaires. Issues related to certain difficult to understand phrases/sentences came into light. So, meaning of these phrases/sentences were also provided at the same site in the English questionnaire in italics putting in the asterix form. Examples of such sentences are “I found it difficult to wind down” and “I feel bogged down by the system”. All such difficulties in comprehension were identified and the meanings were added.

Validity and reliability analysis: Validity and reliability analysis in the present paper is of Gujarati translated questionnaires. Construct validity was measured indirectly by studying multiple measures like face validity, content validity and criterion validity. Face validity and content validity of the original English set of study tools (questionnaires) was assessed primarily by the authors (1st author is also the lead translator). After the translation process was over, it was followed by a discussion with all the translators. The print version of the translated copies was seen for face validity and content validity by the authors and the translators. The study tool comprised of consolidated questionnaire (one set consisted of socio-demographic profile, a qualitative questionnaire and nine scales/questionnaires). The linguistic validity was checked in the initial two sittings, following the translation process. The questionnaire contents, as well as, overall appeal of the questionnaire was discussed with the participants. Divergent validity and convergent validity (for concurrent validity) were assessed for different sub constructs/domains and was statistically checked by preparing Multitrait Multimethod Matrix (26). Convergent validity was seen as positive correlated scores of sub constructs/domains measured through different questionnaires. Discriminant validity of constructs/sub-constructs measured by different questionnaires is proved as non significant correlations.

Reliability analysis (for the collected data) was run which showed a very good reliability for different sub constructs of different scales (Table/Fig 4).

All the different scales measuring stress has a positive significant correlation (Table/Fig 5).

Intrusion of work in private life as a component of work place well-being was negatively correlated with work satisfaction, organisational respect for employees and Employer care. Else all were correlated positively and significant (Table/Fig 6). (Table/Fig 7) shows the assessment of construct validity of Occupational Health and Safety Survey.

Different questionnaires measuring the same construct/dimension are identified and put in same colour as red and blue. All blues and reds are positively correlated with the same type and red and blue were negatively correlated (Table/Fig 8).

Compassion satisfaction, burnout and secondary traumatic stress were three components in professional quality of life measure. Flourishing scale (FS), Brief resilient coping scale, compassion satisfaction and work satisfaction were positively correlated and all these were negatively correlated with all the stress measures, Depression, anxiety measures of Depression, Anxiety and Stress Scale (DASS) Burn out and secondary traumatic stress (Table/Fig 9)a,b.

Discussion

Reliability and validity study of questionnaires is important. A translated questionnaire needs to be validated and should be checked for reliability. A translated questionnaire is a requirement once the researcher feels that the responses may be ambiguous pertaining to improperly understood questions by the intended participants. Accurate information through questionnaires is highly questionable, if participants struggle with linguistic comprehension. In such case, it becomes a felt need to translate the survey instrument. It is an exhaustive exercise. This is also important because it will help in identifying any modification, if required. Use of already validated and a reliable questionnaire requires validity and reliability to be checked every time whenever a new sample is collected. Validity is not of the questionnaire and it is of the scores, scored during each filling of the questionnaires (27).

Once validity and reliability are checked and approved in the research, then the internal validity is strengthened and external validity can be deduced from it. A construct is an abstract quality which cannot be directly observed/measured and is thus measured by face validity, content validity and criterion validity. There can be interperson variability in measurement of face validity and therefore, this type of evidence alone is insufficient to demonstrate the validity of questionnaire in total. It is not the exact measure and is a crude measure of validity. Apart from the experts, asking the participants regarding the questionnaire and its overall appeal will satisfy the face validity (28). Content validity refers to the degree to which a test covers all the characteristics being assessed (29).

Content validity is very important as it is a reflection of the variables of a construct (30). The criterion-related validity of a measure refers to the degree to which it is related to other concepts for a theoretically assumed association (7). Questionnaires of the same dimensionality and construct should have convergent validity and the opposite constructs should have a weak correlation or no correlation suggesting divergent validity. Divergent validity is proved by very little correlation and no correlation at all [31,32]. To assess the construct validity of a set of measures in a study, the Multitrait-Multimethod Matrix is an approach (26). It was developed in 1959 by Campbell and Fiske. Convergent validity is the degree to which concepts that should be related theoretically are interrelated in reality, whereas discriminant validity is the degree to which concepts that should not be related theoretically are not interrelated in reality. In this study also, convergent and discriminant validation was done by the Multitrait-Multimethod Matrix (26). In this study, the same results were found in having positive correlated scores of questionnaires measuring similar construct and negative correlation of opposing constructs. Discriminant validity of constructs/sub constructs measured by the questionnaire is proved as non significant correlations. In criterion related validity, concurrent type was analysed. It showed correct understanding and theoretical aligned responses as the construct of same dimensionality, being directly proportional has positive correlation and construct for opposite domain has negative correlation. Predictive convergent and divergent validity can be assessed for some predicted occupational variables like sickness absenteeism and leaving a job because of dissatisfaction or vice versa. Reliability as it depends on the data, needs to be checked in each study and differs for two different samples in different studies and is not a once and for all entity. Cronbach’s alpha is a property of the responses from a specific group of respondents (33). Cronbach’s alpha only indicates reliability of a questionnaire for a particular population of examinees by measuring internal consistency. In all the questionnaires, Cronbach’s alpha more than 0.70 was found except for Participation in Occupational Health and Safety (POHS) where it was 0.686 (34). If content validity of an instrument is lacking, establishing reliability becomes impossible (35).

Limitation(s)

The limitation of the present study was small sample size, same type of study can be applied on a larger sample to confirm the validity and reliability of different scales. Confirmatory Factor Analysis (CFA) and Structural Equation Modelling (SEM) are not a part of this manuscript owing to the huge length of the manuscript and all the CFAs and SEM are entitled as an individual manuscript.

Conclusion

Divergent and convergent validity was proved for different construct/subconstruct in the translated questionnaires. Variables/questionnaires measuring a similar construct were found to be positively correlated and significant. Same way opposite constructs was negatively correlated. The questionnaires need to be translated in the language in which the participants can understand. This is for a better linguistic comprehension and by these accurate responses can be ensured, which in turn will increase the internal validity of the study.

Acknowledgement

The authors would like to acknowledge Dr. Darshana, Dr. Charvi, Dr. Dhara and Dr. Dhruv, Resident Doctors of the Department of Community Medicine for helping in Gujarati translation process and Dr. Anjali Sharma in Hindi translation process.

DOI and Others

DOI: 10.7860/JCDR/2023/62768.18017

Date of Submission: Feb 05, 2023
Date of Peer Review: Mar 02, 2023
Date of Acceptance: Apr 20, 2023
Date of Publishing: Jun 01, 2023

AUTHOR DECLARATION:
• Financial or Other Competing Interests: None
• Was Ethics Committee Approval obtained for this study? Yes
• Was informed consent obtained from the subjects involved in the study? Yes
• For any images presented appropriate consent has been obtained from the subjects. NA

PLAGIARISM CHECKING METHODS:
• Plagiarism X-checker: Feb 08, 2023
• Manual Googling: Mar 16, 2023
• iThenticate Software: Apr 17, 2023 (5%)

ETYMOLOGY: Author Origin