Brief Original Article

Introduction

Wildfire firefighting is a demanding and dangerous occupation that exposes thousands of workers to significant risks each year. In the United States, an annual average of 100 deaths among wildfire firefighters is reported ^{1 ➤
1.U.S. Fire Administration. Firefighter Fatalities in the United States (2022). Available in: https://apps.usfa.fema.gov/firefighter-fatalities/} . In Mexico, the National Forestry Commission (CONAFOR, by its Spanish acronym) reported that between 1969 and 2022, 274 wildfire firefighters died in action ^{2 ➤
2.CONAFOR, Comisión Nacional Forestal (2018), Nos llaman combatientes. Available in: https://www.gob.mx/conafor/articulos/nos-llaman-combatientes .} . It is important to note that 91% of accidents and injuries occur on the fire line ^{3 ➤
3.Moody, V.J., Purchio, T.J., & Palmer, C. (2019). Descriptive analysis of injuries and illnesses self-reported by wildland firefighters. International Journal of Wildland Fire . doi:10.1071/WF18132 .} . The leading causes of death include medical conditions resulting from exposure to particulate matter and smoke inhalation, such as acute myocardial infarctions and cerebrovascular events, which account for 30% of the deaths. These are followed by motor vehicle accidents (27%) and aviation accidents (26%) during fire suppression maneuvers <sup>4 ➤
4. Butler, C.R., Marsh, S.M., Domitrovich, J.W., & Helmkamp, J.C. (2017). Wildland firefighter deaths in the United States: A comparison of existing surveillance systems. Journal of Occupational and Environmental Hygiene, 14, 258 - 270. doi:10.1080/15459624.2016.1250004 .</sup> . The protection of wildfire firefighters requires significant investment in resources, healthcare, continuous training, as well as the study of working conditions and safety culture within institutions. The National Institute for Occupational Safety and Health (NIOSH), under the Centers for Disease Control and Prevention (CDC), has proposed several tools to analyze workers' health and well-being. Among these, the NIOSH Worker Well-Being Questionnaire (WBG), part of the Total Worker Health initiative, stands out as one of the most advanced tools for assessing the work environment and worker well-being <sup>5 ➤
5. Chari, R., Chang, C. C., Sauter, S. L., Petrun Sayers, E. L., Huang, W., & Fisher, G. G. (2021). NIOSH worker well-being questionnaire (WellBQ). U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health. doi:10.26616/NIOSHPUB2021110revised52021 . 6 ➤
6. Chari, R., Sauter, S. L., Petrun Sayers, E. L., Huang, W., Fisher, G. G., & Chang, C. C. (2022). Development of the National Institute for Occupational Safety and Health Worker Well-Being Questionnaire. Journal of occupational and environmental medicine, 64(8), 707–717. doi:10.1097/JOM.0000000000002585 . 7 ➤
7. Fontana, L. D., Pasquale; Santocono, Carolina; Annarumma, Maddalena; Iavicoli, Ivo. (2023). Validation of the NIOSH Worker Well-Being Questionnaire in Italian Language. Journal of occupational and environmental medicine, 65(6), e402-e412. doi:10.1097/jom.0000000000002835 .</sup> . Worker well-being encompasses aspects of both work and personal life, including occupational safety and health elements. This concept can be crucial for productivity at individual, business, and societal levels and can be measured both subjectively and objectively, or through a combination of both approaches. It includes factors such as prosperity, happiness, satisfaction, sense of purpose, and adequate material and legal conditions <sup>8 ➤
8. Schulte, P. A., Guerin, R. J., Schill, A. L., Bhattacharya, A., Cunningham, T. R., Pandalai, S. P., Eggerth, D., & Stephenson, C. M. (2015). Considerations for Incorporating "Well-Being" in Public Policy for Workers and Workplaces. American journal of public health, 105(8), e31–e44. doi:10.2105/AJPH.2015.302616 .</sup> .

In research that requires the validation of instruments such as surveys, it is essential to use appropriate statistical tools to correctly summarize the ratings provided by experts. One of the main recommendations in this field is the use of Aiken's V, as suggested by Dunn et al. (1999), based on the earlier work of Aiken (1980, 1985) <sup>9 ➤
9. Dunn, J. G. H., Bouffard, M., & Rogers, W. T. (1999). Assessing content-relevance in sport psychology scale-construction research: Issues and recommendations.Measurement in Physical Education and Exercise Science, 3,15–36. doi:10.1207/s15327841mpee0301_2 . 10 ➤
10. Aiken, L. R. (1980). Content Validity and Reliability of Single Items or Questionnaires. Educational and Psychological Measurement, 40(4), 955-959. doi:10.1177/001316448004000419 . 11 ➤
11. Aiken, L. R. (1985). Three Coefficients for Analyzing the Reliability and Validity of Ratings. Educational and Psychological Measurement, 45(1), 131-142. doi:10.1177/0013164485451012 .</sup> . Aiken's V not only effectively summarizes the magnitude of expert ratings but also allows for testing specific hypotheses related to the values obtained for the study population. Thus, this measure has become a valuable tool in qualitative research, contributing to the rigor and precision in interpreting expert ratings <sup>12 ➤
12. Penfield Randall D; Giacobbi, Peter R.. (2004) Applying a Score Confidence Interval to Aiken's Item Content-Relevance Index, Measurement in Physical Education and Exercise Science, 8:4, 213-225, doi:10.1207/s15327841mpee0804_3 .</sup> . Several studies have reported strategies to improve worker well-being. It has been shown that such interventions increase well-being and productivity while reducing health risks, leading to significant short-term improvements in health status and work performance <sup>13 ➤
13. Hamar, B., Coberley, C., Pope, J. E., & Rula, E. Y. (2015). Well-being improvement in a midsize employer: changes in well-being, productivity, health risk, and perceived employer support after implementation of a well-being improvement strategy. Journal of occupational and environmental medicine, 57(4), 367–373. doi:10.1097/JOM.0000000000000433 .</sup> . In another study, it was identified that job and well-being inventories can be used by occupational health professionals to detect risk factors for prolonged absenteeism due to illness and thus provide timely interventions <sup>14 ➤
14. Vendrig, A. A., & Schaafsma, F. G. (2018). Reliability and Validity of the Work and Well-Being Inventory (WBI) for Employees. Journal of occupational rehabilitation, 28(2), 377–390. doi:10.1007/s10926-017-9729-7 .</sup> . Hagan-Haynes et al. (2024) explored worker well-being in the context of early childhood education, evaluating experiences of workplace mistreatment and its impact on mental health. Using the NIOSH WBG, they found that workplace mistreatment, such as condescending treatment and harassment, was associated with an increased number of days of poor mental health <sup>15 ➤
15. Hagan-Haynes, K. M., Virginia; Puma, Jini; Farewell, Charlotte. (2024). Caring for the Caregiver: Work Mistreatment and Well-Being Among Early Childhood Education Staff in Colorado. Early Childhood Education Journal. doi:10.1007/s10643-024-01644-6 .</sup> . The objective of the present study is to evaluate the validity of the scores obtained with the Spanish version of the NIOSH WBG in wildfire firefighters in Mexico.

METHODS

This is a psychometric study that evaluated the content validity of the WBG through expert judgment, focusing on worker well-being. Data collection took place between April and May 2023.

The WBG, developed by NIOSH, provides a comprehensive assessment of worker well-being across five domains: quality of work life, circumstances outside of work, physical health, mental health, and general well-being. The questionnaire consists of 68 items, designed as part of NIOSH’s initiative to expand the Occupational Safety and Health Paradigm, in collaboration with the RAND Corporation <sup>5 ➤
5. Chari, R., Chang, C. C., Sauter, S. L., Petrun Sayers, E. L., Huang, W., & Fisher, G. G. (2021). NIOSH worker well-being questionnaire (WellBQ). U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health. doi:10.26616/NIOSHPUB2021110revised52021 . 6 ➤
6. Chari, R., Sauter, S. L., Petrun Sayers, E. L., Huang, W., Fisher, G. G., & Chang, C. C. (2022). Development of the National Institute for Occupational Safety and Health Worker Well-Being Questionnaire. Journal of occupational and environmental medicine, 64(8), 707–717. doi:10.1097/JOM.0000000000002585 .</sup> .

The original instrument, administered to 1,894 participants, showed solid evidence of validity and reliability through exploratory (EFA) and confirmatory (CFA) factor analyses. These analyses evaluated dimensionality and construct validity, achieving satisfactory fit with indices such as the Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI), both above 0.93. Additionally, the internal consistency of the questionnaire was robust, with Cronbach's alpha values above 0.8 in most cases <sup>6 ➤
6. Chari, R., Sauter, S. L., Petrun Sayers, E. L., Huang, W., Fisher, G. G., & Chang, C. C. (2022). Development of the National Institute for Occupational Safety and Health Worker Well-Being Questionnaire. Journal of occupational and environmental medicine, 64(8), 707–717. doi:10.1097/JOM.0000000000002585 .</sup> .

During the development of the instrument, it was administered in Spanish to a sample of 106 Hispanic workers, and the original authors offer this Spanish version on the NIOSH website <sup>6 ➤
6. Chari, R., Sauter, S. L., Petrun Sayers, E. L., Huang, W., Fisher, G. G., & Chang, C. C. (2022). Development of the National Institute for Occupational Safety and Health Worker Well-Being Questionnaire. Journal of occupational and environmental medicine, 64(8), 707–717. doi:10.1097/JOM.0000000000002585 .</sup> . Other studies have validated the instrument in different languages, such as Italian, where it was administered to 206 workers. In that case, the model showed good fit with CFI and TLI indices ranging from 0.96 to 0.99, and a root mean square error of approximation (RMSEA) between 0.03 and 0.07 <sup>7 ➤
7. Fontana, L. D., Pasquale; Santocono, Carolina; Annarumma, Maddalena; Iavicoli, Ivo. (2023). Validation of the NIOSH Worker Well-Being Questionnaire in Italian Language. Journal of occupational and environmental medicine, 65(6), e402-e412. doi:10.1097/jom.0000000000002835 .</sup> .

For this study, each WBG item was evaluated in terms of clarity, coherence, and relevance using a six-point Likert scale, where one indicated the lowest level of agreement and six the highest <sup>16 ➤
16. Ventura-León, J. (2020). Escalas, inventarios y cuestionarios: ¿son lo mismo? Educación Médica, 21(3), 218-220. doi:10.1016/j.edumed.2019.04.001 .</sup> . The scale design avoided intermediate options to eliminate low-utility responses <sup>17 ➤
17. Simms, L. J., Zelazny, K., Williams, T. F., & Bernstein, L. (2019). Does the number of response options matter? Psychometric perspectives using personality questionnaire data. Psychological assessment, 31(4), 557–566. doi:10.1037/pas0000648.</sup> . Clarity, coherence, and relevance criteria were evaluated following this approach <sup>18 ➤
18. Torres-Malca, J. R., Vera-Ponce, V. J., Zuzunaga-Montoya, F. E., Talavera, J. E., & De La Cruz-Vargas, J. A. (2022). Validez de contenido por juicio de expertos de un instrumento para medir conocimientos, actitudes y prácticas sobre el consumo de sal en la población peruana. Revista de la Facultad de Medicina Humana, 22(2), 273-279. doi:10.25176/rfmh.v22i2.4768 .</sup> .

Six expert judges were selected for the content validation of the WBG: a doctor in experimental psychology (expert 1), a doctor in occupational health sciences (expert 2), a master’s in forestry sciences (expert 3), a master’s in psychology (expert 4), a master’s in clinical research (expert 5), and a senior technician in rescues (expert 6). These judges were chosen based on their education, research, and experience in instrument validation and wildfire firefighting to complete their responses.

Each item was evaluated concerning the criteria of clarity, coherence, and relevance, using the Likert scale options described above: totally disagree, disagree, slightly disagree, slightly agree, agree, and totally agree. Aiken’s V coefficient was used as the decision criterion to determine the retention of items. The exact critical value of Aiken’s V coefficient was calculated following the original formula proposed by Aiken <sup>10 ➤
10. Aiken, L. R. (1980). Content Validity and Reliability of Single Items or Questionnaires. Educational and Psychological Measurement, 40(4), 955-959. doi:10.1177/001316448004000419 .</sup> .

To assess the degree of agreement among expert judges, Aiken’s V coefficient was calculated. A value of Aiken’s V ≥ 0.70 was considered adequate. Additionally, 95% confidence intervals and corresponding variability coefficients were calculated. Statistical analyses were conducted using a database created in Microsoft Excel, Office 365 version 2016 for Windows 11.

The study was approved by the ethics committee of the academic board of the Doctoral Program in Occupational Health Sciences at the Universidad de Guadalajara. This research was conducted in accordance with the Regulations of the General Health Law of Mexico on health research and was classified as a risk-free study. The data were handled confidentially and used exclusively for research purposes. Participation of the experts was voluntary, and all of them signed informed consent before starting their collaboration.

RESULTS

The following tables present the results obtained from the five domains of the WBG for the criteria of "Clarity," "Coherence," and "Relevance." The 68 items of the instrument were evaluated. Each table describes the mean score per item according to the total number of judges, the standard deviation, Aiken's V coefficient, and the 95% confidence intervals.

The results for the "Work Evaluation and Experience" domain show that all items achieved Aiken’s V values above 0.70 for the three criteria analyzed (Clarity, Coherence, and Relevance), as detailed in Table 1..

In the "Workplace Policies and Culture" domain, the items also presented Aiken’s V values higher than 0.70 across the three evaluated criteria. The experts considered all items to be relevant, as shown in Table 2.

For the "Physical Environment at Work and Safety Climate" domain, all evaluated items displayed Aiken’s V above 0.70 across the three criteria, as shown in Table 3. In this domain, an additional item (P33D) refers to "Accommodations for disabilities or special needs" (e.g., wheelchair ramps or breastfeeding rooms). It is recommended to consult the full Spanish version of the instrument, available on the NIOSH website, for more details on this and other related items <sup>5 ➤
5. Chari, R., Chang, C. C., Sauter, S. L., Petrun Sayers, E. L., Huang, W., & Fisher, G. G. (2021). NIOSH worker well-being questionnaire (WellBQ). U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health. doi:10.26616/NIOSHPUB2021110revised52021 . 6 ➤
6. Chari, R., Sauter, S. L., Petrun Sayers, E. L., Huang, W., Fisher, G. G., & Chang, C. C. (2022). Development of the National Institute for Occupational Safety and Health Worker Well-Being Questionnaire. Journal of occupational and environmental medicine, 64(8), 707–717. doi:10.1097/JOM.0000000000002585 .</sup> .

Regarding the "Clarity" evaluation criterion, most items achieved Aiken’s V above 0.70, except for item P44 ("Have you ever had chronic insomnia?"), which obtained Aiken’s V of 0.67, see Tabla 4. lthough this value reflects a slight concern in terms of clarity, the other criteria (coherence and relevance) showed values above 0.70, supporting the content validity of this item. The lower clarity result could be due to contextual or interpretative factors, which do not significantly compromise its functionality within the questionnaire. For this reason, it was decided to retain the item with the possibility of making minor adjustments to its wording.

Finally, the three evaluation criteria for the "Home, Community, and Society" domain also show Aiken’s V values above 0.70 in all analyzed items, as presented in Table 5.

DISCUSIÓN

This study successfully validated the Spanish version of the NIOSH WBG for application in wildfire firefighters in Mexico. The results demonstrate that the questionnaire is a useful tool for assessing occupational well-being in this specific context. Most of the items obtained Aiken’s V coefficient above 0.70, confirming high content validity. These findings are consistent with previous studies, such as the validation of the questionnaire in its Italian version, which also showed robust model fit and high internal consistency <sup>5 ➤
5. Chari, R., Chang, C. C., Sauter, S. L., Petrun Sayers, E. L., Huang, W., & Fisher, G. G. (2021). NIOSH worker well-being questionnaire (WellBQ). U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health. doi:10.26616/NIOSHPUB2021110revised52021 . 6 ➤
6. Chari, R., Sauter, S. L., Petrun Sayers, E. L., Huang, W., Fisher, G. G., & Chang, C. C. (2022). Development of the National Institute for Occupational Safety and Health Worker Well-Being Questionnaire. Journal of occupational and environmental medicine, 64(8), 707–717. doi:10.1097/JOM.0000000000002585 . 7 ➤
7. Fontana, L. D., Pasquale; Santocono, Carolina; Annarumma, Maddalena; Iavicoli, Ivo. (2023). Validation of the NIOSH Worker Well-Being Questionnaire in Italian Language. Journal of occupational and environmental medicine, 65(6), e402-e412. doi:10.1097/jom.0000000000002835 .</sup> .

Although item P44 presented a slightly lower Aiken’s V value for clarity (0.67), its relevance in the coherence and relevance criteria, with values above 0.70, justifies its retention in the questionnaire. The lower clarity score could be attributed to contextual or interpretative factors, without significantly affecting its effectiveness. However, minor adjustments to its wording could be made to improve comprehension without compromising its functionality or content validity <sup>12 ➤
12. Penfield Randall D; Giacobbi, Peter R.. (2004) Applying a Score Confidence Interval to Aiken's Item Content-Relevance Index, Measurement in Physical Education and Exercise Science, 8:4, 213-225, doi:10.1207/s15327841mpee0804_3 . 19 ➤
19. Merino-Soto, C. (2018). Confidence interval for difference between coefficients of content validity (Aiken’s V): A SPSS syntax. [Intervalos de confianza para la diferencia entre coeficientes de validez de contenido (V Aiken): Una sintaxis SPSS] Anales De Psicología, 34(3), 587-590. doi:10.6018/analesps.34.3.283481 . 20 ➤
20. Dieguez-Reyes E, Valdés-Santiago D. Validez de contenido de la escala RED-Tecnoestrés en el sector informático cubano. La Habana, 2023. Revista Cubana de Salud y Trabajo [Internet]. 2023 [citado 17 Nov 2023]; 25 (1) Available in: https://revsaludtrabajo.sld.cu/index.php/revsyt/article/view/403</sup> .

In future research, it is recommended to conduct EFA and CFA to evaluate the factorial structure of the questionnaire in this specific population. Additionally, longitudinal studies examining the impact of interventions based on WBG results on workers' quality of life and productivity would be valuable. A crucial step will be the implementation of a pilot test to assess the practical applicability of the instrument. After this piloting stage, further phases such as gathering validity evidence through internal consistency and, finally, proceeding with factor analysis to determine construct validity will follow. This will ensure that the instrument is not only content-valid but also robust in its ability to measure the construct of interest accurately and reliably in other populations <sup>5 ➤
5. Chari, R., Chang, C. C., Sauter, S. L., Petrun Sayers, E. L., Huang, W., & Fisher, G. G. (2021). NIOSH worker well-being questionnaire (WellBQ). U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health. doi:10.26616/NIOSHPUB2021110revised52021 . 6 ➤
6. Chari, R., Sauter, S. L., Petrun Sayers, E. L., Huang, W., Fisher, G. G., & Chang, C. C. (2022). Development of the National Institute for Occupational Safety and Health Worker Well-Being Questionnaire. Journal of occupational and environmental medicine, 64(8), 707–717. doi:10.1097/JOM.0000000000002585 .</sup> .

The results of this study are particularly relevant for the development of policies and strategies aimed at improving working conditions and well-being among wildfire firefighters ^{21 ➤
21.Koopmans, E., Cornish, K., Fyfe, T. M., Bailey, K., & Pelletier, C. A. (2022). Health risks and mitigation strategies from occupational exposure to wildland fire: a scoping review. Journal of occupational medicine and toxicology (London, England), 17(1), 2. doi:10.1186/s12995-021-00328-w} . Organizations can use this tool not only to regularly monitor employee well-being but also to identify risk factors and design specific interventions that promote a safer and healthier work environment <sup>5 ➤
5. Chari, R., Chang, C. C., Sauter, S. L., Petrun Sayers, E. L., Huang, W., & Fisher, G. G. (2021). NIOSH worker well-being questionnaire (WellBQ). U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health. doi:10.26616/NIOSHPUB2021110revised52021 . 6 ➤
6. Chari, R., Sauter, S. L., Petrun Sayers, E. L., Huang, W., Fisher, G. G., & Chang, C. C. (2022). Development of the National Institute for Occupational Safety and Health Worker Well-Being Questionnaire. Journal of occupational and environmental medicine, 64(8), 707–717. doi:10.1097/JOM.0000000000002585 . 7 ➤
7. Fontana, L. D., Pasquale; Santocono, Carolina; Annarumma, Maddalena; Iavicoli, Ivo. (2023). Validation of the NIOSH Worker Well-Being Questionnaire in Italian Language. Journal of occupational and environmental medicine, 65(6), e402-e412. doi:10.1097/jom.0000000000002835 . 15 ➤
15. Hagan-Haynes, K. M., Virginia; Puma, Jini; Farewell, Charlotte. (2024). Caring for the Caregiver: Work Mistreatment and Well-Being Among Early Childhood Education Staff in Colorado. Early Childhood Education Journal. doi:10.1007/s10643-024-01644-6 .</sup> . This proactive approach will address workers' needs, improving both their quality of life and organizational efficiency.

Among the main limitations of the study is the inherent subjectivity of expert judgment, which could introduce biases in the WBG content evaluation. Moreover, the use of Aiken’s V coefficient, while appropriate for measuring the degree of agreement among experts, does not fully capture the richness and depth of their qualitative opinions. Despite these limitations, a key strength is the successful validation of the WBG for wildfire firefighters in Mexico, ensuring its suitability for measuring occupational well-being in this specific context. The high content validity, with most items obtaining Aiken’s V above 0.70, reinforces the reliability of the instrument, and its results are consistent with previous studies in other international versions, ensuring its future usefulness and application.

Conclusion

In conclusion, the results of this study provide validity evidence supporting the application of the instrument among wildfire firefighter workers in the Mexican context. It is crucial to continue the validation process, including pilot tests and factor analyses, to ensure its applicability and relevance in this specific setting. Furthermore, it is expected that this evidence will encourage other researchers to use the Spanish version of the instrument in different contexts and with other work populations across Latin America and the Caribbean.