Introduction

The field of patient safety has emerged as a critical public health issue, manifesting implications across various clinical domains. This recognition underscores the necessity of a multifaceted approach to patient safety - one that prioritizes interventions, involves all stakeholders, and establishes a global framework for action. This approach is essential for reducing the likelihood of harm to patients and contributes to healthcare excellence grounded in robust evidence. In this context, the promotion of an open and equitable culture, where health professionals are encouraged to report adverse events and errors, becomes imperative ¹.

Across the globe, healthcare systems inadvertently inflict harm to patients, both occasionally and unintentionally. Such adverse events are widespread and have significant emotional and physical ramifications for patients, their families, and healthcare professionals ². This acknowledgment has propelled patient safety to the forefront of healthcare quality improvement efforts in many countries.

Central to enhancing patient safety is the principle of safety itself, which is fundamental to patient care and a key indicator of management quality. Improvement efforts span across the health system, encompassing a wide range of actions from performance improvement and environmental safety to risk management. This broad framework includes aspects like control of healthcare-associated infections, safe medication practices, equipment safety, safe clinical practices, and ensuring a safe environment, implicating all areas and professionals within healthcare ³.

In the wake of the increasing global emphasis on patient safety, various debates have arisen both nationally and internationally, focusing on implementing best practices in patient care settings. To cultivate a safety culture, addressing patient safety issues in diverse healthcare settings is crucial. Such initiatives foster skill development among health professionals, encouraging proactive measures to minimize adverse events and errors ⁴.

A key aspect of enhancing patient safety is assessing health professionals’ literacy on this subject, a task that has proven to be complex. The absence of systematic assessment models necessitates the use of reliable instruments for determining the need for intervention in patient safety. In this respect, the World Health Organization (WHO) has proposed an evaluation framework based on investigating health course curricula to identify common and underexplored contents in patient safety, as detailed in their “Patient Safety Curriculum Guide: Multi-professional Edition” ⁵.

In 2011, the World Health Organization published the Multiprofessional Guide, a guide with a comprehensive program for implementing patient safety training in healthcare settings worldwide to facilitate and support interdisciplinary knowledge, skills, and attitudes on patient safety in healthcare institutions wherever possible. The Multiprofessional Guide has significantly influenced several initiatives such as the Australian Patient Safety Education Framework (APSEF) ⁶ and the Canadian Framework ⁷. Both identified the main areas of activity (knowledge, skills, and behaviors) that contribute to patient safety based on an extensive review of the corpus of knowledge on this topic. These activities were then categorized into learning areas, further subdivided into 22 main themes, and assigned an expected level of competence according to the level of responsibility within the healthcare system ⁶.

The WHO’s initiative underscores the importance of developing and validating scales, a crucial component in health science research. For data collection instruments to be effective, they must undergo rigorous psychometric validation, demonstrating both cross-sectional and longitudinal measurement properties ⁸.

This study, therefore, aimed to translate, culturally adapt, and validate for Portugal the “KIT TOOL-S2 TEXT. Questionnaire to Assess the Implementation of the Multiprofessional Guide” scale of the World Health Organization. By doing so, our study offers significant insights and advancements in the domain of patient safety and healthcare quality, specifically for Portuguese health professionals. The adaptation and validation of the WHO’s KIT TOOL-S2 TEXT Multiprofessional Guide questionnaire to the Portuguese context provides a critical tool for evaluating and enhancing knowledge, attitudes, and practices related to patient safety. Our findings contribute to a more comprehensive understanding of the questionnaire’s applicability across diverse settings, with the potential to positively influence patient safety and healthcare quality both within Portugal and internationally.

Methodology

Results

The data regarding the statistical analysis of each item, including means and standard deviations, the correlation coefficients between each item and the overall value, as well as the Cronbach’s alpha coefficient without the item, were comprehensively evaluated. The mean indices for the items were generally well centered, ranging from 2.91 for items 7 and 12 to 4.34 for item 22. However, item 14, “It is easier to find someone to blame instead of focusing on the causes of the error,” had a lower mean index than expected (mean = 2.4 ± 1.05).

The total item correlation coefficients indicated that items 8, 9, 10, and 11 should be eliminated due to values below 0.20, yet these items were retained to perform a confirmatory factorial analysis and since the global alpha coefficient would remain unchanged. Interestingly, item 14 demonstrated a negative correlation, necessitating its inversion.

The obtained Cronbach’s alpha coefficients for the different items varied between weak and reasonable. Despite this variation, the overall alpha was robust, standing at 0.759. This comprehensive assessment allowed for nuanced understanding and subsequent decision-making based on the individual item performance and overall scale reliability.

Since it was assumed that a similar structure to the original scale would be found, all items were submitted to exploratory factorial analysis using the procedures described in the methodology. The combination of the results obtained with the Kaiser-Meyer-Olkin criterion (KMO = 0.760), which indicates the adequacy of the sample, and the results of Bartlett’s test of sphericity (χ2 = 2478.100; p = 0.000), which tests the hypothesis that the variables are not correlated, led to proceeding with the exploratory factorial analysis. Initially, seven factors were extracted which, together, accounted for 64.13% of the variability. However, the sedimentation or slope plot “Scree plot” pointed to the retention of four factors according to the inflection point of the curve.

The factor analysis was thus forced to four factors, which resulted in a percentage of variance of 49.08%. Factor 1, designated Patient Error, and Safety was composed of error items 1, 2, 3, 4, 5, 6, and 7 and accounts for 22.97% of the variability; factor 2 - Health System Safety - is composed of items 1, 2, 3, 4, and explains 10.87%; factor 3 - Personal Influence on Safety - has the items 1, 2, 3, 4, 6, and 7 and justifies 8.21%; and finally factor 4 - Personal Attitudes towards Patient Safety - comprises the items 1, 2, 3, 4, 5, and the item 5 of personal information of the original scale, with a variability of 7.027%.

This tetra factorial structure was submitted to confirmatory factor analysis. After analyzing the sensitivity of the items, it was found that all of them had minimum and maximum values ranging from 1 to 5, except for error 7 (“The role of health organizations (e.g., hospitals, general practitioners) in error reporting”?), and showed absolute values of Sk and Ku reference values from below 3-7, respectively.

Table 1 shows the trajectories, critical rations, and lambda coefficients obtained. The first column of the table lists the estimated standardized regression weights, which indicate the strength and direction of the relationship between the latent variables and their corresponding observed variables or indicators. The values of the standardized regression weights range from 0 to 1, with higher values indicating stronger relationships.

Table 1 Trajectories, critical ratios, and lambda coefficients 

We note that the trajectories of the items for their corresponding factors are statistically significant, which leads to their maintenance. However, the items that correspond to “error” 7 (“What is the role of healthcare organizations (e.g., hospitals, general practitioners) in error reporting”?) of factor 1 - Error and Patient Safety - and items 3 (“I feel confident talking to someone who is showing a lack of concern for patient safety”) and 6 (“I believe that filling out reporting forms will help improve patient safety”) of factor 3 - Personal Influence on Safety (“info_personal”) are eliminated in further analysis models because their saturations are less than 0.40. We note that, since this is a preliminary study, factor loadings equal to or greater than 0.40 are considered.

Integrating descriptive and inferential statistics with SEM allows for a comprehensive understanding of the relationships among variables and the underlying structure of the dataset, providing valuable insights for data interpretation and hypothesis testing. All items have saturations above the reference values in the four subscales except for the items previously mentioned, which demonstrates the relevance of the factor to predict the items. Nevertheless, the overall goodness-of-fit indexes only proved to be adequate for χ2/df = 3.631 and RMR = 0.064, acceptable for the GFI = 0.807; RMSEA = 0.094; and SRMR = 0.083, and inadequate for CFI = 0.743.

For improvement purposes, we adjusted the goodness-of-fit indexes of the model based on the modification indexes proposed by AMOS. To this end, some errors were correlated, and items with multicollinearity problems and with saturations below 0.40 were eliminated. We verified that factor 1 was composed of 6 items configuring correlations between model errors 1 and 3 and 5 and 6. Factor 2, factor 3, and factor 4 comprise four items each. The overall adjustment became adequate for χ2/df = 2.602; RMSEA = 0.073; RMR = 0.038; and SRMR = 0.062, and the indexes (GFI = 0.895; CFI = 0.898) are at the threshold of adequate.

In summary, it can be stated that after the hypothesized structure was submitted to confirmatory factor analysis, and after refinement, the following items were eliminated due to multicollinearity problems: item 7 (“The role of health organizations (e.g., hospitals, general practitioners) in error reporting”?) from factor 1 - Error and Patient Safety; item 5 (“The healthcare team receives training in patient safety”) from factor 2 - Health System Safety; and items 3, 4, 5, and 6 (“I feel confident talking to someone who is showing a lack of concern for patient safety,” “I know how to talk to people who have made an error,” I am always able to ensure that patient safety is not compromised,” and “I believe that filling out report sheets will help improve patient safety,” respectively) from factor 3 - Personal Influence on Safety” from factor 3 - Personal Influence on Safety. It should be noted that item 5 (“I am always able to ensure that patient safety is not compromised”) of factor 3 - Personal Influence on Safety became part of factor 4 - Personal Attitudes towards Patient Safety.

The moderate correlations between the various subscales presuppose the existence of a final second-order model, which was designated as the original scale, KIT TOOL-S2. We found that the highest correlation is obtained with factor 1 “error” which explains 60% of the variability and the lowest with factor 2 “safety system” with 22.0% of variability. In this model, the overall adjustment quality values showed slight oscillations. However, the factorial validity (χ2/df = 2.614; RMSEA = 0.073; RMR = 0.040; SRMR = 0.065; GFI = 0.892; CFI = 0.896) was not affected (Fig. 1).

Fig. 1 Second-order model. 

Table 2 summarizes the global adjustment indexes obtained in the different models. Note that the initial model has the least robust goodness-of-fit indexes, but after model refinement and item elimination, they became adequate.

Table 2 Goodness-of-fit indexes in the different models 

Table 3 presents the results of CR, MEV, and DV. The CR in factors 1 and 2 can be classified as adequate since it presents values higher than the reference (0.70), but in factors 3 and 4, the CR is acceptable since it is slightly lower than recommended. As for the MEV, the log indicates that the validity is convergent for factors 1 and 2, and divergent for factors 2 and 3. However, the existence of DV is evident among all factors since the squared correlational values are lower than the MEV. The stratified CR is found to be adequate (0.928), although the MEV, with a value of 0.430, is slightly lower than recommended.

Table 3 CR, mean extracted variance (MEV) and DV of the scale “KIT TOOL-S2 TEXT Questionnaire to Assess the Implementation of the Multiprofessional Guide” 

Stratified CR 0.928, MEV= 0.430.In summary, the values obtained indicate that the scale “KIT TOOL-S2 TEXT Questionnaire to Assess the Implementation of the Multiprofessional Guide” may be a valuable resource for studying the literacy of health professionals on care safety.

Finally, reference is made to Pearson’s correlation matrix between the different subscales and their overall value. The existence of positive and significantly correlated values is denoted, which allows stating that the increase or decrease in the scores of one of the subscales is associated with increases or decreases in the scores of the other subscales with which they correlate. With the global factor and the different dimensions, the correlation coefficients are higher, revealing variability above 29.0% (see Table 4).

Table 4 Pearson’s correlation matrix between the factors 

**p < 0.001.

Discussion

The Multiprofessional Guide aims to encourage both health educators and students around the world to share common understanding, knowledge, and skills on patient safety to ensure the delivery of high-quality, safe services as members of local teams operating within a safety culture and globally connected members by advancing patient safety definitions, content, and solutions. The content of the Multiprofessional Guide considers the wide variety of contexts in which healthcare educators and students teach and learn. The World Health Organization’s Expert Working Group of more than 50 international experts was involved in the development of the curriculum to ensure its cultural appropriateness and global reach. Teaching and assessment strategies are designed to consider both the diversity of available resources and the cultural differences that may affect the learning environment. As a freely available resource for all healthcare systems worldwide, the Multiprofessional Guide reinforces the ethos that safety is all patients’ right and not a “luxury” confined to better resourced healthcare systems ¹⁶.

Although the importance of patient safety training has been widely recognized for over a decade, it remains underutilized and undervalued in many countries ¹⁷. Therefore, the World Health Organization developed the Multi-professional Patient Safety Curriculum Guide to provide schools with the requirements and tools to incorporate patient safety into curricula. It was validated by Farley et al. ¹⁷ in a study with 12 participating schools in six World Health Organization regions evaluating its effectiveness in teaching about patient safety to students in a variety of settings. The results of this study indicate that all participants were positively satisfied with the Multiprofessional Guide, stating that it emphasizes patient safety topics of universal importance, that it was culturally appropriate for their countries, and that they gave it credibility and created a patient safety focus in their schools. It was also shown that students’ perceptions and attitudes about patient safety improved substantially during the on-site application, and their knowledge of the topics they were taught doubled from 10.7% to 20.8% correct answers.

In view of the above, the present study aimed at translating into European Portuguese, cross-culturally adapting, and validating for Portugal the scale “KIT TOOL-S2 TEXT Questionnaire to Assess the Implementation of the Multiprofessional Guide” of the World Health Organization. Translations and back translations were carried out since the original instrument is written in English and required a panel of experts for semantic corrections, especially in terms of sentence construction and replacement of some terms adapted for European Portuguese. The results were obtained from a sample of 300 health professionals working in a hospital center in the central region of the country.

The translation and cultural adaptation of assessment scales should follow the “emic-etic” paradigm, that is, the term “emic” refers to looking at the phenomenon from the very context or culture in which it occurs, while the term “etic” refers to generalizing the observed phenomenon for the purpose of comparison across different cultures 8. In this context, the translation of a scale requires linguistic caution, to the extent that certain terms may have different meanings and specificities intrinsic to each language. On the other hand, semantic validation is essential to certify that the instrument is understandable by the target population. Thus, for the translation, cross-cultural adaptation into Portuguese, and back translation into English of the scale “KIT TOOL-S2 TEXT”, we chose researchers with knowledge in the area. The back translation was essential to ensure that the contents, from the conceptual point of view, did not have to be modified in the translation phase, and this was the key to establishing the semantic equivalence of the scale.

The validity of an instrument is probably the best indicator of its quality. It indicates whether the instrument measures what it is intended to measure, that is, whether the results obtained reflect or translate reality, in the sense of accepting them as indisputable facts. Thus, validity is a complement of reliability because before we can ensure the validity of the instrument, we must ensure its reliability. The literature indicates that the validity of an instrument cannot be taken as an intrinsic characteristic but as a characteristic of the instrument itself when applied to a sample. Thus, it is inferred that the characteristics of the study population can directly influence the structure of the scale “KIT TOOL-S2 TEXT Questionnaire to Assess the Implementation of the Multiprofessional Guide” of the World Health Organization. Our exploratory factor analysis of this scale resulted in a tetra factor structure that accounts for 43.0% of the variance and with an overall Cronbach’s alpha coefficient of 0.759. The overall goodness-of-fit indexes were found to be reliable, confirming the quality of the model fit to the empirical data (χ2/df = 2.602; GFI = 0.895; CFI = 0.898; RMR = 0.038; RMSEA = 0.073) and SRMR = 0.062. The evidence for the validity of the scale was presented, showing all the possibilities of its use in the Portuguese context, allowing the establishment of bases for the construction of a new instrument that assesses the literacy of health professionals on care safety.

The World Health Organization’s Multiprofessional Guide has been instrumental in shaping patient safety training initiatives worldwide. It has inspired various frameworks, including the Australian Patient Safety Education Framework and the Canadian Framework, which identified main areas of activity contributing to patient safety. The Multiprofessional Guide aims to promote a common understanding of patient safety among health educators and students worldwide to ensure safe, high-quality healthcare services. The curriculum considers cultural appropriateness and global reach and is designed to consider both the diversity of available resources and the cultural differences that may affect the learning environment. Although patient safety training has been recognized for over a decade, it remains underutilized and undervalued in many countries. To address this, the World Health Organization developed the Multiprofessional Patient Safety Curriculum Guide to provide schools with requirements and tools to incorporate patient safety into their curricula. The present study aimed to cross-culturally adapt and validate the scale “KIT TOOL-S2 TEXT Questionnaire to Assess the Implementation of the Multiprofessional Guide” for Portugal. The validity of the instrument was established by exploring its factor structure, accounting for 43.0% of the variance, and with an overall Cronbach’s alpha coefficient of 0.759. The goodness-of-fit indexes confirmed the quality of the model fit to the empirical data. Overall, this study contributes to the establishment of a new instrument that assesses the literacy of health professionals on patient safety in the Portuguese context.

Conclusions

This study’s investigation into the “KIT TOOL-S2 TEXT Questionnaire to Assess the Implementation of the Multiprofessional Guide” scale underscores its efficacy as a psychometric tool. The scale’s satisfactory validation underscores its utility in assessing healthcare professionals’ literacy regarding patient and healthcare safety. Our findings highlight the scale’s robust factorial structure and its ability to accurately represent the underlying constructs of interest while maintaining its psychometric integrity.

The confirmatory analyses conducted support the scale’s construct validity, suggesting its applicability in diverse healthcare settings. This validation endorses the scale as a reliable instrument for future research endeavors focused on patient safety.

The “KIT TOOL-S2 TEXT” scale offers a promising avenue for developing and evaluating interventions aimed at enhancing patient safety attitudes and behaviors among healthcare professionals. Its application in longitudinal studies presents an opportunity to observe temporal changes in these attitudes and behaviors. Additionally, it facilitates the assessment of the impact of patient safety training on healthcare outcomes, an area ripe for exploration in future research endeavors.

In summary, the validation of this scale marks a significant contribution to the field of patient safety and healthcare quality. It opens new possibilities for research focused on improving patient care and safety across various healthcare contexts, both within Portugal and internationally.

Statement of Ethics

This research was conducted following the ethical standards and guidelines of our institution. All procedures involving human participants were reviewed and approved by the Institutional Review Board (IRB) at our institution, with reference number 04/21/10/2019. Informed consent was obtained from all individual participants involved in the study. Adequate measures were taken to ensure the confidentiality and privacy of the participants. The data collected have been used solely for this research study.

Conflict of Interest Statement

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Funding Sources

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author Contributions

We affirm that all authors have made substantial contributions to the accomplishment of this manuscript. All authors were involved in the conception and design of the study, the collection and analysis of data, the writing of the manuscript, and the critical review of its intellectual content. Moreover, all authors have approved the final version to be published and agreed to be accountable for all aspects of the work, ensuring that questions related to the accuracy or completeness of any part of the work are appropriately investigated and resolved.

Data Availability Statement

This study adheres to the principles of data transparency and accessibility, in alignment with the journal’s data sharing policies. We hereby declare that the datasets underpinning the conclusions of this manuscript are available without undue restriction, within the ethical and legal boundaries. Data access is granted upon direct request to the primary authors, who will provide the data responsibly following ethical research practices. Where legal or ethical limitations preclude public data sharing, these are explicitly stated, detailing the conditions under which data access may be permitted. The research team ensures the privacy and confidentiality of participants in compliance with institutional guidelines and regulatory standards for data protection.