RESEARCH ARTICLE

Cross-Cultural Adaptation and Validation of the Portuguese Version of the Physical Therapy Outpatient Satisfaction Survey

Adaptação e validação intercultural da versão portuguesa do Physical Therapy Outpatient Satisfaction Survey

 

Luís M. Cavalheiro^ab Jan M. Cabri  ^c Pedro L. Ferreira ^bd

^a Department of Physical Therapy, Polytechnic Institute of Coimbra, College of Health Technology of Coimbra, Coimbra, Portugal

^b University of Coimbra, Centre for Health Studies and Research, Coimbra, Portugal

^c Department of Physical Performance, Norwegian School of Sport Sciences, Oslo, Norway

^d Faculty of Economics, University of Coimbra, Coimbra, Portugal 

 

ABSTRACT

The aim of this study was to adapt and validate the Physical Therapy Outpatient Satisfaction Survey (PTOPS) for the Portuguese culture. This version was obtained by a forward/backward translation, consensus panels, and pre-test. The Portuguese PTOPS was administered to 76 physical therapy outpatients in 10 health services. The content analysis (panels of experts and lay people) and the factor analysis resulted in a reduction of the original 34 items to 28 items that validly identify 3 constructs. The reliability was acceptable for both internal consistency (Cronbach’s alpha = 0.73) and reproducibility (ICC between 0.84 and 0.87), which represent acceptable levels of validity and reliability.

Keywords: Outpatient satisfaction Physical therapy Measures Healthcare quality

 

RESUMO

Foi objetivo deste estudo adaptar e validar para a cultura portuguesa o Physical Therapy Outpatient Satisfaction Survey (PTOPS). Esta versão resultou do processo de tradução, retroversão, painéis de consenso e pré teste. A PTOPS foi administrada a 76 doentes ambulatórios de fisioterapia, em 10 instituições de saúde. Da análise de conteúdo (painéis de peritos e gente comum) e da análise fatorial resultou uma redução dos 34 itens da versão original para 28 itens, que identificam validamente 3 constructos. A fiabilidade foi aceitável quer na coerência interna (α de Cronbach = 0,73), quer na reprodutibilidade (ICC entre 0,84 e 0,87). Evidenciando níveis aceitáveis de validade e fiabilidade.

Palavras-chave: Satisfação dos utentes · Fisioterapia · Medidas de resultado · Qualidade dos cuidados

 

Introduction

The assessment of patient satisfaction and the measurement of healthcare outcomes are a priority for quality management of health services (¹ -⁴ ). In the context of physical therapy and its own specificities, the lack of adequate satisfaction instruments prevents that desideratum. A healthcare process characterized by longer provider/patient interactions, involving more physical contact, often requiring the active involvement of the patient, and more frequent visits are some of the features identified, which may influence both the outcome of treatment and the view of its quality (⁵ -⁹ ).

However, in the literature some instruments oriented towards the satisfaction with the physical therapy care can be found, including the Physical Therapy Outpatient Satisfaction Survey (PTOPS) ⁹, the Physical Therapy Patient Satisfaction Questionnaire (PTPSQ) ¹⁰, the Patient Survey Instrument (PSI) ⁵, and the Patient Satisfaction Questionnaire for Physical Therapy (PSQ-PT) ⁷.

The PTOPS is a measure of patient satisfaction with the outpatient physical therapy care, which was developed in 3 phases that led to the identification of a structured model in 4 dimensions related to satisfaction and dissatisfaction with the provision of such care ⁹.

The PTOPS is an adequate instrument to measure the satisfaction of physical therapy patients in both clinical practice and research. In order to make its use possible in the Portuguese culture, an adaptation and intercultural validation process is required (¹¹ -¹⁴ ). The aim of this study was to perform the translation and adaptation of PTOPS for the Portuguese language and culture and to examine its validity and reliability.

 

Methods

Cross-Cultural Adaptation

The PTOPS cultural adaptation process followed a sequential methodology, which is the most common procedure for this purpose (¹¹ -¹⁵ ).

The process began after the authors’ permission ⁹. The original version was delivered to two Portuguese translators with high fluency in English who, independently, made the English/Portuguese translations. Both versions were checked in the presence of the translators in order to obtain a consensus version (first preliminary version). Later, this version was translated back to En­glish by two other translators. The translations and back-translations were used in a second consensus panel, resulting in a second preliminary version.

For the understandability analysis, the second preliminary version was submitted to a panel of 6 experts in the management and/or healthcare quality fields and a panel of lay people, users of outpatient physical therapy care, involving 10 subjects. The outcome of the reached consensus was the pre-final Portuguese version of the PTOPS.

Validation Study

Sample

The pre-final Portuguese version of the PTOPS was administered to a sample of patients of health services in the central region of Portugal. The subjects were selected after obtaining informed consent. To be included in the study, subjects were required to be finishing or have finished an outpatient physical therapy care in less than 1 week, to be literate, and to not have any cognitive disorder that would prevent the completion of the questionnaire. The participation of the health services in the study was subject to prior ethics authorization.

Measurements

The PTOPS is a measure of patient satisfaction with the out­patient physical therapy care designed to be self-administered. The original version provides 34 items and 4 dimensions: (i) Enhancer – related to the physical environment and personal interactions, which when present, enrich in a positive way the experience of the patient; (ii) Detractor – recognition of basic interpersonal and physical needs of a patient which, when absent, generate a negative feeling; (iii) Location – ease of locating and traveling to a clinic; and (iv) Costs – compatibility between the perceived value of service and cost. The scores of the dimensions are derived from the arithmetic mean of their items, with a minimum of 1 and a maximum of 5 points. In the Enhancer and Location dimensions, higher scores correspond to greater satisfaction and in the Detractor and Costs dimensions, higher scores indicate less satisfaction.

Statistical Analysis

The construct validity was assessed through factor analysis which, in essence, examines the correlations between different variables to find a set of factors/components that, theoretically, represent what the examined variables have in common ¹⁶.

Commonalities were analyzed, excluding the items with a weight factor of less than 0.4. For the definition of the main components, the existence of multicollinearities of the input variables was observed using the Bartlett test of sphericity and the Kaiser-Meyer-Olkin measure of sampling adequacy, obtaining values of the original correlated variables greater than or equal to 0.5. The number of main components to retain took into account the eigenvalues larger than 1, also the observation of its graphical representation through the screen plot, and by the criterion of the percentage of explained variance. For extraction of the main components the method of varimax rotation was used (component matrix with orthogonal rotation “T”) with the Kaiser standards criterion (¹⁷ , ¹⁸ ).

The validity was further analyzed by the study of relations obtained by the application of the measure and those registered using independent questions that could be related to the meaning of the responses to PTOPS: (i) I would recommend these services to family or friends; (ii) I would come back to this service if I needed physical therapy treatment again, (iii) In general I am satisfied with my experience of physical therapy.

For each question, a Likert scale of 5 points was used (1 = strongly disagree, 2 = disagree, 3 = uncertain, 4 = agree, 5 = strongly agree). An identical procedure was used and advocated by Goldstein et al. ¹⁰.

The study of relations was performed with Spearman’s correlation coefficient. For the interpretation of correlation coefficients, the criteria of Cohen and Holliday ¹⁹ were used, which suggest the following classification: very low correlation for values less than 0.20; low if between 0.20 and 0.39; moderate if between 0.40 and 0.69; high if between 0.70 and 0.89; and very high for values equal to or greater than 0.90.

The test-retest reliability was examined by the application of the measure with an interval of 1 week, using the intraclass correlation coefficient (ICC). A coefficient of reproducibility equal to or greater than 0.70 was considered acceptable to compare groups ²⁰.

The internal consistency or homogeneity of PTOPS was tested by Cronbach’s alpha. Values between 0.70 and 0.95 were considered indicative of acceptable reliability (¹⁸ -²¹ ).

Values of p < 0.05 were considered statistically significant. Statistical analysis was performed using the Statistical Package for the Social Sciences – SPSS version 17.0 for Windows (SPSS Inc., Chicago, IL, USA).

 

Results

Cross-Cultural Adaptation

After obtaining the second preliminary version, 2 minor changes were identified as necessary by subsequent panels (lay people and experts). (i) Adding the code “9 = not applicable” to the Likert scale that accompanies each item. In fact, it was detected that some items are not entirely suited for the Portuguese context. These items are, especially, related to costs, since in the Portuguese health system, patients do not pay or only pay a small share of the value of the treatment carried out, a part being directly allocated to the health system. (ii) Adding a brief introduction on how to complete the measure in order to improve its understandability – “Please place an X in the appropriate box to indicate your degree of satisfaction with each of the following statements (1 = strongly disagree, 2 = disagree, 3 = uncertain, 4 = agree, 5 = strongly agree. If not applicable to your situation, please check 9).”

After these changes were made, the understanding of the panels was the following: the pre-final Portuguese version of the PTOPS is a short measure that is easy to answer, understandable, and useful and the used language is simple, clear, and colloquial.

Validation Study

Sample

After the abovementioned changes, the pre-final Portuguese version of the PTOPS was administered to a sample of 76 subjects (Table 1) who were finishing or had already finished their outpatient physical therapy care in 10 health services (4 public and 6 private) in the central region of the country. Of the total sample, 16 subjects (21%) also participated in the reproducibility assessment.

 

 

Validity

In the study of the pre-final Portuguese version of the PTOPS, the factor analysis confirmed the problems initially anticipated for the Costs dimension. Indeed, the analysis identified the items 4, 9, 17, 25, and 30 as indeterminate. Item 16 (Enhancer – I have to wait too long between appointments) was also considered indeterminate. After removing these items, it was possible to identify the retention of 3 factors (fig. 1), classified as Enhancer, Detractor, and Location.

 

 

The component matrix after orthogonal rotation, which allowed the extraction of variables associated with each of the factors, is shown in Table 2. Item 8 was retained in the component Location and not in the component Detractor, since the correlation values are identical and this was its position in the original measure.

 

 

Thus, in the Portuguese version of the PTOPS, the 34 original items were reduced to a version consisting of 28 items distributed among 3 dimensions: Enhancers – 2, 6, 10, 14, 18, 20, 21, 24, 28; Detractors – 1, 3, 5, 7, 9, 11, 13, 15, 16, 19, 22, 25, 27; Location – 4, 8, 12, 17, 23, 26. The scores of items 8, 17, and 23 are subject to reversal (1 = 5, 2 = 4, 4 = 2, 5 = 1). All the remaining analyses of psychometric properties were made with this version.

The study of associations between the scores of the PTOPS and the ones observed in the independent questions demonstrated the existence of relations of significant value, which was positive with the Enhancer and Location dimensions and negative with the Detractor dimension (Table 3).

 

 

Reliability

The Cronbach’s α coefficient for the total of the items was 0.73. The values for the subscales were 0.66 in the Enhancer dimension, 0.96 in Detractor dimension, and 0.84 in the Location dimension.

Regarding the reproducibility of the PTOPS, the results obtained through the ICC were 0.84 for the Enhancer, 0.87 for the Detractor, and 0.86 for the Location component.

 

Discussion

Our final sample was composed of 76 subjects, mostly female (52.6%), with qualifications equal to or less than basic education (55.3%) and a mean age of 45.43 years, covering physical therapy care in both public and private sectors. These characteristics are overlapping the ones of the Portuguese population that normally uses healthcare in general and physical therapy in particular (²² , ²³ ).

The methodology followed in the translation of the PTOPS for the Portuguese people and culture followed the most often recommended criteria in the literature, including those described by Hunt (¹⁴ , ¹⁵ ), Meadows et al. (¹¹ , ¹² ), and Beaton et al. ¹³.

By the methodology adopted and the results obtained, we believe that the items in the translated version have the identical meaning as the original measure, with existing congruence in the meaning of concepts between the English and Portuguese culture, pointing to the fact that the Portuguese version of the PTOPS holds functional equivalence. In our opinion, the use of panels, experts, and common people for the analysis of acceptance and understanding is also a suitable methodology.

Panelists felt that the instrument did not present any problems as to its clarity and that there were no redundancies or lack of relevant questions. The only problems reported in the acceptance of the pre-final version concerned the items related to the Costs dimension of the measure, as they are considered to be inappropriate for the Portuguese reality. In fact, models of financing care in which the user pays nothing or almost nothing make irrelevant the weight that the price factor has on the satisfaction with care.

This issue was subsequently confirmed in the validation tests and the items in question were removed. Thus, the results suggest that the Portuguese version of the PTOPS has an acceptable level of content validity. All recommended changes were forwarded to the authors of the original measure.

For the study of construct validity, we used factor analysis. Thus, items 4, 9, 16, 17, 25, and 30 of the original measure were identified as indeterminate relating mainly to the Costs dimension. After eliminating these items, the 28 remaining items were naturally distributed in a model of 3 components, corresponding, in essence, to the dimensions Enhancer, Detractor, and Location, originally defined in the model of Roush and Sonstroem ⁹.

Thus, these results are quite revealing of acceptable levels of construct validity of the Portuguese version of the PTOPS.

On the other hand, the study of the relations of the values obtained in the Portuguese version of the PTOPS with the independent questions (I would recommend these services to family or friends; I would come back to this service if I needed physical therapy treatment again; In general I am satisfied with my experience of physical therapy), showed results as expected. In other words, direct relations with the positive dimensions (Enhancer and Location) and inverse relations with the negative dimension (Detractor) were obtained.

The obtained values of correlation, although weak to moderate, are nevertheless acceptable. Actually, much higher values were not expected because, while these questions are considered some of the most discriminating in terms of satisfaction (¹⁰ -²⁴ ), they do not cover all of the multidimensionality of the construct in question.

Thus, these results support the inference withdrawal in terms of the validity of the Portuguese version of this instrument of patient satisfaction with ambulatory physical therapy care.

The results also show that the Portuguese version of the PTOPS presents equally acceptable values of reliability, both in regard to internal consistency and reproducibility.

Values of Cronbach’s alpha higher than 0.70, for the entire scale, are suggestive of acceptable levels of the internal consistency of the Portuguese version of the PTOPS. Although the original study does not refer to values related to the test-retest reliability, the ICC values found for the adapted version of the PTOPS were always greater than or equal to 0.7 ²⁰, representing acceptable levels of intertemporal stability.

Some limitations of this study should be acknowledged. The sample used is not representative of the entire population of Portuguese physical therapy outpatients. Further validation in large samples is therefore advised.

In conclusion, in view of the obtained results, the PTOPS is semantically equivalent to the original measure and has acceptable levels of validity and reliability. Therefore and because this measure is useful for physical therapy, its use is recommended in both clinical practice and research.

 

Acknowledgements

The authors would like to thank the physical therapy staff from the healthcare institutions. Also, the patients who participated in this study deserve our deep recognition.

 

Disclosure Statement

The authors report no conflicts of interest.

 

Received: August 2, 2016

Accepted: November 27, 2017

 

REFERENCES

¹ Gassée JP, Dehon M, Meiresonne A: Les plaintes de patient constituent-elles un indicateur utile pour l’amelioration de la qualité des soins? Gestions Hospitalières 1996; 356: 389–392.

² Brown K, Sheehan E, Sawyer M, Raftos J, Smyth V: Parent satisfaction with services in an emergency department located at a paediatric teaching hospital. J Paediatr Child Health 1995; 31: 435–439.10.1111/j.1440-1754.1995.tb00853.x8554865

³ Schulmeister L, Quiett K, Mayer K: Quality of life, quality of care, and patient satisfaction: perceptions of patients undergoing outpatient autologous stem cell transplantation. Oncol Nurs Forum 2005; 32: 57–67.10.1188/05.ONF.57-6715660144

⁴ Donabedian A: An Introduction to Quality Assurance in Health Care. Oxford, Oxford University Press, 2003.         [ Links ]

⁵ Beattie PF, Pinto MB, Nelson MK, Nelson R: Patient satisfaction with outpatient physical therapy: instrument validation. Phys Ther 2002; 82: 557–565.12036397

⁶ Keith RA: Patient satisfaction and rehabilitation services. Arch Phys Med Rehabil 1998; 79: 1122–1128.10.1016/S0003-9993(98)90182-4 9749695

⁷ Monnin D, Perneger TV: Scale to measure patient satisfaction with physical therapy. Phys Ther 2002; 82: 682–691.12088465

⁸ Arnetz JE, Almin I, Bergström K, Frazén Y, Nilsson H: Active patient involvement in the establishment of physical therapy goals: effects on treatment outcome and quality of care. Adv Physiother 2004; 6: 50–69.10.1080/14038190310017147

⁹ Roush SE, Sonstroem RJ: Development of the physical therapy outpatient satisfaction survey (PTOPS). Phys Ther 1999; 79: 159–170.10029056

¹⁰ Goldstein MS, Elliott SD, Guccione AA: The development of an instrument to measure satisfaction with physical therapy. Phys Ther 2000; 80: 853–863.10960933

¹¹ Meadows K, Bentzen N, Touw-Otten F: Cross-cultural issues: an outline of the important principles in establishing cross-cultural validity in health outcome assessment; in Hutchinson A, Bentzen N, Konig-Zahn C (eds): Cross Cultural Health Outcome Assessment: A User’s Guide. Ruinner, ERGHO, 1997, pp 34–40.

¹² Meadows K: Cross-cultural issues in the development and use of health outcome measures; in Long AF, Blitzer E (eds): Health Outcomes and Evaluation: Context, Concepts and Successful Applications. Leeds, University Print Services, 1997, pp 67–78.

¹³ Beaton DE, Bombardier C, Guillemin F, Ferraz MB: Guidelines for the process of cross-cultural adaptation of self-report measures. Spine 2000; 25: 3186–3191.10.1097/00007632-200012150-0001411124735

¹⁴ Hunt SM, Alonso J, Bucquet D, Niero M, Wiklund I, McKenna S: Cross-cultural adaptation of health measures. European Group for Health Management and Quality of Life Assessment. Health Policy 1991; 19: 33–44.10.1016/0168-8510(91)90072-610117390

¹⁵ Hunt SM: Cross-cultural issues in the use of socio-medical indicators. Health Policy 1986; 6: 149–158.10.1016/0168-8510(86)90004-710277126

¹⁶ Hill MM, Hill A: Investigação por questionário. Lisbon, Edições Sílabo, 2002.         [ Links ]

¹⁷ Bryman A, Cramer D: Análise de dados em ciências sociais: introdução às técnicas utilizando o SPSS para Windows. Oeiras, Celta Editora, 2003.         [ Links ]

¹⁸ Pestana MH, Gageiro JN: Análise de dados para ciências sociais: a complementaridade do SPSS. Lisbon, Edições Sílabo, 2005.         [ Links ]

¹⁹ Cohen L, Holliday M: Statistics for Social Scientists. London, Harper and Row, 1982.         [ Links ]

²⁰ Aaronson N, Alonso J, Burnam A, Lohr KN, Patrick DL, Perrin E, et al: Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res 2002; 11: 193–205.12074258

²¹ Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al: Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007; 60: 34–42.10.1016/j.jclinepi.2006.03.01217161752

²² Portugal, Ministério da Saúde, Instituto Nacional de Saúde: Inquérito Nacional de Saúde: Continente, dados gerais – 98/99. Lisbon, Observatório Nacional de Saúde, 1999.

²³ Portugal, Ministério da Saúde, Direção Geral da Saúde: Elementos estatísticos: informação geral: saúde 2002. Lisbon, Direção Geral da Saúde, 2005.

²⁴Labran C: Quels indicateurs de qualité pour les établissements hospitaliers?: une étude internationale. Gestions Hospitalières 1996; 356: 363–369.