Participants

This was a cross-sectional study conducted on the general Portuguese population. The data was collected through online surveys from April 5 to May 4, 2021, immediately after the end of the second period of confinement due to COVID-19 (January 22 to April 4). Recruitment was performed through email and social media advertisement, using the snowball technique, and the questionnaires were made available using the Google Forms platform.

Only respondents residing in Portugal with 18 or more years of age were considered. The study received approval from the university’s ethics board (Atlântica Ethics Commission, references ATLÂNTICA/CE/Processo nº01/2021 and nº02/2021) and an informed consent was obtained from all participants before answering the questionnaire.

Measures

The Impact of Event Scale-Revised was part of two questionnaires used to assess possible associations between the psychological impact of the confinement and food behaviour, namely food addiction, administered from April 5 to 19, 2021 (^{dos Santos, 2021}), and food choices, applied from April 19 to May 4, 2021 (^{Simões, 2021}). In addition to the specific scales to evaluate food behaviours and the IES-R, the questionnaires also contained sections of general sociodemographic questions and some questions directly concerning COVID-19.

The IES-R is a 22-item self-reported questionnaire that is used to assess symptoms of post-traumatic stress disorder (PTSD), based on the criteria of the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (American Psychiatric Association, 1994). The IES-R is divided into three subscales: Intrusion (8 items), Avoidance (8 items), and Hyperarousal (6 items) (^{Weiss, 2004}). Each item is rated on a 5-point Likert scale, ranging from 0 (Not at all) to 4 (Extremely). The total score on the IES-R can range from 0 to 88, with higher scores indicating greater risk of PTSD symptoms. The scale is not a diagnostic tool, but is a reliable and valid measure of PTSD symptoms. The subscale scores are presented as the mean of the responses and may range from 0 to 4 for each construct (Weiss, 2004).

IES-R has been translated into Portuguese and used in a variety of settings, always showing good reliability (^{Cunha et al., 2017}; ^{Paulino et al., 2021}; ^{Pedras et al., 2019}; ^{Vieira et al., 2020}). The translation used in the current study (Appendix) was taken from ^{Monteiro (2011}).

Data analysis procedures

Data was extracted from Google Forms and saved in an MS Excel file. General characteristics of the respondents were analysed by descriptive statistics and presented as percentages. IES-R total scores were calculated by adding the answers to all questions; scores for the Intrusion, Avoidance and Hyperarousal dimensions were obtained as means of the items comprising each subscale.

Skewness, kurtosis, and Shapiro-Wilk values were used to evaluate the normal distribution of the data. One-way ANOVA was used to search possible associations between sociodemographic characteristics and IES-R scores. Internal consistency of IES-R was determined using Cronbach’s alpha, with 0.7 considered the cut-off value (^{Cho & Kim, 2015}).

Construct validity of the IES-R scale, applied to the two samples, was evaluated using principal component analysis (PCA), exploratory factor analysis (EFA), and confirmatory factor analysis (CFA). EFA was performed using the minimum residual solution and oblimin and promax rotations. The Kaiser-Meyer-Olkin (KMO) test and the Bartlett’s test of sphericity were applied to verify the adequacy of the sample to perform factor analysis. Items with loading values greater than 0.3 were considered appropriate, and those with communalities of less than 0.2 were excluded from EFA. The Root Mean Square of Error Approximation (RMSEA, ≤ 0.10), Tucker-Lewis Index (TLI, ≥ 0.90), and Standardized Root Mean Square Residual (SRMR, ≤ 0.08) were used to select among models in EFA (^{Hair et al., 2018}). The Pearson correlation coefficient was used to investigate the extent of correlation for factor scores. The different domains should not be very highly correlated, as an indication that the subscales measured different psychological attributes (^{Craparo et al., 2013}).

CFA was conducted using the maximum-likelihood method, and the RMSEA, TLI, and Comparative Fit Index (CFI, ≥ 0.90) indices were used to assess goodness of fit (^{Hair et al., 2018}; ^{Kline, 2016}).

Data was analysed with a 95 % confidence level, using R 4.2.1 (R Core Team, 2022). The packages psych (^{Revelle, 2022}) and lavaan (^{Rosseel, 2012}) were used to perform factor analysis.

Sociodemographic variables

Two online questionnaires were administered, both containing sociodemographic questions and the items from the IES-R scale. The first sample, collected between April 5 and 19, 2021, consisted of 338 individuals, and the second sample, collected between April 19 and May 4, 2021, had 207 respondents. According to the literature, to statistically test the reliability and validity of a tool, the sample size should be at least five times the number of items, which in the present case is 22 × 5 = 110 (^{Tabachnick & Fidell, 2018}). Additionally, to perform factor analysis, the sample size should be at least 200 (^{MacCallum et al., 1999}). Therefore, both samples met the criteria.

Respondents were predominantly female in both samples (Table 1). The skewness, kurtosis, and Shapiro-Wilk test values indicated that neither sample presented a normal distribution. However, the sample sizes were large enough to justify the use of parametric statistical tests.

Impact of Event Scale-Revised Scores

The IES-R scoring criteria have been applied slightly differently by several authors. This study adopted the division of the score into three categories: 24 or more points indicates a mild psychological impact; 33 or more points is considered the cut-off corresponding to moderate impact and a high risk of PTSD diagnosis; more than 37 points indicates changes to the immune system (^{Craparo et al., 2013}). The first sample had an average score of 20.5 (± 15.6), while the second sample had a score of 20.7 (± 14.9). However, 70 individuals (20.1 %) in sample 1 and 41 respondents (19.8 %) in sample 2 had scores of 33 or higher. A one-way ANOVA showed a significant association between high scores on the IES-R and isolation, or quarantine situations experienced by members of both samples.

The average scores for each of the three subscales are presented in Table 2. All values were between 0.9 and 1.0, indicating a low score for every construct and no substantial differences between them.

Reliability of the IES-R Scale

Cronbach’s alpha was used to assess the internal consistency of the scale applied to the two samples, and yielded good values on both: 0.95 on the first and 0.94 on the second. The values calculated for the three subscales were also acceptable: 0.91 for Intrusion in sample 1 and 0.88 in sample 2; 0.85 for Avoidance in the first sample and 0.87 in the second; 0.87 for Hyperarousal in sample 1 and 0.86 in sample 2.

Principal Component Analysis

Bartlett’s test of sphericity was significant in both samples (p < 0.001) indicating a significant correlation between items. The KMO measure of sampling adequacy was 0.96 (all items above 0.90) for sample 1 and 0.93 (all items > 0.84) for sample 2, meaning that the items shared sufficient variance to justify factor analysis (^{Hair et al., 2018}). Pearson correlations between the three subscales yielded significant but not very high values, suggesting that they measure different dimensions. Higher correlation values were found between Intrusion and Hyperarousal items in both samples. The existence of correlation between factors supported the use of oblique rotation procedures (promax criterion).

Performing a PCA on both samples and analysing the respective scree plots, as well as applying the Kaiser criterion of retaining only components whose eigenvalues were greater than 1.0, gave an indication of the existence of three components for both samples. These three components explained 61.0 % and 60.5 % of the total variance, respectively in sample 1 and sample 2. As the first components accounted for 50.2 % and 46.6 % of total variance, the possibility of a unidimensional scale was also considered. In fact, the one-dimensionality test of the psych package (^{Revelle, 2022}) provided values of 0.94 and 0.91 for samples 1 and 2 respectively, which are considered high.

Thus, it was decided to test the validity of solutions with several factor structures by means of exploratory factor analysis.

Exploratory Factor Analysis

Minimum residual exploratory factor analysis was performed with oblimin or promax rotations to test one- to four-factor solutions. The results obtained with both rotation methods were very similar. Using promax rotation, a 3-factor solution provided better fit indices (SRMR = 0.03, TLI = 0.91, RMSEA = 0.07) in sample 1, while sample 2 suggested a better fit with a 4-factor solution (SRMR = 0.03, TLI = 0.91, RMSEA = 0.07).

In sample 1, the correlations between factors in the 3-factor model were below 0.7. However, questions 5, 7, 17, and 21 (see Appendix) had a complexity of around 2 and similar loads in two of the factors. Moreover, no item loaded on factor 3. If questions 5, 7, and 17 (Avoidance items on the original scale) and 21 (Hyperarousal) were not retained, a 3-factor structure (Figure 1, left), with only items 2 and 15 loading on factor 3, presented a slightly better fit than the one with 2 factors (SRMR = 0.03, TLI = 0.92, RMSEA = 0.07).

Figure 1 EFA Factor Structure of IES-R. Left: Sample 1 (items 5, 7, 17, and 21 excluded); Right: Sample 2 (items 5, 9, 18, and 21 excluded). 

For sample 2, correlations between factors were also below 0.7. Questions 5, 9, 18, and 21 had complexities around 2 and questions 1, 5, 9, 18, and 21 had similar loads on two factors. Removing items 5 (Avoidance), 9 (Intrusion), 18, and 21 (Hyperarousal), EFA delivered a better fit for a 4-factor solution (Figure 1, right), with slightly better indices than the ones found for the 22-item scale (SRMR = 0.03, TLI = 0.93, RMSEA = 0.07).

The retained items, in both samples, produced good communality and uniqueness values and satisfactory loadings on a single dimension (Table 3). Correlations between factors and between factors and the observed data (Table 4) were all significant (p < 0.05). The values of correlation between factors suggest that, in both samples, each dimension measured a different construct. On the other hand, the reasonably high correlations found are in accordance with the low level of symptoms determined in both samples.

Confirmatory Factor Analysis

Maximum likelihood CFA was performed on the data from both samples, based on the results provided by EFA. For sample 1, three models were tested, all with correlated factors: 1) two factors with all 22 items; 2) two factors without items 5, 7, 17, and 21; 3) three factors without items 5, 7, 17, and 21. Model 3 (Figure 2, top) with three factors provided the best fit, with CFI = 0.94, TLI = 0.93, and RMSEA = 0.07. Considering the EFA results, models with three and four factors were evaluated for sample 2, excluding items 5, 9, 18, and 21. The four-factor model (Figure 2, bottom) provided the best fit, with CFI = 0.92, TLI = 0.91, and RMSEA = 0.08.

Figure 2 CFA Factor Structure of IES-R. Top: Sample 1 (items 5, 7, 17, and 21 excluded); Bottom: Sample 2 (items 5, 9, 18, and 21 excluded). 

Psychological disturbances during the COVID-19 confinement

The two samples of respondents could not be considered representative of the Portuguese population, as they presented considerable differences in sex, age, and education (Pordata, 2021). Thus, the scores obtained on the IES-R are to be interpreted as indicative only.

Most respondents presented low scores on the whole scale and also on the three subscales, suggesting that the confinement due to COVID-19 was not a very traumatic event for them. However, both samples presented approximately 20 % of individuals with total scores of at least 33, indicating a moderate risk of a PTSD diagnosis. Since this is a cross-sectional study, no causality with the confinement could be ascertained. However, other authors have already mentioned the existence of stress-related disorders during the pandemic (^{Ifthikar et al., 2021}; ^{Khan et al., 2020}; ^{Zhang & Ma, 2020}). Moreover, the association found, by means of a one-way ANOVA, between isolation or quarantine periods suffered by the respondents and higher scores on the IES-R, suggests such a causal relationship.

In line with previous results, no significant association was found between psychological impact and age (^{Paulino et al., 2021}; ^{Vieira et al., 2020}; ^{Zhang & Ma, 2020}). Moreover, contrary to other studies that have found females to be more susceptible to developing PTSD when exposed to traumatic events than men (^{Ifthikar et al., 2021}; Paulino et al., 2021), no significant association was observed between IES-R score and sex. However, this result may be masked due to an overrepresentation of female respondents.

Reliability and factor structure of the IES-R

Similarly to other studies (^{Craparo et al., 2013}; ^{Nia et al., 2021}; ^{Park et al., 2021}; ^{Paulino et al., 2021}), the scale showed good internal consistency, with Cronbach’s alpha values ranging from 0.85 to 0.91 for the three constructs and 0.94, and 0.95 for the whole scale in both samples. Thus, the IES-R can be regarded as a reliable tool to measure the psychological impact of the confinement due to COVID-19.

After performing EFA and CFA, the best fits for the data collected with the two samples provided different factor structures for the IES-R: a 3-factor structure for sample 1 and a 4-factor structure for sample 2. In both cases, the model structures only attained good fits after removing some items from the original questionnaire. In sample 1, questions 5, 7, 17 (Avoidance on the original scale), and 21 (Hyperarousal) were eliminated; and in sample 2, questions 5 (Avoidance), 9 (Intrusion), 18, and 21 (Hyperarousal) were excluded.

Item 5, which was removed in both samples to achieve better fits for the factor structures, was originally classified as an Avoidance item (^{Weiss, 2004}). It was also excluded by other authors (^{Craparo et al., 2013}; ^{Vieira et al., 2020}) or classified as Intrusion (^{Park et al., 2021}). The initial part of the phrase may be associated with Avoidance, but the latter part, “…reminded of the event” (Appendix), may be perceived as Intrusion by some respondents, thus affecting the factor structure.

Item 12, associated with the Avoidance construct on the original work, has a fairly complex internal psychological process concept. It may not be easily comprehensible to all respondents and has been associated with both Intrusion and Hyperarousal items throughout (^{Craparo et al., 2013}; ^{Weiss, 2004}). In the present work, it loads on the first factor, for both samples, together with Intrusion and a few Hyperarousal items.

Item 19, starting with “Reminders of it…”, also loaded together with most Intrusion items, not into a Hyperarousal factor, as in the original structure. In both samples, the average score on this item was rather low (0.5), indicating that noticeable physical reactions (sweating, etc.) were low among the respondents.

After the above-mentioned items were excluded, the results of the EFA explained 57 % of the total variance in sample 1 and 59 % in sample 2. This was in line with the 59.22 % obtained in a study also related to the COVID-19 pandemic in Iran (^{Nia et al., 2021}). In sample 1, the first factor consisted of 12 items, the second contained 4 items, and the third only 2 items. The first factor consisted of 7 items originally identified as belonging to the Intrusion construct (^{Weiss, 2004}): items 4 and 19, originally associated with Hyperarousal, but also identified as Intrusion by ^{Lim et al. (2009}); items 10 and 18 (originally classified as Hyperarousal); and item 12, usually classified as Avoidance, but also as Intrusion and Hyperarousal (^{Craparo et al., 2013}). This factor was thus called Intrusion (Table 3).

Factor two only consisted of items typically associated with the Avoidance construct and was designated as such. The third factor contained only items 2 and 15, both associated with sleep, and was termed Sleep disturbance. The mixing of items originally identified as Intrusion and Hyperarousal in a single factor may be further supported by the high Pearson correlation values found (0.88 for sample 1 and 0.85 for sample 2) between those two subscales if their original items were considered. Comparable combinations of items were also found by other authors (^{Asukai et al., 2002}; ^{Báguena Puigcerver et al., 2001}; ^{Brunet et al., 2003}), and a very similar factor structure was previously reported when studying traumatic events on a sample of police officers (^{Weiss, 2004}).

For sample 2 (Table 3), the first of the four factors found comprised six items, with three usually associated with the Intrusion construct: item 19, identified as Hyperarousal or Intrusion (see above); item 10 (originally Hyperarousal); and item 12, classified originally as Avoidance. To be consistent with the choice for sample 1, this factor was called Intrusion. The second factor consisted of six of the items originally associated with Avoidance, thus maintaining that designation. Factor three contained items 1, 3, and 6, usually classified as Intrusion, and item 4, identified as Hyperarousal or Intrusion (see above). Those items concerned sensations or symptoms caused by remembering the event, and this factor was thus designated as Hyperarousal. Finally, factor four consisted of only items 2 and 15, as with sample 1, and was also named Sleep disturbance.

An autonomous Hyperarousal factor was not statistically confirmed on one of the samples and is not clearly defined on the other. Its items loaded together with Intrusion items, as already shown in other studies (^{Park et al., 2021}; ^{Vieira et al., 2020}). It seems that there is some shared perception between intrusive thoughts and physiologic reactions, since both can be felt as involuntary and uncontrollable experiences (Vieira et al., 2020). It was also suggested that questions starting with “I felt” may be perceived as symptoms, influencing the results of factor analysis (Park et al., 2021). Items 2 and 15, both related to sleep, were classified into an independent factor, Sleep disturbance, the existence of which was already corroborated by other authors (^{Eun et al., 2005}; Park et al., 2021; ^{Weiss, 2004}).

Sleep disturbance in healthcare workers associated with the COVID-19 pandemic have been reported (^{Marvaldi et al., 2021}; ^{Park et al., 2021}), and since the two samples participating in the present study contained approximately 30 % of individuals with health-related education, our results may be further indication that the characteristics of traumatic symptoms reflect occupational backgrounds.

It has been previously stated that the time elapsed between responding to the questionnaire and the traumatic event may affect the quality of the answers (^{Horowitz et al., 1979}; ^{Weiss, 2004}). Therefore, the 2-week interval between data collection for the two samples may also have impacted on the different structures found.

Furthermore, the difficulty in establishing a clear factor structure for the IES-R may reflect the absence of a direct connection between the items of the scale and the DSM-5 (American Psychiatric Association, 2013) diagnostic criteria for PTSD.

The inconsistencies showed here, as in other studies, regarding the factor structure of the IES-R, may be due, in part, to the nature of the samples and the particular traumatic event considered. The low score presented by most participants, meaning low symptomatology, is usually associated with a unidimensional structure and higher subscale intercorrelations (^{Weiss, 2004}), which was not found with either of the samples studied here. Researchers need to carefully consider the characteristics of the sample (occupational and cultural backgrounds) when applying and interpreting the results of the IES-R (^{Park et al., 2021}).

However, the present study, which was the first to apply the IES-R scale in Portugal to people who underwent a long confinement due to a pandemic, showed that the Portuguese version of the scale has good psychometric properties and could be used as a tool to measure psychological distress associated with traumatic situations like the COVID-19 pandemic. It could also assist in the definition of early interventions to deal with such distress.