INTRODUCTION
Worldwide, suicide takes the lives of more than 800.000 persons annually (World Health Organization, 2021). In Portugal, 1061 persons died by suicide in 2017, representing 1% of the country's mortality (National Institute of Statistics, 2019). Understanding the factors related to suicide risk and assessing those using sound psychometric measures is an important clinical task goal. Recent reviews (Ducasse et al., 2018; Verrocchio et al., 2016) have emphasized the role of psychological pain in understanding suicidality. Further, in his model of suicide, Shneidman (1993) proposed that suicide is caused by unbearable psychological pain that he termed psychache. Psychache is a state of deep anguish resulting from frustration of basic psychological needs; it is a more primal, savage mental pain than occurs in general distress or depression, and escaping from this pain is an important motivation for attempting suicide (Holden et al., 1998). Although conceptually overlapping with depression and hopelessness, psychache is a factor analytically distinct from these other constructs (Troister & Holden, 2013).
Ducasse et al. (2018) indicate that higher levels of psychological pain are linked both to suicidal ideation and to suicidal actions. The added value of assessing psychological pain relative to risk indicators such as depression to predict suicidal behaviors has been demonstrated (Berlim et al., 2003; Demirkol et al., 2019). Further, mental pain assessment has outperformed the assessment of depression and hopelessness for screening suicide risk (Troister et al., 2015). Mental pain has also been found to fully and partially mediate the links between general distress and suicide ideation in a community sample (Campos et al., 2016). Furthermore, mental pain has also been shown to statistically predict suicide attempts (Leenaars & Lester, 2005) and ideation (Flynn & Holden, 2007) among students.
In asserting that psychological pain is a core clinical feature for suicide, Verrocchio et al. (2016) advocate for routinely evaluating mental pain when suicide risk is clinically assessed. Evaluation of psychological pain as a suicide indicator also has merit when assessing clients who mask (Shneidman, 1994) or who need to appear flawless to others (Roxborough et al., 2012). With this accumulating evidence, tools for assessing mental pain continue to develop and require verification of their structure and scale properties. As such, the current investigation evaluated a promising Chinese measure of psychological pain for its structural integrity in Western populations.
Instruments for assessing psychological pain are available, including the Psychache Scale (Holden et al., 2001) and the Mee-Bunney Psychological Pain Assessment Scale (Mee et al., 2011). More recently, Li et al. (2017) developed the Three-Dimensional Psychological Pain Scale (TDPPS) that, unlike other psychological pain scales that are unidimensional, attempts to differentiate psychological pain into affective, cognitive, and behavioral facets, based on the Li et al. (2014) three‐dimensional psychological pain model of suicide. According to this model, the affective dimension involves subjective and bodily symptoms that represent painful feelings. The cognitive dimension relates to memories of past traumas such as loss, frustration, and social exclusion. The avoidance component is the tendency to view suicide as the means to escape from psychological pain, and it is hypothesized to be the most important predictor of suicidal motivation. Overall, the Li et al. (2014, 2017) model attempts to partition more finely the construct of psychological pain than has been done previously by more unidimensional measures that focus primarily on affect. To date, only a few studies have examined this three-dimensional pain model using the 17-item TDPPS. Research has shown that pain avoidance (i.e., the behavioral scale of the TDPPS) may be a primary motivator for suicidal behaviors, even in the absence of depression (Li et al., 2014; Xie et al., 2014). The pain avoidance subscale has displayed superior performance in accurately identifying suicide attempters among patients with major depressive disorder compared to measures of impulsivity, depression, hopelessness, psychache, and the acquired capability for suicide (Sun et al., 2020).
Importantly, using the same measure (i.e., the TDPPS), this finding has been replicated but just in a few studies in Western societies (Campos, Holden, et al., 2019; Campos, Simões, et al., 2019). Although the TDPPS, in general, and its Pain Avoidance scale, in particular, have demonstrated merit, the TDPPS factor structure has not been well established, particularly outside of its Chinese origins. To our knowledge, the only structural analysis of the TDPPS items has been that of Li et al. (2017), who, for a sample of 1,185 Chinese undergraduate students responding to the Chinese version of the TDPPS, reported that an exploratory factor analysis yielded three components explaining 55.5% of the total variance and that a confirmatory factor analysis demonstrated adequate fit for the hypothesized three-factor model.
As such, the purpose of the present study was to investigate the structure of the TDPPS in Western samples and with non-Chinese versions of the measure. Testing and replicating the structure of the measure could support its use with Western populations. It would also allow for evaluating the Li et al. (2014) three-dimensional model and, thus, contribute to clarifying the dimensionality of psychological pain as a construct to be assessed in practice and research. We hypothesized that a 3-factor solution would replicate across samples, that scores for the three reliable dimensions would relate with another psychological pain measure, the Psychache Scale, and a related but distinct construct, depression. We also hypothesized that all three dimensions, especially pain avoidance, would explain significant, unique variance in suicidal behaviors.
METHOD
Participants
The examination of the TDPPS used six distinct data sets. Sample 1 consisted of 331 undergraduate volunteers (186 women, 145 men) at a midsize Portuguese university (> 95% Caucasian) who volunteered and were not compensated. Mean age was 19.89 years (SD = 1.74, Median = 20). Sample 2 included 232 Portuguese community participants (148 women, 84 men) recruited to participate in a study of psychological variables, suicidal ideation, and risk behaviors. They were not compensated. Participants had a mean age of 34.98 years (SD = 11.80, Median = 32). Sample 3 consisted of 384 undergraduate volunteers (193 women, 191 men) at a midsize Portuguese university (> 95% Caucasian) who were not compensated. These participants had a mean age of 19.62 years (SD = 2.20, Median = 19). Sample 4 included 228 students (204 women, 23 men, 1 unreported) at a midsize Canadian university (80% Caucasian) who received psychology course credit for participation. Mean age was 18.36 years (SD = 1.12, Median = 18). Sample 5 comprised 247 undergraduates (219 women, 28 men) at a midsize Canadian university (> 80% Caucasian) with a mean age of 20.06 years (SD = 5.69; Median = 18) who, for participation, received introductory psychology course credit. Sample 6 included 207 American community participants (136 men, 70 women, 1 unreported) recruited through Amazon’s Mechanical Turk and were paid 5.00 USD to participate in a study on personality, motivations, life history, and suicide ideation. The mean age of these participants was 33.37 years (SD = 8.76, Median = 31).
Measures
Three-Dimensional Psychological Pain Scale (TDPPS). The TDPPS (Li et al., 2014) is a 17-item inventory assessing psychological pain within three facets: cognitive (e.g., “I feel rejected and misunderstood by people around me, which is the cause of my pain”), affective (e.g., “The pain I feel is in my mind and is much more severe than any physical pain”), and avoidance (e.g., “My pain hurts so badly that death could be the only way to escape from it”). Items are answered on 5-point Likert ratings ranging from 1 - “not at all” to 5 - “extremely so.” Although the development of the TDPPS has not been fully published, Li et al. (2017) have reported TDPPS scale score coefficient alpha reliabilities over .76. They have shown that scale scores can differentiate among suicide ideators, major depressive disorder patients, and healthy controls. As elsewhere detailed (Campos, Holden, et al., & 2019; Campos, Simões, et al., 2019), the TDDPS was translated to Portuguese from an English version sent by authors of the original Chinese version. Several clinical psychologists and a bilingual translator, who performed a back-translation, participated in the translation process.
Psychache Scale. The Psychache Scale (Holden et al., 2001) is a 13-item measure of psychological pain (e.g., “My soul aches”) answered on 5-point ratings varying from 1 (never or strongly disagree) to 5 (always or strongly agree). Scale scores have strong reliability in samples of university (α = .94; Troister & Holden, 2010) and offender (α = .95; Mills et al., 2005) participants. Validity for scale scores has been shown through correlations with indices of suicide ideation (r = .65), suicide attempts (r = .45), likelihood of future suicide commission (r =.33), and self-injury (r = .50) (Holden et al., 2001). Samples 1, 2, and 3 responded to the scale's Portuguese version (Campos et al., 2018). Results with the Portuguese version supported the unidimensional scoring of the scale, its ability to differentiate between individuals at-risk for suicide from individuals not at-risk, its relationship with different but related constructs, and its ability to predict suicide ideation statistically. Alpha reliability coefficients in the current samples were .90, .94, .93, .96, .96, and .98, respectively for the six samples.
Suicidal Behaviors Questionnaire-Revised (SBQ-R). The SBQ-R (Osman et al., 2001) includes four multiple-choice items (e.g., “Have you ever thought about or attempted to kill yourself”) assessing: a history of suicide ideation and/or attempts; recent suicide ideation (i.e., during the past year); the communication of suicidal intentions to others; and the likelihood of a future suicide attempt. Item 1 has six response options, item 2 has five response options, item 3 has five response options, and item 4 has seven response options. SBQ-R scale scores have acceptable internal consistency reliability across multiple samples (alpha reliability coefficients from .76 to .87). They have been effective in differentiating suicidal from non-suicidal individuals (Osman et al., 2001). Samples 1, 2, and 3 responded to the questionnaire's Portuguese version (Campos & Holden, 2019). Results have supported the scale internal consistency, its unidimensionality, and both concurrent and 5-month predictive validity. Alpha reliability coefficients in the current six samples were .75, .72, .82, .80, .82, and .82, respectively.
Center for Epidemiologic Studies Depression Scale (CES-D). The CES-D (Radloff, 1977) is a 20-item scale measuring the previous week’s frequency of depressive symptoms. Items (e.g., “I felt that everything I did was an effort”) are responded to on 4-point Likert ratings from 0 ("Never or very rarely - less than 1 day") to 3 ("Very frequently or always - 5-7 days"). Radloff (1977) has reported scale score coefficient alpha reliabilities of .85 and .90 for community and clinical samples, respectively. For scale score validity, Erford et al. (2016) found a .72 correlation between Beck Depression Inventory-II and CES-D scale scores across 11 studies. Samples 1, 2, and 3 responded to the scale's Portuguese version (Gonçalves & Fagulha, 2004). Coefficient alpha for the Portuguese version values have ranged from 0.85 to 0.92 in several Portuguese samples. Alpha reliability coefficients in the current study were .92, .91, .91, .92, .93, and .95, respectively, for the six samples.
Procedure
For confirmatory and exploratory analyses, this examination of the TDPPS re-analyzed six distinct data sets from Campos, Holden, et al. (2019), Campos, Holden, Spínola, et al. (2019), and Holden et al. (2020). Participants supplying data were all treated in accord with the ethical principles of the American Psychological Association. Data collection procedures were approved by the two institutional ethics research boards associated with the researchers’ universities and took approximately 2 to 3 weeks per sample.
All individuals provided informed consent. Samples 1 and 3 completed printed versions of questionnaires while samples 2, 4, 5, and 6 completed materials online. All participants received debriefing material regarding available counseling resources and including telephone numbers for participants wishing to contact a mental health professional.
Data Analytic Plan
Data analysis focused on eight steps:
1. A maximum likelihood confirmatory factor analysis (CFA) was computed for each sample. The model tested was the three-factor model proposed by Li et al. (2017). Sellbom and Tellegen (2019) indicate RMSEA and SRMR values of .06-.08 as acceptable fit with values < .06 as good fit, and CFI and TLI values of .90-.94 as acceptable fit with values of > .95 being good fit.
2. Because CFA can be regarded as overly stringent in not permitting items to cross-load (Marsh et al., 2014), the three-factor model in the first step was also tested using maximum likelihood exploratory structural equation modeling (ESEM). The ESEM model specified three factors but, unlike CFA, permitted non-zero cross-loadings.
3. If the model from the CFA or ESEM analyses didn’t fit adequately, principal components analysis was undertaken guided by parallel analysis (Zwick & Velicer, 1986) and the eigenvalue-greater-than-one heuristic for determining the number of dimensions. An orthogonal rotation of the solution was used to reduce intercorrelations among subsequently developed scales. This was done for two samples combined (Samples 1 and 4) to leave four independent samples for assessing cross-validation.
4. Item selection in developing scales maximized an item efficiency index (IEI; Holden, 1996; Jackson et al., 1989; Waring et al., 1998), again to reduce intercorrelations among developed scales:
IEIi = Item Efficiency Index for the ith item
Aih = is the ith item’s loading on its most salient component
Aic = is the ith item’s loading on the other component(s)
nc = the number of components
5. CFAs for each sample were undertaken based on items scored on the developed scales.
6. ESEMs for each sample were undertaken based on items scored on the developed scales.
7. Developed scales’ internal consistency and validity were evaluated. For validity, this was undertaken by correlating developed scales’ scores with measures of mental pain, suicidal behaviors, and depression.
8. The distinct contributions of developed scales were evaluated. This was tested by regressing suicidal behavior scores simultaneously onto scores on the developed scales and examining the significance of obtained regression coefficients.
RESULTS
Table 1 reports fit statistics for the CFAs. Whereas no RMSEA, CFI, or TLI value indicated good or acceptable fit, SRMR values indicated good fit (one sample) or acceptable fit (five samples) for the three-factor model. Although fit measures tended to improve, in some instances from poor to acceptable fit, with ESEM (also Table 1), overall, interpretations of good fit or lack thereof were similar to those found for the CFAs.
For each sample, the eigenvalue > 1 heuristic indicated two components. Based on parallel analysis (1,000 replications), two components were indicated for five (all Portuguese and Canadian) of the six samples, and one component was favored in the American sample. Given the convergence of 11 of 12 indicators for two dimensions, two components were extracted in a dataset combining Samples 1 and 4 (one Portuguese sample and one Canadian sample), rotated to a varimax criterion, and the IEI applied. Based on IEIs, four items were selected for each of the two scales. Scale 1 (TDPPS items 2, 9, 11, 12) is labeled Pain Escape, and Scale 2 (TDPPS items 4, 15, 16, 17) is labeled Pain Emotions.
Samle | Model | χ2 | df | χ2df | RMSEA | CFI | TLI | SRMR | AIC |
---|---|---|---|---|---|---|---|---|---|
Samle1 | CFA | 633.55 | 116 | 5.46 | .116 | .878 | .857 | .065 | 12993.784 |
Samle2 | CFA | 495.30 | 116 | 4.27 | .119 | .860 | .836 | .074 | 9291.321 |
Samle3 | CFA | 650.45 | 116 | 5.61 | .109 | .870 | .847 | .063 | 15874.993 |
Samle4 | CFA | 592.38 | 116 | 5.11 | .134 | .847 | .820 | .064 | 9183.174 |
Samle5 | CFA | 443.00 | 116 | 3.82 | .107 | .885 | .865 | .064 | 10961.066 |
Samle6 | CFA | 514.68 | 116 | 4.44 | .129 | .892 | .873 | .047 | 9186.802 |
Samle1 | ESEM | 328.55 | 88 | 3.73 | .091 | .943 | .913 | .028 | 12744.789 |
Samle2 | ESEM | 240.46 | 88 | 2.73 | .086 | .944 | .913 | .030 | 9092.487 |
Samle3 | ESEM | 360.72 | 88 | 4.10 | .090 | .934 | .897 | .033 | 15641.260 |
Samle4 | ESEM | 260.06 | 88 | 2.96 | .093 | .945 | .915 | .029 | 8906.847 |
Samle5 | ESEM | 245.52 | 88 | 2.79 | .085 | .944 | .914 | .031 | 10819.595 |
Samle6 | ESEM | 264.77 | 88 | 3.01 | .099 | .952 | .926 | .026 | 8992.895 |
For the eight items associated with the new scales, CFA and ESEM were applied to the two samples, separately, that were used in the principal components solution and application of the IEI to derive the new scales, and to the other four samples, separately, as cross-validation samples (Table 2).
Sample | Model | χ2 | df | χ2/df | RMSEA | CFI | TLI | SRMR | AIC |
---|---|---|---|---|---|---|---|---|---|
Sample 1 | CFA | 141.08 | 19 | 7.43 | .139 | .921 | .883 | .077 | 6258.489 |
Sample 2 | CFA | 55.13 | 19 | 2.90 | .091 | .965 | .948 | .063 | 4285.179 |
Sample 3 | CFA | 142.38 | 19 | 7.49 | .130 | .926 | .890 | .062 | 7234.352 |
Sample 4 | CFA | 103.29 | 19 | 5.44 | .139 | .928 | .893 | .059 | 4306.126 |
Sample 5 | CFA | 113.34 | 19 | 5.97 | .142 | .918 | .879 | .079 | 5095.847 |
Sample 6 | CFA | 61.77 | 19 | 3.25 | .105 | .967 | .951 | .040 | 4609.961 |
Sample 1 | ESEM | 39.74 | 13 | 3.06 | .079 | .983 | .963 | .022 | 6169.149 |
Sample 2 | ESEM | 16.72 | 13 | 1.29 | .035 | .996 | .992 | .016 | 4258.779 |
Sample 3 | ESEM | 64.14 | 13 | 4.03 | .101 | .969 | .933 | .027 | 7168.122 |
Sample 4 | ESEM | 44.13 | 13 | 3.39 | .102 | .973 | .942 | .024 | 4258.967 |
Sample 5 | ESEM | 57.03 | 13 | 4.39 | .118 | .962 | .917 | .030 | 5051.542 |
Sample 6 | ESEM | 27.02 | 13 | 2.08 | .073 | .989 | .976 | .022 | 4587.219 |
For the six CFAs, only one RMSEA indicated at least acceptable fit (< .10), all CFIs indicated acceptable fit (> .90) with two exceeding the .95 criterion for good fit, two TLIs met or exceeded the .95 criterion for good fit, and all SRMRs met the .08 or less criterion for a good fit. For the six ESEMs, one RMSEA met the .06 or less criterion for good fit and four met the criterion for acceptable fit (< .10), all CFIs met the criterion for a good fit, three TLIs met the criterion of .95 for a good fit with the other three meeting a .90 criterion for acceptable fit, and all SRMRs met the standard for a good fit. Of note, for each sample, the AIC value for the 8-item, 2-factor solution was approximately half (M = .48; Range = .46 to .51) the value associated with the corresponding 17-item, 3-factor solution. With results across criteria and samples generally providing acceptable support for a two-factor model for the eight items, ESEM was subsequently applied to all samples combined yielding fit statistics: χ 2 = 143.15, χ 2/df = 11.01, RMSEA = .078 [90% CI .067, .090], CFI = .985, TLI = .967; SRMR = .017, AIC = 32412.903. The standardized solution and associated TDPPS items are reported in Table 3, and the two factors correlated .53, p < .01.
For the derived scales of Pain Escape and Pain Emotions, Table 4 reports scale score properties. For internal consistency reliability, all Cronbach α’s exceeded .83, all McDonald ω’s surpassed .85, and all scale mean interitem correlations were greater than .50. For scale validity, correlations with another psychological pain measure, the Psychache Scale, varied between .36 and .89 (Median = .67). As statistical predictors of suicidal behavior, correlations with the SBQ-R ranged from .41 to .77 (Median = .58). Correlations with a related but distinct construct, depression, measured by the CES-D, varied between .38 and .78 (Median = .58).
Table 5 reports simultaneous regressions of SBQ-R scores onto scores of the new scales of Pain Avoidance and Pain Emotions. Across samples, R 2 values ranged from .24 to .61 and, within each sample, scores on each of the two predictors explained significant, unique variance in suicidal behavior.
DISCUSSION
The purpose of the present study was to investigate the factor structure of the TDPPS in Western samples. Testing the structure of the measure evaluates the potential merits of its use with Western populations. Also, it tests the three-dimensional model of psychological pain, thus contributing to the clarification of this construct's dimensionality. We hypothesized that a 3-factor solution would replicate across samples. The three dimensions' scores would demonstrate high internal consistency and relate to another psychological pain measure and a related but distinct construct, depression. We also hypothesized that the three dimensions, in particular pain avoidance, would explain significant, unique variance in suicidal behaviors.
Current analyses found that, in non-Chinese samples, the nature of the TDPPS did not conform to its hypothesized structure and proposed scoring key of affective, cognitive, and behavioral scales. Instead, a two-dimensional structure of pain escape and pain emotions emerged. Based on this solution, a new 8-item, two-scale scoring key is proposed. This revision has a structure that replicates across samples and provides scale scores that have promising psychometric properties. This 8-item revision could be a more efficient, stand-alone version in that it has less than half the items of the original TDPPS or, if preferred, it could represent an alternative scoring for the full 17-item TDPPS. Scale scores, based on the two-dimensional solution, are related to scores of another mental pain measure and scores for a distinct but related construct, depression. As hypothesized, the avoidance (escape) dimension significantly explained a greater proportion of variance in suicidal behaviors.
A question arises as to why the three-factor TDPPS structure is found in the original Chinese sample but not in the current, Western samples. A possible explanation may be the double-barreled nature of some items. For example, item 3, “The pain I’m feeling is nearly unexplainable and comes out of nowhere,” seems to address two issues, not one, and, as a result, could blur the underlying structure. Nevertheless, despite potential structural issues, non-Chinese versions of the TDPPS demonstrate substantial merit (e.g., Campos, Holden, et al., 2019; Campos, Simões, et al., 2019), indicating that the measure has strengths that are to be noted, and suggesting that a revised scoring system may be appropriate. Results also raise the question regarding whether psychological pain would be better conceptualized as a three-facet or a two-facet construct. Specifically, the affective and cognitive dimensions may be highly collinear and, practically, unamenable to assessing different facets of the same construct.
Potential limitations to our research exist. First, results are for measures translated among Chinese, Portuguese, and English languages. Although multiple bilingual psychologists and a back-translation by a bilingual translator were involved, translations can be fallible. Second, measures were self-report and relied on participants being able to and willing to provide honest answering. Third, our design was cross-sectional and limited the ability to draw causal inferences among the constructs studied. Fourth, participants were from nonclinical samples. Fifth, although similar samples were used, they may not be entirely comparable. Replication with other samples, other data collection modes (e.g., reports of significant others), longitudinal designs, and clinical patients will serve to establish the generalizability of current findings.
In conclusion, the observed Western data do not support the hypothesized three-factor structure of the TDPPS. Results suggest the TDPPS assesses two dimensions, pain escape, and pain emotions. Scales for these two dimensions can be scored from the TDPPS, and they demonstrate appropriate psychometric properties.
Assessing suicide risk is a complex but indispensable clinical task. Psychometrically sound measures such as the TDDPS are important tools whose psychometric properties (e.g., structure, reliability, validity) should be made available for relevant health professionals. In assessing the psychological experience of pain, the tendency to avoid pain may be crucial both in the initial clinical assessment of at-risk individuals and in their follow-up assessment. Evaluating psychological pain may even be relevant in assessing persons at risk for suicide even when they do not report significant suicide ideation. Further, assessing psychological pain may be more acceptable to clients than is inquiring directly about serious suicidal ideation or suicidal behavior and, as such, may address issues of concealment of suicidal urges that some individuals, such as perfectionists, may have (Flett et al., 2014).