Inglês (pdf)
Artigo em XML
Referências do artigo
Tradução automática
Enviar este artigo por email
Citado por SciELO 
Similares em
SciELO 
Permalink
Violence in intimate relationships is a severe global social issue, recognized by the World Health Organization as an undervalued and misunderstood public health matter (García-Moreno et al., 2005). Reflecting an unequal relationship, recurring violence carries deep consequences to physical and also psychological, emotional, sexual and social health and well-being (Banyard & Cross, 2008), which may go beyond the victim-perpetrator dyad, with implications for the whole community system (Debnam & Temple, 2021).
Most research on the prevalence of violence in intimate relationships reports high victimization rates of up to 45% in population of all ages (Smith et al., 2018). However, these data are highly variable, tending to be restricted to the United States, and focus mainly on severe physical violence and/or sexual violence (Tomaszewska & Schuster, 2021). These limitations suggest an underestimation of the prevalence of the phenomenon, which would drastically increase if studies were to include more subtle forms of violence (such as psychological, social…; Hickman et al., 2004).
The same applies to prevalence studies in youth. In the US, 71.1% of female victims of sexual violence, physical violence, and/or stalking by an intimate partner firstly experienced it before the age of 25, with the same being true for 55.8% of male victims (Smith et al., 2018). In Europe, a systematic review by Tomaszewska and Schuster (2021) shows victimization rates of psychological teen dating violence (TDV) ranging from 5.6% to 95.5%, and of physical TDV ranging from 0.8% to 32.9%, whereas perpetration rates range from 7% to 97% for psychological TDV, and from 2.1% to 46% for physical TDV. The reviews indicate that girls tend to report higher victimization rates, and boys higher perpetration rates (Smith et al., 2018; Tomaszewska & Schuster, 2021).
In Portugal, a recent study by UMAR (2020) carried out amongst youth between 11- and 21-years old shows that 67% of youth legitimates at least one form of violent behavior in an intimate relationship, with controlling behavior and stalking being the most tolerated displays of violence (26% and 23% acceptance rates, respectively). The same study also reports that 58% of participants have suffered from at least one form of violent behavior from their partner, with insults (30%) and control of social life (23%) being the most highly reported indicators of victimization (UMAR, 2020). Similar to what was found elsewhere (e.g., Smith et al., 2018), boys tend to more frequently legitimate violent behaviors in an intimate relationship, while girls tend to more frequently report having been victims of violent behaviors (UMAR, 2020). These prevalence rates are consensual with what was found in other Portuguese studies (Neves et al., 2016) and elsewhere (e.g., Wincentak et al., 2017).
Research also indicates that this trend continues into early adulthood. A national study with university students in Portugal reported that 53.8% of young adults had suffered from at least one act of dating violence, whereas 34.4% of students reported they had practiced at least one of these same behaviors (Neves et al., 2021).
The growing research on dating violence, particularly with adolescents, points out mental health consequences that may translate into risk behaviors, such as substance abuse, self-injury, suicidal tendencies, low self-esteem, or depressive symptoms (e.g., Murta et al., 2013). Experiencing a violent dating relationship also conditions future intimate relationships, being a strong predictor of domestic violence in adult age (Jennings et al., 2017).
Primary prevention of dating violence
Increased awareness of the incidence of violence in intimate relationships among youth over recent decades has resulted in an increase in research in this area, as well as in the development of prevention and intervention programs (Murta et al., 2013).
International political, social and health organizations (such as the World Health Organization, UNICEF, or the OECD) acknowledge the importance of implementing strategies for the prevention, rather than amelioration, of violence in intimate relationships, particularly working with youth aiming at sensibilization and primary prevention (WHO, 2015). Moreover, various literature reviews support the efficacy of violence prevention programs during the youth dating period (Cornelius & Resseguie, 2007; De La Rue et al., 2017; Hickman et al., 2004). In their review, Murta et al. (2013) found that efficacy studies of dating violence prevention programs showed an increase of participants’ skills for the effective identification of violent situations, as well as problem solving and negative emotions’ management skills, while also promoting greater awareness of dating violence, its causes, and consequences. An increase was also noted in the participants’ intention of providing assistance - and in the knowledge on how to do so - if in the presence of a dating violence situation. The same review also noted a positive change in beliefs and attitudes toward gender equality and violence intolerance (Murta et al., 2013).
The same trends are noted in De La Rue et al.’s (2017) meta-analysis, which also adds to the importance of addressing social skills’ training and the role of others in dating violence prevention programs among teenagers in school settings (De La Rue et al., 2017). These results highlight the importance of primary prevention of gender violence dynamics among youth, namely in educational contexts, in a systemic, systematic, and continued approach, with the aim of raising awareness among youth about healthy intimate relationships and the non-legitimization of abusive behaviors (UMAR, 2020).
The importance of raising awareness and preventing these issues during adolescence relates to a developmental stage in which youth tends to initiate intimate relationships, thus standing as a privileged time frame for approaching matters related to what is a healthy, non-violent, equal dating relationship (Shorey et al., 2017). This is also true for working on personal skills, such as communication, social problem-solving strategies, or relational conflict management, which play a large role in paving the way for healthy intimate relationships (Fernández-González et al., 2018).
Adolescence is a period of particular vulnerability for dating violence perpetration and/or victimization, and the question of when it is more effective to implement a prevention program arises. Although scarce, research on this question shows that interventions for reducing violence once it is already taking place are minimally effective (De La Rue et al., 2017; Shorey et al., 2017). Then again, Bonomi and colleagues (2012) found most teenagers report being perpetrators and/or victims of violence in dating relationships around the ages of 16-17 (Bonomi et al., 2012), whereas Johnson et al. (2015) found these phenomena becoming relevant in teens’ personal and social lives between the ages of 13-16 (Johnson et al., 2015). Moreover, between the ages of 15 and 25, a positive relation between age and the perpetration, victimization, or tolerance of dating violence has been consistently found (e.g.: Ontiveros et al., 2020).
If an intervention with the aim of being preventive takes place too late (e.g., early adulthood), the deviant behavior or the victimization may already be a reality, and unhealthy beliefs towards dating relationships may already be beyond the primary prevention scope. Conversely, if conducted too early, dating violence prevention programs may witness its positive effects dissipate before the expected age of the onset of deviant behavior in an intimate relationship (Shorey et al., 2017). Thus, mid to late adolescence seems to be a key time frame for the emergence of deviant attitudes towards dating violence, and hence for the implementation of prevention programs.
School settings as privileged intervention grounds
School has long been considered a context of excellence for the development and implementation of primary prevention programs with children and youth. Those which specifically address dating violence matters are no exception, with literature showcasing this trend (Cornelius & Resseguie, 2007). Furthermore, data has shown that peers play a key role in preventing dating violence amongst teenagers (e.g., Van Camp et al., 2014), with school representing an important ecosystem where the role of other students as bystanders is a powerful protective factor for dating violence.
Dating violence prevention programs have shown mixed findings internationally when empirically evaluated. Interventions tend to show positive results in increasing participants’ knowledge about and attitudes towards dating violence, despite the relationship not always being statistically significant (Lee & Wong, 2020). As for behavioral outcomes (related to perpetration or victimization), however, findings vary considerably in both significance level and direction of effect, with studies showing both positive and negative impacts (Lee & Wong, 2020).
Youth dating violence prevention in Portugal
In Portugal, there has been a recent effort to set up national awareness campaigns - entities such as the Commission for Citizenship and Gender Equality (CIG), or the Portuguese Institute for Sports and Youth (IPDJ), have been organizing awareness campaigns on dating violence directed at youth in the last decade. Other initiatives (such as online games; Almeida et al., 2018) or awareness events with the communities by law enforcement, for instance, are also being more frequently implemented. However, these low-frequency, low-intensity types of campaigns and initiatives, although valuable for the community, tend to be less effective than intervention programs that are longer in duration, and have greater intensity and specificity regarding content and target-groups (Caridade & Machado, 2013; Lundgren & Amin, 2015).
With this need in mind, Portugal also witnessed a slight increase in school-based interventions aiming to prevent youth dating violence. In the early 2000s, Matos and colleagues (2006) tested the effects of a school-based single session prevention intervention with high school students, which showed impacts on decreasing tolerance for dating violence both at post-test and at two-month follow-up (Matos et al., 2006). In 2008, Saavedra et al. (2013) implemented the Portuguese adaptation of The Fourth R (Wolfe et al., 2006), a 28-session program for students aged 13 to 18 years old, focusing on topics such as peer and dating violence, sexual risk behaviors, substance use and abuse, and gender equality. Impact evaluation showed a significant decrease in the legitimization of male psychological violence, male sexual violence, female psychological violence, female physical violence and female sexual violence, as well as a significant increase on the adoption of positive/non-abusive conflict resolution strategies (Saavedra et al., 2013). The IUNO II Project was a three-session intervention implemented in school settings between 2005 and 2006, with a sensibilization and information strategy with participants between 14 and 21 years old. Impact results showed efficacy in significantly decreasing acceptance towards different forms (sexual, physical and psychological) of dating violence, as well as a (non-significant) increase in positive conflict solving strategies (Saavedra & Machado, 2012). More recently, the program Lights4Violence (Vives-Cases et al., 2019), implemented in Portugal and five other European countries, consisting of 15 to 17 school based sessions with participants aged between 13 and 17 years old, reported a significant decrease in participants’ tolerance of benevolent sexism (Sanz-Barbero et al., 2022), but not in any other outcome variables when compared to the control group (Pérez-Martínez et al., 2022).
Interventions on dating violence, particularly in schools, have been on the rise in the national and international scene. They have not, however, been accompanied by studies on its efficacy (Matos et al., 2006), which means there is scarce information on the characteristics of these programs, and on how (or if) they are effective. Thus, despite there being few rigorous impact evaluations, and findings on its efficacy being small and mixed in effect, size and significance, school-based interventions on dating violence prevention stand as one of its most promising approaches (Fulu et al., 2014).
Hands: a prevention program for teen dating violence
The Hands Project was commissioned by an urban municipality in the greater Lisbon area, after local stakeholders became aware of the prevalence of teen dating violence amongst local youth, particularly witnessed in school grounds across the city. The municipality challenged a youth association to develop a prevention intervention on dating violence directed at teens in schools. Hands was thus developed as a brief universal intervention at the classroom and school levels, which addressed myths, beliefs, and attitudes towards teen dating violence, but also aimed to work on the social and emotional skills necessary for conflict resolution, positive communication strategies or bystander agency.
Brief universal interventions on teen dating violence have been proven effective (Jennings et al., 2017), and prevention programs in school settings with highly dynamic and interactive components seem to engage participants more effectively, and to obtain more positive outcomes (Darling-Hammond et al., 2019). Therefore, Hands was designed to use active and participative methodologies, such as role-play, debating, or the “world café” method, and built on the role of the student as an agent of change in their school setting and local community. Throughout three in-class sessions, one facilitator with the assistance of a teacher, implemented group dynamics and moments of reflection with the aim to: 1) deconstruct myths and beliefs regarding teen dating violence (session one), 2) work on conflict resolution and problem solving within an abusive relationship, either as victim, perpetrator, or bystander (session two); and 3) prompt participants to reflect and elaborate on their individual and collective roles as bystanders and members of a community, culminating with the elaboration of a sharable output (such as peer training sessions, videos or posters to share at the school; session three). These sessions thus covered the emotional, cognitive, and behavioral dimensions of students’ attitudes towards dating violence, while incentivizing their proactivity in being agents of change in their school communities.
Monitoring and validating Hands
Monitoring and evaluating intervention projects is a factor in their success, making it possible to determine the merit, value and meaning of a given intervention and to understand the advantages of its generalization (Alexandre et al., 2018). At the same time, monitoring and evaluation processes are fundamental for later decision-making processes carried out at different levels - micro (i.e., local, centered on a community or territory) and/or macro (i.e., in the role they can play for the change or development of policies or strategies and comprehensive action).
The evaluation of Hands included a monitoring (or process/implementation evaluation) and a summative (or impact) evaluation. Before planning the summative and process evaluation, a Theory of Change (Alexandre & Barata, 2020; Buitrago, 2015) for the Hands Project was produced to explain in more detail the activities to be developed, objectives and resources, as well as indicators for their measurement (see Figure 1).
Regarding the evaluation of program implementation, a monitoring process was conducted throughout the implementation period with all intervention groups. Specifically, this monitoring process evaluated implementation aspects like participants’ responsiveness and implementation quality, and included the administration of in-session observation tools, and feedback questionnaires for students and teachers. Monitoring implementation is a crucial aspect of the success of a program, since it allows teams to understand how the intervention is being implemented and to improve it as needed, according to context and participants’ needs (Durlak & DuPre, 2008). It also helps to understand and give meaning to impact results: if the evaluation tells us if a given intervention impacted on the intended outcomes, the monitoring process tells us why it did (or did not; Durlak et al., 2011). Particularly, participants’ responsiveness (Hennessey & Humphrey, 2020) and implementation quality (Dowling & Barry, 2020) stand as strong predictors of how effective a program is in promoting skills and altering behavior (Alexandre et al., 2018).
In addition, to assess the impact of the Hands project on participants’ attitudes towards dating violence, a study was conducted with students who received the three-session intervention program, and a comparison group, who received no intervention. Although in recent years there has been an increase in interventions on teen dating violence in Portugal, there is still little evidence on their efficacy (Ferreira et al., 2020). Since the Hands project aims to be a universal intervention focusing on prevention, its purpose was not to directly address perpetration and victimization but rather to work on attitudes, beliefs, and prevention strategies which, in turn, have been shown to decrease dating violence perpetration and victimization (Fellmeth et al., 2015). Thus, the need to assess Hands efficacy in promoting healthier attitudes towards dating violence in adolescents became pressing.
The present study
This paper describes the monitoring and impact evaluation of Hands, a brief school intervention carried out with adolescents from an urban municipality in the greater Lisbon area on the prevention of teen dating violence or, more precisely, on teens’ attitudes towards dating violence.
Particularly, we addressed the following research questions:
1) How was the program implemented? Specifically:
i. What were the levels of in-session observed quality of implementation?
ii. What were the levels of participants’ feedback about the project?
2) What was the impact of the Hands Project on the attitudes toward teen dating violence in its participants, when compared with a comparison group?
Method
Participants and setting
Monitoring sample. The pre-test sample of the monitoring evaluation corresponded to the intervention group, composed of 163 participants between 11 and 20 years old (M = 15.80, SD = 2.30) within five middle and high schools. The sample was 73.8% male, and a total of 10 classes were involved; 68.1% of students attended high school (corresponding to the 10th to 12th grades of the Portuguese schooling system) and the remaining 31.9% attended middle school (corresponding to the 7th to 9th grades of the Portuguese schooling system). More than half (54.6%) of the total sample attended a vocational course. As for the observation of session quality, a total of 15 sessions were to be observed in nine different classes, distributed by all five schools. Due to the sudden COVID-19 prompted lockdown, four of the planned observation sessions did not take place. The monitoring sample at post-test included answers to students’ feedback questionnaires from 49 participants, and to teachers’ feedback questionnaires from three teachers, as well as the observation of a total of nine sessions with eight different classes.
Impact sample. The study sample of the impact evaluation at pre-test included a convenience sample of 308 adolescents between 11 and 20 years old (M = 15.59, SD = 2.16) from the same five mentioned schools. The sample was 72.1% male, and a total of 20 classes were involved: 11 in the intervention group and nine in the comparison group. Sixty-four percent of students attended high school, and the remaining 36% attended middle school. More than half (51.9%) of the total sample attended a vocational course. Statistically significant differences were found between the intervention group and comparison group at pre-test for participants’ age (intervention group M = 15.80, SD = 2.30; comparison group M = 15.35, SD = 1.98, p = .034), and school grade, namely 9th (intervention group 0%, comparison group 7.8%, p < .001), 11th (intervention group 6.2%, comparison group 9.7%, p = .016) and 12th (intervention group 17.5%, comparison group 5.2%, p < .001) grades (Table 1).
Table 1 Differences in demographic characteristics between the intervention group and comparison group at pre-test.
| Variables | Intervention Group | Comparison Group | t | df | |||
|---|---|---|---|---|---|---|---|
| n | M (SD)% | n | M (SD)% | ||||
| Participant age | 161 | 15.80 (2.30) | 145 | 15.35 (1.98) | 1.837* | 304 | |
| Participant was male | 118 | 38.7% | 102 | 33.4% | -0.661 | 303 | |
| Participant was in 7th grade | 28 | 9.1% | 20 | 6.5% | 0.816 | 306 | |
| Participant was in 8th grade | 24 | 7.8% | 15 | 4.9% | 1.162 | 306 | |
| Participant was in 9th grade | 0 | 0% | 24 | 7.8% | -5.344** | 144 | |
| Participant was in 10th grade | 38 | 12.3% | 40 | 13% | -0.859 | 306 | |
| Participant was in 11th grade | 19 | 6.2% | 30 | 9.7% | -2.144* | 274 | |
| Participant was in 12th grade | 54 | 17.5% | 16 | 5.2% | 4.881** | 306 | |
| Participant attended vocational course | 89 | 28.9% | 71 | 23.1% | -0.986 | 306 | |
Note: *p < .05, **p < .001
Due to the outburst of the COVID-19 pandemic in Portugal and the consequent suspension of all school activities, there was also a high participant mortality rate in the impact evaluation sample from pre-test to post-test (78.9%) in both intervention and comparison groups. This affected both the impact evaluation and implementation monitoring processes.
The impact study sample at post-test included 61 students (45 in the intervention group and 16 in the comparison group), all attending the senior year of high school (12th grade in the Portuguese schooling system) in a vocational course in two different schools. Ages ranged between 17 and 20 years old (M = 17.92, SD = 1.01) and 76.9% of participants were male. Table 2 shows differences in demographic characteristics between the sample without post-test (n = 243) and the sample with post-test (n = 61); significant differences were found regarding participants’ age (pre-test sample M = 14.96, SD = 1.95, post-test sample M = 17.92, SD = 1.01; t(204) = -16.759, p < .001), school grades (all participants from post-test sample attended 12th grade: 100%, versus 2.1% of pre-test only sample; t(242) = -107.328, p < .001), and type of course attended (all participants from post-test sample attended a vocational course: 100%, versus 39.1% of pre-test only sample; t(242) = -19.417, p < .001).
Table 2 Differences in demographic characteristics between the sample without post-test (n = 243) and the sample with post-test (n = 65).
| Variables | Intervention Group | Comparison Group | t | df | ||
|---|---|---|---|---|---|---|
| n | M (SD)% | n | M (SD)% | |||
| Participant age | 65 | 17.92 (1.005) | 241 | 14.96 (1.949) | -16.759* | 204 |
| Participant was male | 50 | 76.9% | 170 | 70.8% | 1.010 | 107 |
| Participant was in 7th grade | 0 | 0% | 48 | 19.8% | 7.718* | 242 |
| Participant was in 8th grade | 0 | 0% | 39 | 16% | 6.802* | 242 |
| Participant was in 9th grade | 0 | 0% | 24 | 9.9% | 5.150* | 242 |
| Participant was in 10th grade | 0 | 0% | 78 | 32.1% | 10.696* | 242 |
| Participant was in 11th grade | 0 | 0% | 49 | 20.2% | 7.818* | 242 |
| Participant was in 12th grade | 65 | 100% | 5 | 2.1% | -107.328* | 242 |
| Participant attended vocational course | 65 | 100% | 95 | 39.1% | -19.417* | 242 |
Note: *p < .001
Measures
Below, we describe the instruments that were used to monitor program implementation, as well as to assess its impact on participants.
Impact sample.
Observation Tool. An observational tool was developed by the Hands’ design and evaluation team for the purpose of observing different aspects of in-session implementation quality. It was inspired by other similar instruments (e.g., the Student Participation Questionnaire, by Finn & Zimmer, 2012). Its final version was designed and reviewed in several staff meetings, and tested in one intervention session, after which followed final improvements (such as the wording of the items and improving the accuracy of the scoring process). Due to a shortage in team staff, only one team member administered this tool, so no kappa coefficient was calculated. The observation tool was composed of 33 items (α = 0.78), organized in four dimensions (three pertaining to the three agents in the session - students, teacher and facilitator -, and one related to structural aspects): a) 10 items to monitor students’ involvement and participation in the session (α = 0.75; e.g.: “Students were persistent when confronted with difficult problems, i.e., they actively sought a solution”); b) four items to register the teachers’ role in the session and as co-facilitators (α = 0.89; e.g.: “The teacher incentivized student participation”); c) 15 items to monitor the facilitators’ performance and skills in relating to the students, managing the session, and addressing its content (α = 0.89; e.g.: “The facilitator encouraged students to answer each other’s questions”); and d) four items to describe logistics, such as duration, space and materials (α = 0.69; e.g.: “All necessary materials were available”). Scores were provided on a five-point Likert type scale, ranging from “Totally disagree” to “Totally agree”, and the mean score was calculated for each dimension.
Student Questionnaire. A student questionnaire was developed by the project team to assess participants’ responsiveness, to be administered to all participating students at post-test. It was composed of seven items (α = 0.84) to be answered by students on a five-point Likert type scale (from “Totally disagree” to “Totally agree”), addressing overall feedback regarding facilitator’s performance, the content and duration of the sessions, and personal perceptions of program impact (e.g.: “The Hands Project taught me how to help victims of dating violence”). The final score was calculated using the mean of all seven items.
Teacher Questionnaire. A teacher questionnaire was developed to assess participating teachers’ responsiveness, and to be administered at post-test to the teachers who would assist session implementation. It included nine items on a five-point Likert type scale (from “Nothing” to “Very much”) concerning aspects such as the suitability of the sessions regarding students’ characteristics, or the teacher’s own involvement in the sessions (e.g.: “The sessions are useful to promote awareness and incentivize youth to combat dating violence”). No alpha was calculated for the teacher questionnaire because there was little variation in most of the items, and only three observations. Final score was calculated using the mean of all nine items. There was also a final “yes/no” question, asking if the teacher would recommend this program to other colleagues.
Impact evaluation measures.
ADV. The Attitudes toward Dating Violence Scales (ADV; Price et al., 1999; Portuguese adaptation by Saavedra, 2010) is a self-report 76 item questionnaire aiming to assess adolescents’ and young adults’ attitudes and beliefs towards dating violence. Answers are provided in a 5-point scale (from strongly disagree to strongly agree), with higher scores indicating greater acceptance of dating violence. The ADV scales are composed of six subscales organized by type of violence (psychological, physical and sexual) and the gender of the perpetrator (male or female), resulting in three subscales for male dating violence (psychological male violence - AMDV-Psychological - 15 items, e.g., “Relationships always work best when girls please their boyfriends.”; physical male violence - AMDV-Physical - 12 items, e.g., “It is never O.K. for a guy to hit his girlfriend.”; sexual male violence - AMDV-Sexual - 12 items, e.g., “A girl who goes into a guy's bedroom is agreeing to sex.”) and three subscales for female dating violence (psychological female violence - AFDV-Psychological - 12 items, e.g., “Girls have a right to tell their boyfriends what to do.”; physical female violence - AFDV-Physical - 13 items, e.g., “It is no big deal if a girl shoves her boyfriend.”; sexual female violence - AFDV-Sexual - 12 items, e.g., “It is alright for a girl to force her boyfriend to kiss her.”). Since the Hands project did not have the aim to explicitly work on sexual violence, as described in its Theory of Change (Figure 1), the two corresponding subscales were not administered, resulting in a final version of 52 items and four subscales. In this study, overall internal consistency at pre-test for each administered subscale ranged between acceptable and good (male psychological violence α = 0.61, male physical violence α = 0.81, female psychological violence α = 0.74, and female physical violence α = 0.84), while it was excellent for total scale (α = 0.90). At post-test, internal consistency ranged between good and excellent, (male psychological violence α = 0.88, male physical violence α = 0.90, female psychological violence α = 0.85, and female physical violence α = 0.90), including for total scale (α = 0.96). Subscale scores were computed using the mean, and total scale score was calculated by summing mean scores for each subscale. Participants’ gender and age were collected for each student as part of the ADV measure, and grade and type of course attended were collected by the project team directly from the school administrations.
Procedures
All five high schools in the municipality were invited by the city council to adopt the intervention and take part in the study, indicating a project coordinator at each school. The project coordinator then selected at least two classes from each school, ranging between 7th and 12th grades, to benefit from the intervention. Only one school selected three classes to receive the intervention. For each selected class, another class of similar characteristics (i.e., same school year, age range, type of course, and perceived by the school as similar in terms of overall academic performance) was chosen for the comparison group. Since there was not the possibility to randomly assign participants to each group, there is no control of exogenous variables, hence the option for the designation of “comparison group” (Murnane & Willett, 2010).
Schools were able to discuss the monitoring and evaluation protocols with the Hands’ team, in order to ensure all procedures were in line with school practices, and that families’ and school’s ethical concerns were addressed. Written informed consents were then collected from each student’s legal tutor. A member of the intervention team (who was not the main facilitator) then administered the Attitudes Towards Dating Violence Scales in the classroom, with the assistance of a teacher, on all selected classes of the intervention and comparison groups, prior to the beginning of the intervention. Participants’ confidentiality, anonymity and volunteerism were assured at all times. The sessions took place in normal classrooms (except with two groups, with whom the sessions occurred in the school theater and in the music classroom), and the dates were chosen by the teacher in charge of each class. These were teachers from various courses, such as Portuguese, Physics, Chemistry or Mathematics, and thus the sessions took place during these classes.
For the post-test assessment, the same procedures were planned to be carried out with all 11 intervention classes and nine comparison classes immediately following the final intervention session in each class. However, due to the COVID-19 pandemic outburst, only three of the 11 intervention groups received all three intervention sessions and completed the post-test measure. As for the comparison group, only one of nine groups completed post-test measure. As soon as in-person activities were suspended, the intervention team lost touch with all participants, which prevented the pursuit of adequate further post-test data collection.
As for the implementation monitoring procedures, the observational tool was administered by the same data collector (external to the intervention) to 15 randomly selected sessions (around 50% of the total) throughout the implementation period, five of each session (i.e., we randomly selected five of the first sessions, five of the second, and five of the third). This resulted in the random selection of nine different classes, across all five schools, to be observed at least once. Due to the sudden COVID-19 prompted lockdown, four of these observation sessions did not take place. The student and teacher feedback questionnaires were administered at post-test to the three classes that concluded the intervention.
Data analysis
For the evaluation of the implementation process, descriptive statistics were calculated for each dimension of the observation tool, and the student and teacher questionnaires.
For the impact evaluation, we first computed the total score of each of the subscales of our outcome measure. Then, to detect statistically significant differences between intervention and comparison groups in all subscale scores, a repeated measures ANOVA was fit to the total score of each of the subscales, controlling for pre-test levels, participants’ age and gender, and adding a group*time interaction. The estimate for the interaction, if significant, indicated that the difference in change in total scores from pre to post-test between groups may be attributable to the intervention. Since a decrease in scores means lesser acceptance of dating violence, an estimated significantly negative value of the interaction was interpreted as a positive impact for the intervention. The effect size was computed using Partial Eta Squared, which indicates the proportion of variance accounted for by the interaction or, in this case, the intervention effect.
All data cleaning and analysis was processed in SPSS Statistics software - Version 28.
Results
What were the levels of in-session observed quality of implementation?
Table 3 presents means and standard deviations for observed in-session quality. Average student participation and engagement tended to increase from session one (M = 3.25, SD = 0.51) to session two (M = 3.48, SD = 0.36), as did the facilitator’s performance (session one M = 3.98, SD = 0.47, session two M = 4.15, SD = 0.23). Teacher involvement in the intervention was consistently smaller, decreasing in the second session (S1 M = 3.06, SD = 0.97, S2 = 2.38, SD = 0.60). Space and materials scored above scale average throughout the intervention (S1 M = 4.38, SD = 0.43, S2 M = 4.06, SD = 0.24, S3 M = 3.50).
Table 3 In-session quality observation tool.
| Variables | Session 1 | Session 2 | Session 3 |
|---|---|---|---|
| M (SD) | M (SD) | M (SD) | |
| Student outcomes | 3.25 (0.51) | 3.48 (0.36) | 3.20 (n.a.) |
| Teacher outcomes | 3.06 (0.97) | 2.38 (0.60) | 3.00 (n.a.) |
| Facilitator outcomes | 3.98 (0.47) | 4.15 (0.23) | 3.80 (n.a.) |
| Space & Materials outcomes | 4.38 (0.43) | 4.06 (0.24) | 3.50 (n.a.) |
Note: n.a. = not available, due to N sessions = 1.
What were the levels of participants’ feedback about the project?
Table 4 presents the results of students’ responsiveness to the Hands Project with feedback questionnaires. Facilitator’s performance was the highest rated aspect of the intervention (M = 4.44, SD = 0.72), whereas the lowest score applied to the program’s effectiveness in making the participant more aware of dating violence situations (M = 4.11, SD = 0.71). It is, nonetheless, considered a high score since it is more than two standard deviations above average. Overall program satisfaction was above scale average (M = 4.26, SD = 0.91).
Table 4 Students’ feedback questionnaire
| Variables | n | Min | Max | M (SD) |
|---|---|---|---|---|
| The Hands Project motivated me to be more attentive to dating violence situations. | 46 | 3 | 5 | 4.11 (0.71) |
| I liked the way the facilitator implemented the sessions. | 46 | 2 | 5 | 4.44 (0.72) |
| The sessions allowed me to acquire more knowledge about gender inequality. | 46 | 2 | 5 | 4.17 (0.80) |
| Attending the Hands Project taught me how to help possible victims. | 46 | 2 | 5 | 4.28 (0.66) |
| The Hands Project gave me knowledge on how to prevent a violent relationship. | 46 | 2 | 5 | 4.20 (0.75) |
| Overall, I think the sessions had an adequate duration. | 46 | 1 | 5 | 4.20 (0.83 |
| Overall, I enjoyed attending the Hands Project. | 46 | 1 | 5 | 4.26 (0.91) |
Note: n.a. = not available, due to N sessions = 1.
As for teachers’ responsiveness to the Hands Project measured with feedback questionnaires, all three teachers gave the maximum score to all program indicators (M = 5.00, SD = 0.00), except for the suitability of the number of sessions that constitute the program (M = 4.67, SD = 0.58), and the perceived effort students make in respecting their peers after attending the intervention (M = 4.67, SD = 0.58; in both cases, one of the teachers scored “four”). All teachers also claimed they would recommend Hands to colleagues.
What was the impact of the Hands Project on the attitudes toward teen dating violence in its participants, when compared with a comparison group?
Figure 2 presents the results for the impact of the intervention on all four subscales of the Attitudes Towards Dating Violence Scales. For the male psychological violence subscale, statistically significant results were found for pre-test and post-test mean differences, F(1, 63) = 8,526, p = .005, but not for group effects, F(1, 63) = 0,851, p = .360, nor for time*group interaction, F(1, 63) = 0.101, p = .752, η 2 = 0.002. This means there was a significant decrease in the acceptance of male psychological violence from pre-test to post-test, but this decrease was not statistically different between the intervention group and comparison group. The estimated effect size or eta squared was also quite small, indicating that only 0.2% of the variation in male psychological violence was accounted for by the interaction.
As for male physical violence, although scores increased between pre-test and post-test in both groups, indicating higher tolerance of this type of dating violence, no statistically significant results were found for both time, F(1, 63) = 0,167, p = .684, or group effects, F(1, 63) = 0,251, p = .618, nor for time*group interaction, F(1, 63) = 0.130, p = .720, 𝜂 2 = 0.003. This means the increase between pre-test and post-test is not statistically significant, neither are the differences between the intervention and comparison group. Once more, the estimated effect size or eta squared was quite small, indicating that only 0.3% of the variation in male physical violence was accounted for by the interaction.
The same goes for female psychological violence. Although the comparison group scored higher at post-test, no statistically significant results were found for both time, F(1, 63)= 0.052, p = .820, or group effects, F(1, 63) = 2,318, p = .133, nor for time*group interaction, F(1, 63) = 0.120, p = .731, 𝜂 2 = 0.003. The estimated effect size or eta squared was also quite small, indicating that only 0.3% of the variation in female psychological violence was accounted for by the interaction.
Finally, female physical violence showed a very slight decrease between pre-test and post-test for the intervention group and an increase for the comparison group. There was a statistically significant interaction between time and group, F(1, 59) = 4.955, p = .031, 𝜂 2 = 0.095, meaning the difference in scores between pre-test and post-test is significantly different for the intervention group and the comparison group. Specifically, tolerance towards female physical violence decreased among those exposed to the intervention, while increasing in the comparison group. The estimated effect size or eta squared indicated that only 9.5% of the variation in female physical violence was accounted for by the interaction.
Figure 2 Estimated marginal means for all four subscales of the Attitudes Towards Dating Violence Scales for the intervention and comparison groups at pre-test and post-test, controlling for students’ gender and age. Note: Negative mean differences translate into a decrease in scores between pre-test and post-test, which is interpreted as a positive outcome, indicating a lesser tolerance of dating violence.
Discussion
The present study aimed to monitor and evaluate the Hands Project - a school-based prevention intervention for teen dating violence - with a sample of high school participants.
Regarding monitoring results, the in-session observational tool, developed by the project team, showed that quality was overall high, and participants and facilitator were engaged during implementation. Aspects of teacher involvement were observed to score below scale average in the second session, which may be due to session characteristics and dynamics. The second session involved several role-play exercises, in which teachers may have felt less comfortable in interacting with their students. This result offers a lesson for improvement to the implementation team, who may learn from this involvement feedback and try to improve it for future implementation experiences. We must consider, however, that the observation took place in four first sessions, four second sessions, and only one third session (due to the sudden cease of in-person activities prompted by the pandemic). As such, results of observation regarding the third session must be considered with caution, due to small sample size.
Results from the participants’ feedback questionnaires showed overall high satisfaction with the program, both by participating students and teachers. Students reported enjoying attending the sessions, particularly praising the facilitators’ performance, and illustrating her positive relationship with participants. This aspect is key to student engagement in the intervention, and a valuable indicator of the success of an intervention (Durlak & DuPre, 2008).
Similarly, although small in sample size, teachers’ feedback was remarkably positive, and results demonstrated overall program satisfaction. Teacher involvement in school-based interventions is still an overlooked aspect of implementation quality (Wright & Irwin, 2018), and stands as a recommendation of improvement for the Hands Project and other school-based interventions aiming to potentiate their effects on young participants. In order to do so, a higher engagement by educational agents is due, while also capacitating and providing tools for teachers to effectively manage dating violence situations. School-based interventions are reported to be more successful the more often they are addressed in the classroom, and the more teachers feel motivated to handle the subject at hand (Avery-Leaf & Cascardi, 2002).
Regarding the results of the Hands project impact evaluation, overall findings were mixed, with positive results only for the female physical violence subscale, i.e., the program was effective in improving participants’ attitudes towards this form of violence. There were also positive results, showing a decrease in tolerance, towards female psychological violence and male psychological violence for the intervention group, although not significant. Still, they stand as promising indicators of positive change in the attitudes and beliefs of those who benefited from the intervention, offering specific guidance on practice and intervention for high school students from vocational courses: a group largely ignored.
Lastly, an increase in acceptance was seen regarding male physical violence, although non-significant and present in both groups. Overall, this does not prove the efficacy of the intervention on improving the participants’ attitudes towards dating violence, since no significant differences between the two groups were found on three of the four subscales. As found elsewhere (e.g., Magalhães et al., 2016), psychological violence stands as the most tolerated form of abuse, and thus less acknowledged by youth; these beliefs may be more resistant to program effects.
When considering effect sizes, these are considered very small (Tanner-Smith et al., 2018) for three of the subscales, which is in line with the absence of statistically significant results. It is larger for attitudes on female physical violence - the only subscale with significant results. The estimated effect size indicated that 9.5% of the variation in female physical violence was accounted for by the intervention. This estimated impact is in line with most effect sizes found in prevention interventions for the increase of prosocial behaviors/decrease of tolerance of violence in social contexts with youth (Tanner-Smith et al., 2018).
Longer interventions have proven better results in improving attitudes toward different types of dating violence, i.e., being more effective in decreasing acceptance of physical and psychological violent behaviors in youth dating relationships, particularly when assessed with the same instruments used in this study (e.g.: Saavedra et al., 2013). However, promising results in terms of improving attitudes have been found in shorter interventions (e.g., Saavedra & Machado, 2012), which puts in perspective the relevance of dosage for dating violence interventions focusing on decreasing tolerance. More research needs to be done regarding other program components, such as the effect of more sensibilization/information oriented interventions versus ones with more dynamic strategies dedicated to skill development.
Limitations of the study
The scarce significant results for the program’s impact may be due to several study limitations. The drastic decrease in sample size from pre to post-test, due to the outburst of the COVID-19 pandemic, considerably hindered the robustness of the data analysis process, since a smaller sample size implies less statistical power to detect significant effects. This is especially true when considering the small sample size of the comparison group, and its size discrepancy in relation to the intervention group, but most particularly when they are different regarding demographic characteristics (such as age or school grade, as is the case). Then again, the groups who completed the implementation and assessment procedures (i.e., the post-test sample) are very homogeneous in sociodemographic characteristics - e.g., all students are in their senior year of a vocational course. A more detailed demographic description of the participants could potentially allow for further interpretation. However, no additional details regarding family characteristics or socioeconomic status were asked of students, because schools felt it would prompt a decrease in parental consent to participate, as well as a potential decrease in program interest by participating students.
The finding that the Hands program is effective in decreasing tolerance towards psychological dating violence in high school senior students from vocational courses is informative but also limits our results and subsequent generalization. It leaves several questions unanswered: Would younger students display higher program effects? Would high school students from non-vocational courses be more sensitive to the intervention? Or show positive results regarding other types of dating violence? These questions open the field for further experimentation in programs to reduce dating violence.
Lastly, we must consider characteristics of program design and implementation to interpret efficacy results. The monitoring of program implementation showed high participant responsiveness, including involvement, engagement, and overall satisfaction with the intervention. There is also observed in-session quality, including facilitator’s performance, teacher and student engagement, or the climate and interactions between all these participants. But the intervention is also small in its dosage (three 90-minute sessions) when compared with other aforementioned programs implemented in Portugal. Despite the Hands project contemplating factors that are fundamental for the efficacy of dating violence programs (such as the school as intervention grounds, using interactive methods for transferring information, or promoting participants’ self- and social awareness; Saavedra, 2010), it may not be enough in duration or intensity to generate the desired outcomes.
As such, there is also room for improvement in program design. Decreased teacher involvement in more active sessions may be prevented by adding a teacher training component to the intervention, i.e., aiming to set expectations regarding their role in the Hands’ sessions. This should be a priority improvement because literature has shown that in-class interventions in which teachers are highly engaged and participative tend to be more effective with students (Fulu et al., 2014). Also, improvements to the intervention regarding it being more explicit and intentional with the work on participants’ social and emotional skills (Jones et al., 2021, as well as broadening its scope within the school setting (Mahoney et al., 2020), are worth investing in. Based on existing evidence on these aspects of intervention quality, the Hands program can be more present outside the classroom, and more systemic, effectively engaging teachers, other school staff, and other students who do not benefit directly from the sessions, in order to promote its efficacy in improving attitudes toward teen dating violence throughout the whole school system.
Recommendations for research and practice
Program evaluations tend to criticize interventions on dating violence that are limited to informing participants on how to identify dating violence situations, and changing participants’ attitudes regarding the acceptance of violence. When compared to interventions teaching interpersonal skills for conflict resolution in romantic relations, they seem to report less efficacy (Murta et al., 2013). Thus, the design of prevention interventions that combine changes in awareness, beliefs and attitudes, and the development of social and emotional skills applied to teen dating violence is recommended for future practitioners.
Besides the negative correlation between experiencing teen dating violence and personal well-being (Banyard & Cross, 2008), the phenomenon has also been highlighted in the literature as a predictor of violence in intimate relationships later in life (Jennings et al., 2017). It is fundamental to continue investing in learning about the causes and consequences of violence in intimate relationships during youth, namely the main risk factors and longitudinal outcomes. This stands as essential research for the design and implementation of age and context appropriate prevention interventions. But future research and practice must also keep in mind that the most effective universal interventions are those which actively involve different agents, settings and systems of the participant’s life. They must engage the different stakeholders of school and community, in order to promote effective positive change (Mahoney et al., 2020).
With these aspects in mind, future implementation of the Hands project must also consider (1) increasing its dosage, namely by adding sessions that are (2) more focused on skill development and prosocial behavior, and (3) intentionally include other member of the school and local community, specifically teachers, by promoting workshops for this target population dedicated to their role as key members of the school system in the prevention of teen dating violence.
It is, thus, important to understand violence in intimate relationships as a complex and multilayered problem, whose prevention and intervention should encompass the cooperation of health, education and social services, as well as of each individual. This would contribute to a healthier, more equal social environment, where citizens strive for non-violent relations based on respect for the dignity and rights of others.
Credit authorship contribution statement
Catarina Castro: Conceptualization; Data curation; Formal analysis; Methodology; Project administration; Writing - Original Draft; Writing - Review & Editing. Carla Colaço: Conceptualization; Methodology; Project administration; Writing - Original Draft; Writing - Review & Editing. Clara Barata: Writing-Original Draft; Writing-Review & Editing. Margarida Fonseca: Data curation; Investigation.
