| Author | Affiliation |
|---|---|
| Lillie B. Huddleston, EdS | Georgia State University, Counseling and Psychological Services, Atlanta, GA |
| Kris Varjas, PsyD | Georgia State University, Counseling and Psychological Services, Atlanta, GA |
| Joel Meyers, PhD | Georgia State University, Counseling and Psychological Services, Atlanta, GA |
| Catherine Cadenhead, PhD | Georgia State University, Counseling and Psychological Services, Atlanta, GA |
ABSTRACT
Introduction:
Bullying is a serious public health problem that may include verbal or physical injury as well as social isolation or exclusion. As a result, research is needed to establish a database for policies and interventions designed to prevent bullying and its negative effects. This paper presents a case study that contributes to the literature by describing an intervention for bullies that has implications for practice and related policies regarding bullying.
Methods:
An individualized intervention for an identified bully was implemented using the Participatory Culture-Specific Intervention Model (PCSIM; Nastasi, Moore, & Varjas, 2004) with a seventh-grade middle school student. Ecological and culture-specific perspectives were used to develop and implement the intervention that included psychoeducational sessions with the student and consultation with the parent and school personnel. A mixed methods intervention design was used with the following informants: the target student, the mother of the student, a teacher and the school counselor. Qualitative data included semi-structured interviews with the parent, teacher and student, narrative classroom observations and evaluation/feedback forms filled out by the student and interventionist. Quantitative data included the following quantitative surveys (i.e., Child Self Report Post Traumatic Stress Reaction Index and the Behavior Assessment Scale for Children). Both qualitative and quantitative data were used to evaluate the acceptability, integrity and efficacy of this intervention.
Results:
The process of intervention design, implementation and evaluation are described through an illustrative case study. Qualitative and quantitative findings indicated a decrease in internalizing, externalizing and bullying behaviors as reported by the teacher and the mother, and a high degree of acceptability and treatment integrity as reported by multiple stakeholders.
Conclusion:
This case study makes important contributions by describing an intervention that is targeted to specific needs of the bully by designing culture specific interventions and working with the student’s unique environmental contexts. Contributions also are made by illustrating the use of mixed methods to document acceptability, integrity and efficacy of an intervention with documented positive effects in these areas. In addition, implications for policy and practice related to the treatment of students identified as bullies and future research needs are discussed.
INTRODUCTION
Bullying is one of the most significant school problems experienced by children and adolescents and affects approximately 30% of students in U.S. public schools.1 This included 13% as bullies, 10.6% as victims and 6.3% as bully-victims.2 Bullying has been defined as repeated exposure to negative events within the context of an imbalanced power relationship.3 Bullying is a serious public health problem that may include verbal or physical injury, as well as social isolation or exclusion.3–4 As a result, research is needed to establish a database for interventions designed to prevent bullying and its negative effects within the context of school policies.4
Researchers have found that bullying may have deleterious effects for both perpetrators and victims, including social, emotional, mental health and academic concerns, as well as loss of instructional time.5–12 For example, a relationship has been found between bullying behavior and internalizing problems (i.e., depression and anxiety), as well as externalizing problems (i.e., aggression and hyperactivity).11–12 Further, bullies have been found to have more conduct problems and less favorable views of school than their non-bullying peers, which may lead to academic disengagement.5
Rationale for the Case Study
The purpose of this case study is to describe the implementation of an individualized psychoeducational intervention with an identified bully and to report the outcomes of the intervention in terms of acceptability, integrity and efficacy.13 This case study was unique because we used mixed methods (i.e., both qualitative and quantitative methods) to contribute to the database on acceptability, integrity and efficacy by providing a rich description of the cultural and contextual variables that may influence the implementation and outcomes of the intervention.14This case study was distinctive because it used the Participatory Culture-Specific Intervention Model (PCSIM) to design, implement, and evaluate the intervention.15 Based on an ecological-developmental stance, PCSIM addresses individual and cultural factors related to mental health and promotes cultural competence using culturally valued resources and coping skills. 16–18 PCSIM uses an iterative data collection process that incorporates feedback from stakeholders to promote treatment acceptability and cultural validity, treatment integrity and efficacy.15 The research questions were: (1) What was the nature of acceptability from the perspectives of stakeholders? (2) What was the treatment integrity of intervention implementation? (3) Was there a reduction in this student’s: (a) externalizing symptoms, (b) internalizing symptoms and (c) bullying behaviors?
METHODS
Context and Informants
We conducted this study in a southeastern urban public school district with 2,484 students and 499 students at the target middle school. The population was diverse with respect to ethnicity (approximately 40% African American, 52% Caucasian, 2% Asian, 2% Hispanic and 4% multiracial) and socioeconomic status (30% free and reduced lunch). The research team had an ongoing collaborative relationship with this school district for eight years.19 Bullying behavior was addressed in the district discipline policies, which were distributed to students at all grade levels. The school response to bullying depended on severity and could include: student participation in a conference with school personnel, assignment to alternative lunch area, partial or full day in-school suspension (ISS), out of school suspension, financial restitution for the repair of any damage, or consideration of an alternative placement for up to 10 school days.
The informants included the mother of the target student, the interventionist, a classroom teacher, the seventh-grade school counselor and the target student. The target student’s mother, Ms. S., was an African-American woman who worked in the education field. The interventionist was an African-American female doctoral-level school psychology graduate student who was certified as a school psychologist and had 10 years of classroom teaching experience. The seventh grade counselor was an African-American female masters-level school counselor who had been employed by the district for many years. Based on the tenets of PCSIM, stakeholders participated as informants by providing data to develop intervention goals and to assess intervention acceptability, integrity and efficacy.15
Qualitative Data
Interviews
All interviews were semi-structured and produced qualitative data. Interviews were conducted with the mother, teacher and the target student. Interviews were conducted with all informants prior to intervention to facilitate development of the intervention sessions. The pre-intervention student interview was audio taped, transcribed and coded for major themes. The interventionist took ethnographic notes during all other interviews. Teacher and parent interviews were conducted post-intervention to enhance outcome data. Parent interview questions included a focus on the target student’s behavior at home and school, parent concerns related to his behavior, and the results of previously employed strategies. The course instructor, which this student received the lowest conduct grade, participated in data collection (i.e., interviews, observations, and surveys). Examples of the questions from the student, teacher and parent interviews are reported in Table 1.
Behavioral observations
The referred student was observed in structured (classroom) and less structured settings (hallway, lunch) to determine the frequency and nature of bullying behaviors and to aid in intervention development. We used a narrative approach (i.e., rich description) for conducting behavioral observations to gain information regarding peer and teacher interactions.
Evaluation/Feedback Forms
We used qualitative student evaluation and interventionist feedback forms to gather narrative information related to intervention implementation, including acceptability and integrity of the intervention. The student feedback forms were completed at the end of each intervention session and were used to determine what the participant liked about the session, as well as what he would change about the session. The interventionist feedback form was completed following each session and provided documentation about culture-specific modifications as well as treatment acceptability and self-assessment of the interventionist’s performance.
Quantitative Measures
Behavior Assessment Scale for Children: Second Edition
The Behavior Assessment Scale for Children (BASC-2) was administered to the teacher, parent and student pre- and post-intervention.23 These data from the student were not considered because of observations indicating that the student did not read the items carefully and, instead, provided invalid responses. The BASC-2 is a behavior rating scale that was designed to evaluate personality characteristics, emotions, self-perceptions or parent/teacher perceptions of adolescents. At-risk T-scores range from 60 to 69 while T-scores of 70 or above are considered clinically significant. This instrument has high test-retest reliability (r = .91) and internal consistency (α = .89).23 We used the internalizing, externalizing and bullying scales for this case study.
Child Self Report Post Traumatic Stress Reaction Index
The Child Self Report Post Traumatic Stress Reaction Index (CPTS-RI) was administered before and after the intervention to determine change in symptoms related to post-traumatic stress experienced by the target student.20 The CPTS-RI was used to supplement information provided by the BASC-2 regarding internalizing problems. The CPTS-RI has high internal consistency (α = .86) and test-retest reliability (r = .84). Although the CPTS-RI does not yield standard scores, raw scores of 38 and above have been described as clinically significant in previous research.21,22
Qualitative Data Analysis Procedures
The qualitative data (interviews, observations, & evaluation feedback forms) were subject to thematic analysis by having one coder read through each piece of data to create a list of themes that were reflected by these data.24 We employed a deductive approach to coding in which the coder identified information regarding externalizing, internalizing and bullying behaviors in the data.17After the first coder had read through all data to generate a list of themes, a group of three coders read through all of the data again and used a consensus-based approach to confirm or modify each theme. This team also selected quotes illustrating these themes.25
Quantitative Data Analysis Procedures
We analyzed the pre/post quantitative data (internalizing and externalizing from the BASC-2) using a two-step process that included calculation of the Reliable Change Index (RCI) and determination of whether an observed change was clinically significant.26–28 We calculated the RCI based on the standard error of measurement or reliability of the instrument and the student’s pre- and post-scores for each instrument. We used the following formula based on Jacobson & Truax (RCI = X2 − X1/Sdiff). Sdiff is calculated by taking the square root of 2(SE)2, where SE is the test’s standard error of measurement.27 RCI scores of 1.96 or greater are considered to be statistically significant. Mean scores from the CPTS-RI and bullying content scales were analyzed descriptively. We did not calculate RCI scores for these two variables because standard scores are not reported for the CPTS-RI and there are insufficient data about reliability and standard error of measurement for these two instruments.
Background of the Case Study
The target student for the intervention was David, a 12-year-old African-American student in the seventh grade. David’s mother (Ms. S.) provided background and medical information. David lived with his mother and nine-year-old sister. His family history included a recent martial separation. However, regular contact with his father was maintained through weekend and extended holiday visitation. David’s medical history included a diagnosis of Attention Deficit Hyperactive Disorder, which was managed through medication and counseling.
Reason for referral
David was referred for the bullying intervention by members of the administrative and counseling staff and was described as a “provocative bully” by administrators and teachers. An administrator indicated that David had a tendency to “annoy” his peers verbally until they “reach[ed] their limit” and as a result became physically aggressive with him. The administrator described David’s behavior as verbal bullying. The school counselor expressed concerns about his limited ability to engage in prosocial interactions with peers and school personnel, as David appeared to “ignore the comments of adults” and seemed unaware of how his actions or remarks were perceived by peers. Ms. S. (David’s mother) expressed concern that her son was becoming verbally aggressive in reaction to being bullied at school. She cited school reports of inappropriate comments to teachers and peers as evidence of David’s verbal aggression and indicated that his bullying behaviors persisted or escalated irrespective of school and home interventions. Ms. S. and the school personnel stated that they were interested in determining the best ways to intervene.
INTERVENTION
Data obtained from interviews, surveys, review of records and observations were used to develop an individualized eight session intervention to address David’s bullying behavior.29 Intervention sessions are described in Table 2 including the sessions, the goals, and cultural modifications that resulted in the individualization of the curriculum.29
RESULTS
Consistent with the PCSIM, we evaluated this case by examining both the process and the outcomes of the intervention that was implemented with a student who had been identified as a bully-victim. We answered the acceptability, integrity, and efficacy of the intervention for this case study.15,30–32
Acceptability: Research Question 1
We defined acceptability as the extent to which stakeholders (e.g., mental health professionals, parents, teachers and students) find a particular treatment or intervention to be fair, appropriate, reasonable and consistent with their expectations of treatment.31 We collected acceptability data through parent, facilitator, student and teacher report and used data to modify the curriculum in an effort to increase acceptability and efficacy.15 For example, David reported in the session evaluation that activities that were less contingent upon verbal interaction were more acceptable than those that required him to discuss emotions. Through the recursive process of the PCSIM, subsequent sessions were adapted to allow for choice between various less verbally demanding tasks, such as those that allowed David to respond to the curriculum by creating artwork such as drawings or collages.16
Examples of high acceptability also were revealed through post-intervention data obtained from all stakeholders. For example, Ms. S. indicated that she viewed the intervention as an important resource to address her son’s social deficits related to interpersonal relationships with peers and family members. David’s teacher acknowledged the value of the intervention as a reinforcement tool by informing David of her ongoing communication with the interventionist to encourage him to behave appropriately in order to have positive remarks relayed about his behavior. We also obtained measures of acceptability from the interventionist after each session, suggesting that initial sessions were less acceptable due to the resistance encountered and the slow development of rapport between the interventionist and the target student. However, treatment acceptability increased during subsequent sessions as rapport developed due to curriculum modifications made based on student feedback (i.e., less verbal input was required).
Integrity: Research Question 2
We defined integrity as the degree to which core program elements are implemented and cultural adaptations are documented.15 This study employed a partnership model to maintain treatment integrity, by focusing on collaboration with stakeholders in order to be culturally responsive while maintaining the essential components and content of the intervention.30 We obtained integrity data through the interventionist feedback forms to evaluate the ways in which session goals were met. Based on a thematic analysis of these forms, treatment integrity was high as session goals were met in all of the intervention sessions (meeting the threshold of greater than 80% implementation of intervention components).31
Efficacy: Research Question 3a –Externalization
We collected qualitative and quantitative results related to David’s externalizing behaviors from the teacher and parent report. The teacher reported in an exit interview that David no longer engaged in disruptive activities after completing assignments but instead chose to read. David’s mother reported a decrease in the number of phone calls received regarding disciplinary concerns from the school during and after the intervention. There was a clinically significant difference in the teacher pre- and post-intervention BASC-2 scores reflecting reduced externalizing behaviors (RCI = − 3.74). There was no change indicated by the parent pre- and post-test BASC-2 scores on externalizing behaviors (Table 3).
Efficacy: Research Question 3b- Internalization
The school counselor reported that David was less withdrawn at the end of the intervention. For example, she indicated that he made eye contact and acknowledged the statements or requests of school personnel, which were skills addressed in sessions related to empathy and perspective taking. Although David’s CPTS-RI raw score of 20 did not meet the threshold of clinical significance (i.e., 38 and higher), his post-intervention score of seven suggested a lower perception of internalizing symptoms associated with post-traumatic stress after the intervention. Specifically, he indicated that he had fewer bad dreams and was better able to concentrate at school. Quantitative findings from the BASC -2 included a clinically significant decrease in Internalizing Behaviors based on Teacher report (RCI = −3.79). However, there was no change related to internalizing symptoms based on parent report.
Efficacy: Research Question 3c- Bullying
The results of the BASC-2 completed by his teacher revealed that David’s bullying behavior decreased based on pre-post test data. His score on the bullying content scale from the teacher BASC-2 decreased from the at-risk range (SS = 66) to within normal limits (SS = 59) for students his age. Ms. S. reported no change on the parent BASC-2 from pre- (SS= 62) to post-test (SS = 62) in regards to David’s bullying behavior. However, as mentioned earlier, she reported the number of discipline referrals decreased during and after the intervention. Further, qualitative findings from school personnel also suggested improvement in David’s behavior after the intervention. Additional support for positive change in this area is that there were no additional counseling or disciplinary referrals for the remainder of the school year (Table 3).
DISCUSSION
This case study contributes to the literature related to intervention with bullies by providing an in-depth description of a promising intervention model and by using mixed methods resulting in evidence that this intervention had high acceptability, integrity and efficacy.13 Using the PCSIM, this intervention successfully integrated data about the culture of bullying within the target school, as well as using knowledge gained through collaboration with parents, teachers and school personnel.15,20 This psychoeducational intervention engaged multiple stakeholders, including school personnel, the mother, and the target student, to facilitate intervention acceptability and integrity and thereby increased the likelihood that the desired outcomes would be achieved.15,30 Further, the use of mixed methods and multiple informants strengthened validity of the intervention and evaluation by examining findings across multiple informants and multiple sources of data.14
An important finding in this case study was related to the efficacy of this intervention. Based on prior literature, the referral concerns and the pre-intervention data, the intervention was designed to reduce behaviors and symptoms associated with externalization, internalization, and bullying.11–12Predicted reductions in externalizing behaviors and bullying were partially confirmed with quantitative findings reflected by the RCI for externalization and clinical significance on the bullying scale from the BASC-2.26–28 Additional support was provided by qualitative data from interviews and observations. Similarly, the predicted reductions for internalization were partially confirmed based on the RCI for internalization on the BASC-2 as well as by descriptive data from the CPTS-RI. These quantitative findings were confirmed by qualitative data obtained from school personnel. However, it is noted that the findings for internalizing were not supported by parent report.
The participatory approach to problem identification and intervention development incorporated in the PCSIM was successful in several ways.15 For example, school personnel and the target student’s mother identified ongoing communication with the interventionist as a strength of the intervention. This enabled teachers to provide insight into the daily interactions of the students, the previous intervention efforts of school personnel, and an overview of the student’s social, emotional and academic strengths and challenges. Further, collaboration with the interventionist provided teachers with an opportunity to experience the target student in a different light by examining the influence of family context on the student’s behavior. This interaction between stakeholders and the interventionist exemplified the recursive nature of the PCSIM and illustrated the potential importance of mental health consultation in facilitating positive outcomes when intervening with bullies.15,33
LIMITATIONS AND FUTURE RESEARCH
Since this case study was conducted with a single participant, more research is clearly needed to demonstrate the acceptability, integrity and effectiveness of this individualized intervention with identified bullies. In addition, given the range of findings from both the parent and teacher, future efforts should be designed to include input over time from multiple participants and to use these data for recursive revision of intervention plans. School-based (e.g., school counselors, school psychologists, school nurses) and mental health practitioners are uniquely qualified to design and implement culture-specific interventions for bullies in schools by using their relationships with stakeholders, along with ongoing data collection, to increase intervention acceptability, integrity and efficacy.15 Future research may include a greater emphasis on systematic evaluation of the processes used to consult with educators and parents, particularly since educators and parents can have different views, while also having great potential to influence children. Based on information gained through the iterative process of the PCSIM, the intervention might be used as a method of primary prevention by extending it to younger students.15 Further, research is needed to examine the range of ways that this intervention may need to be modified to address the characteristics of other bullies and their unique cultural and ecological circumstances. Such modifications might include multiple sessions per week, meeting with members of the target student’s peer group, and a greater focus on behavior management strategies.
POLICY AND PRACTICE IMPLICATIONS
This case study has important implications for practice in the context of public policy. While the ideas discussed in this paper may have the potential to create meaningful change in some bullies, it requires intense levels of data collection and analysis to address the acceptability, integrity and efficacy of this type of intervention. This requires a public commitment to the expense needed to carry out such intervention effectively. It also may require research based on public health models that seek less expensive methods of intervention and that emphasize a full range of preventive interventions, including primary prevention.4 In this context, it is noted that policies in place within a school, school district and/or community may play a role in strengthening intervention efforts.4For example, the intervention described in this paper was implemented in the context of school policies that did not tolerate bullying and that had clear guidelines for school responses to bullying. Also, schools policies of service delivery referred to as response to intervention that include a simultaneous focus on a range of services including primary prevention, risk reduction, secondary prevention and tertiary prevention.34 Research is needed to develop an understanding about the impact of such policies on the efficacy of individualized interventions such as this.
Footnotes
We would like to thank the student, parent, and school personnel who participated in this intervention. Funding for this work was supported by the American International Group, Inc. Additional funding was provided by the Center for School Safety, School Climate, and Classroom Management and the College of Education Dean’s Office at Georgia State University.
Supervising Section Editor: Monica H. Swahn, PhD, MPH
Submission history: Submitted January 20, 2011; Revision received January 21, 2011; Accepted March 7, 2011
Reprints available through open access at http://scholarship.org/uc/uciem_westjem.
Address for Correspondence: Dr. Kris Varjas
Counseling & Psychological Services, P.O. Box 3980, Atlanta, GA, 30302-3980
E-mail: kvarjas@gsu.edu
Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources, and financial or management relationships that could be perceived as potential sources of bias. The authors disclosed none.
REFERENCES
1. Wang J, Iannotti RJ, Nansel TR. School bullying among adolescents in the United States: physical, verbal, relational, and cyber. Journal of Adolescent Health. 2009;45:368–75. [PMC free article][PubMed]
2. Nansel TR, Overpeck M, Pilla RS, et al. Bullying behaviors among US youth: prevalence and association with psychosocial adjustment. J of the Am Med Assoc. 2001;285:2094–100.
3. Olweus D. Annotation: Bullying at school: basic facts and effects of a school based intervention program. J of Child Psychology and Psychiatry and Allied Disciplines. 1994;35:1171–90.
4. Anthony BJ, Wessler SL, Sebian JK. Commentary: guiding a public health approach to bullying. J of Pediatric Psychology. 2010;35:1113–5.
5. Beaty LA, Alexeyev EB. The problem of school bullies: what the research tells us. Adolescence.2008;43:1–11. [PubMed]
6. Merrell KW, Gueldner BA, Ross SW, et al. How effective are school bullying intervention programs? A meta-analysis of intervention research. School Psychology Quarterly. 2008;23:26–42.
7. Sourander A, Jensen P, Ronning JA, et al. What is the early adulthood outcome of boys who bully or are bullied in childhood? The Finnish ”From a boy to a man” Study. Pediatrics. 2007;120:397–404. [PubMed]
8. Ttofi MM, Farrington DP. Bullying: short-term and long-term effects, and the importance of defiance theory in explanation and prevention. Victims and Offenders. 2008;3:289–312.
9. Srabstein J, Piazza T. Public health, safety and educational risks associated with bullying behaviors in American adolescents. International J of Adolescent Med and Health. 2008;20:223–33.
10. Bosacki S, Marini Z, Dane A. Voices from the classroom: pictorial and narrative representations of children’s bullying experiences. J of Moral Edu. 2006;35:231–45.
11. Klomek AB, Sourander A, Kumpulainen K, et al. Childhood bullying as a risk for later depression and suicidal ideation among Finnish males. J of Affective Disorders. 2008;109:47–55.
12. Bacchini D, Affuso G, Trotta T. Temperament, ADHD and peer relations among schoolchildren: the mediating role of school bullying. Aggressive Behavior. 2008;34:447–59. [PubMed]
13. Yin RK. Case Study Research Design and Methods, Fourth Edition. Thousand Oaks, CA: Sage Publications; 2009.
14. Tashakkori A, Teddlie C. Mixed methodology: Combining qualitative and quantitative methodology (Applied Social Research Methods, No 46) Thousand Oaks, CA: Sage; 1998.
15. Nastasi BK, Moore RB, Varjas KM. School-based mental health services: Creating comprehensive and culturally specific programs. Washington, DC: American Psychological Association; 2004.
16. Bronfenbrenner U. Ecological systems theory. Annals of Child Development. 1989;6:187–249.
17. Varjas K, Nastasi BK, Moore RB, et al. Using ethnographic methods for development of culture-specific interventions. J of School Psychology. 2005;43:241–58.
18. Nastasi BK, Varjas K, Sarkar S, et al. Participatory model of mental health programming: lessons learned from work in a developing country. School Psychology Review. 1998;27:260–76.
19. Varjas K, Meyers J, Meyers B, et al. Positive psychology and the prevention of school-based victimization. In: Gilman R, Huebner ES, Furlong MJ, editors. Handbook of Positive Psychology in the Schools. Routledge Publishers; 2009. pp. 323–38.
20. Frederick C, Pynoos R, Nader K. Childhood PTSD reaction index. Unpublished manuscript. 1992.
21. Goenjian AK, Pynoos RS, Steinberg AM, et al. Psychiatric comorbidity in children after the 1988 earthquake in Armenia. J of the American Academy of Child and Adolescent Psychiatry.1995;34:1174–84.
22. Roussos A, Goenjian AK, Steinberg AM, et al. Posttraumatic stress and depressive reactions among children and adolescents after the 1999 earthquake in Ano Liosia, Greece. American Journal of Psychiatry. 2005;162:530–7. [PubMed]
23. Reynolds CR, Kamphaus R. The Behavior Assessment Scale for Children: Second Edition. Circle Pines, MN: AGS Publishing Inc; 2004.
24. Merriam SB. Qualitative Research: A Guide to Design and Implementation. San Fransico, CA: Jossey-Bass; 2009.
25. Lincoln YS, Guba EG. Naturalistic Inquiry. Newbury Park, CA: Sage; 1985.
26. Heaton RK, Temkin N, et al. Detecting change: A comparison of three neuropsychological methods using normal and clinical samples. Archives of Clinical Neuropsychology. 2001;16:75–91.[PubMed]
27. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology. 1991;59:12–9. [PubMed]
28. Wise EA. Methods for analyzing psychotherapy outcomes: A Review of clinical significance, reliable change, and recommendations for future directions. J of Personality Assessment.2004;82:50–9.
29. Varjas K, Meyers J, Parris L, et al. Bullying Intervention Curriculum 2007–2008. Atlanta, GA: Center for Research on School Safety, School Climate, and Classroom Management, Georgia State University; 2008.
30. Power TJ, Blom-Hoffman J, Clarke AT, et al. Reconceptualizing intervention integrity: A partnership-based framework for linking research with practice. Psychology in the Schools.2005;42:495–505.
31. Kazdin AE. Acceptability of child treatment techniques: the influence of treatment efficacy and adverse side effects. Behavior Therapy. 1981;12:493–506.
32. Perepletchikova F, Kazdin EE. Treatment integrity and therapeutic change: issues and research recommendations. Clinical Psychology: Science and Practice. 2005;12:365–83.
33. Caplan G, Caplan R. Mental Health Consultation and Collaboration. San Francisco, CA: Jossey-Bass; 1993.
34. Fletcher JM, Vaughn S. Response to intervention: preventing and remediating academic difficulties. Child Development Perspectives. 2009;3:30–7.