Ongoing Projects & Studies

Despite increased scholarly attention on classroom management, classroom management assessment, and consultation-based professional development activities, the availability of feasible, flexible and defensible classroom management assessments remains limited (Reddy, Fabiano, & Jimerson, 2013). Given the reported deficits in pre- and in-service professional development in conjunction with a growing emphasis on prevention within a tiered service delivery approach, coaches, consultants, and trainers are increasingly being called upon to support improvement in educator classroom management practice. Use of performance feedback and coaching incorporating screening and formative assessment data imbedded within a collaborative consultation framework attempt to shift professional development goals away from awareness-raising to behavior change (Mitchell et al., 2017; Reinke et al., 2008; Simonsen et al., 2017). The DBR-CM was developed to address the limited availability of feasible classroom management assessments.

PST 1-2-3 provides explicit yet flexible guidance to educators as they engage in team-based problem-solving and data-based decision-making activities to address challenges facing students, staff, classrooms, buildings, and districts within tiered service delivery models. PST 1-2-3 draws from core problem-solving components, behavioral consultation processes, and practical, real-world experiences to address frequent barriers to ineffectiveness and inefficient SB PST. PST 1-2-3 uses structured agendas, repetition and consistency, time limits, and deemphasis of prior training or experience to address barriers to success such as resistance, inconsistency, poor training, and lack of knowledge or skills relative to intervention, supports, data-collection, or problem-solving. Since initial pilot implementation in 2008, PST 1-2-3 has been refined through incorporating additional research and theory, feedback from users, and outcomes for students. The effectiveness of PST 1-2-3 is rooted in its unique features. PST 1-2-3 requires little training for most participants, it uses a consistent 3-meeting cycle, and activities are driven by simple, structured, time-bound agendas.  

  The NASP Practice Model identifies consultation as a professional practice domain for problem-solving and the delivery of evidence-based interventions (NASP, 2020). School psychologists frequently engage in consultation activities, collaborating with consultees (e.g., teachers) to define a socially significant problem, analyze the problem, develop a plan (i.e., an intervention) to address the problem, and establish a monitoring system to evaluate the intervention (Kratochwill et al., 2008). Although consultation continues to function as a major mechanism for delivering interventions to children and adolescents in school settings, there remains a lack of available tools for evaluating the quality of interventions selected or generated within the consultative process. The implications of this shortcoming extend to practice and intervention research where, without knowledge of intervention quality, faulty conclusions regarding the effectiveness of consultation may result. To address this problem, the Brief Measure of Intervention Quality (BMIQ) was developed for the purpose of evaluating the quality of consultation-derived interventions.

  In the last decade, teachers have consistently reported student behaviors as a top concern in their classrooms, while also reporting deficits in their pre-service classroom and behavior management training (Reinke et al., 2011). A classroom environment characterized by frequent disruptions and inappropriate behaviors prevents students from learning and teachers from teaching effectively (Skiba & Rausch, 2006). Behavior problems predict low academic achievement, school dropout, and drug abuse in adolescence. Without appropriate intervention, these behaviors often escalate and could ultimately lead to placement in special education or exclusion from school (Walker et al., 2000). This is problematic because special education is linked to negative outcomes for students and exclusionary discipline practices and special education evaluations disproportionately impact students from marginalized underserved groups (Bradley et al., 2008; Skiba, 2002). 

Many of these problems can be mitigated through a continuum of tiered interventions that target problem behaviors in the classroom and are minimally time and resource intensive (i.e., tier II; Sugai, 2009). This tiered system of intervention allows for implementation of early interventions without a significant disruption to student or educator routines. One class of interventions frequently employed at Tier II, are broadly labeled self-monitoring interventions. 

Generally, self-monitoring interventions are characterized by explicitly teaching student self-management strategies such as how to monitor (i.e., observe and record their behavior) and evaluate (i.e., compare their behavior rating to an external standard), combined with reinforcement (Shapiro & Cole, 1994). Previous research has demonstrated the effectiveness of self-management interventions in increasing on-task behavior (e.g., DiGangi et al., 1991; Harris et al., 2005; Mathes & Bender, 1997) and academic productivity and accuracy (e.g., Harris et al., 2005; Shimabukuro et al., 1999), as well as decreasing disruptive behavior (e.g., Hoff & DuPaul, 1998; Koegel et al., 1992).

     Unfortunately, the progress monitoring of these interventions, while practical, may lack the psychometric evidence necessary to support their use for data-based decision making. However, there is a robust evidence-base supporting the flexibility, usability, and defensibility, of the DBR Single Item Scale (DBR-SIS) making it well-suited to serve as an intervention mechanism especially for those incorporating a self-monitoring component like the present study (Chafouleas, 2011). The DBR-SM intervention was developed as a self-monitoring intervention using the DBR-SIS assessment methodology and formatting. 

     To this end, the Direct Behavior Rating-Self-Monitoring (DBR-SM) was developed to serve as a minimally a self-monitoring intervention using the DBR assessment methodology and DBR-SIS formatting. DBR-SM combines self-monitoring and performance feedback intervention mechanisms using the DBR-SIS assessment methodology and formatting. The use of a structured, standardized assessment tool (e.g., operationally defined behaviors, item scoring, standardization) as the base for a self monitoring intervention that includes a reliability check (i.e., performance feedback) appears advantageous. Continuity in assessment and intervention in a single format (i.e., form, target behaviors) across the course of intervention process (i.e., screening, baseline, progress monitoring, maintenance) represents the opportunity for time and inference savings and improved defensibility. Furthermore, given the psychometric support for DBR-SIS, the DBR-SM offers users a level of defensibility not shared by other self-monitoring interventions. 

The development of the URP-WR is  grounded in the understanding that consumer subjective definition of a usable resources was likely, typically skewed towards accessibility. Usability more formally, refers to “the extent to which a system, product, or service can be used by the specified users to achieve specific goals with effectiveness, efficiency, and satisfaction in a specified context of use” (International Organization for Standardization, 2018). In other words, high usability means that the school psychologist is able to easily use and apply the recommendations provided in the resource to their setting with confidence. Low usability means that the school psychologist is unable to easily use the web resource, whether this be due to low quality recommendations, low quality evidence to support these recommendations, or the inaccessibility of the resource itself on the internet (i.e., one has to scroll through five pages of hits in order to find the resource). 

Initial development and validation efforts included an initial content validation procedure following by exploratory factor analytic and item reduction procedures using pilot study data. Analytic procedures resulted in 31 Likert scale (i.e., 0 to 6) items, across four domains, including accessibility, appearance, plausibility (which includes items relating to the feasibility of the recommendations as well as the credibility of the research supporting them), and system support (referring to the amount of support needed from a school or other system in order to use the resource effectively). The URP-WR exceeds the utility of other similar measures currently available (see Lydia M. Olsen Library, 2018; Schrock, 2019) as itallows users to engage in better digital citizenship practices, or more objectively and quantifiably evaluate and compare web-based EBP/I resources. Additionally, over time, accumulation and aggregation of consumer evaluations of web-based resources using the URP-WR would allow for new consumers an additional data source to use when evaluating web-based resources. In short, URP-WR scores could serve as a more objective version of a Yelp star system.

Overall, the URP-WR is a promising tool in development that aims to help school psychologists and educators make well-informed decisions regarding web resource selections in schools. The URP-WR seeks to (a) lessen the burden on practitioners by providing a tool to guide web resource selection and (b) narrow the research to practice gap through promotion and selection of evidence-based resources.