This article is presented in 5 sections, with each new section sent out weekly. The focus is on 5 critical factors to have “right” when preparing to make data informed decisions: getting the “right people” the “right data” in the “right format” within the “right time” using the “right process” in order to make well informed decisions for divisions, departments, schools, staff, classrooms, students and families.
Introduction:
In our last segment, the importance of alignment with teaming structures was discussed in order to ensure the right people were at the table to engage in data driven decision making. Click here to review this previously published piece. In this segment, the focus will extend to the critical nature of relevant data available to these teams.
Right Data
Once the right people are at the table and the team composition is aligned to support data driven decision making, the right data needs to be available. Educators often have limited time during the instructional day to collaboratively problem solve. As such, it is critical that they have the right information at their fingertips. For instance, teams that are analyzing discipline data would be able to have well informed discussions if information shared included the number of monthly office referrals broken down into categories such as: gender, race, type of referral (major vs. minor), location of incident, time of day/day of week, and a brief description of the behavior (i.e. the 5 W’s). Additional information such as longitudinal data would also be helpful for teams to make comparisons across specified timeframes to look for patterns such as an increase or decrease in behaviors occurring at certain times of the year. In turn, this information could be leveraged to create real time interventions and proactive strategies aligned with the identified needs which is likely to result in improved outcome measures.
When barriers such as access, insufficient, inaccurate, or irrelevant data show up in meetings, it can greatly reduce the effectiveness of the team. For example, if a division level or school level team were trying to determine how well students were responding to core instruction in a given content area and the only data available to consider was SOL performance data, it is likely that the team will have difficulty making this determination for two reasons:
- While teams could consider student performance measures related to the state standards and could also drill down to sub group performance rates within the division and across the state, they could not consider how their students are progressing with the division selected curriculum and/or if there are concerns related to the alignment between the two (i.e. what is taught vs what is tested).
- In addition, SOL data may not be timely enough to help inform teams considering student performance in the current school year.
In this example, additional data points such as universal screening data, curriculum based measures, benchmark assessments, or formative assessments may be other quantitative data points needed for decision making. Teams also benefit from including qualitative information sources such as data from family focus groups and student interviews, to help provide context and additional perspectives. Considering the descriptive input from these sources can help inform the discussion in a richer, more in depth way that sheer numerical analysis cannot. The intentional inclusion of qualitative data can assist teams to move beyond what the problem is (e.g. noticing that students with disabilities are underperforming in the area of reading when compared to their non-disabled peers), to analyzing why the difference exists ( e.g. a review of division documents suggests there are not sufficient researched based reading interventions available). Problem solving in this instance becomes more inclusive, thereby more responsive, and more likely to avoid unintended consequences as a possible result of potential blind spots.
An incomplete picture can impact the way the team responds to the data and can lead to inaccurate conclusions or misalignment of resources. It is therefore incumbent on teams to identify which data sources or systems are available to them in each area of focus as well as what information each of those data sources provides. In support of this idea, Flannery et al. (2019) suggest that the following questions are useful for teams to consider as part of establishing an effective data informed decision making process:
- What are the decisions the team needs to make?
- What are the data needed to make these decisions?
- Who collects and summarizes the needed data?
- How and when can the team access the data?
Mandinach (2012), states that there are at least two critical components to ensuring that teams have the right data. First, are the technological tools or data management systems that are utilized to support the data inquiry process. Due to greater availability and potential for increased accessibility and function, many school systems have shifted to incorporate data management systems or platforms that house important educational data such as attendance, discipline, or academic performance. These systems broadly allow stakeholders to drill down more intentionally and analyze data in a deeper and more robust way. Data systems can generate on demand reports that can help teams examine which students are meeting expectations and those that require additional support. Some platforms embed components that allow for universal screening and progress monitoring which can be used to aid in the implementation of a multi-tiered system of support. Although data management systems offer efficiency and other potential benefits, they can also be the unintended cause of internal barriers if teams do not carefully consider the above listed questions.
The second component Mandinach (2012) highlights is human capacity. While it is critical for the right people and the right data to be present, in order for educators to use data effectively, they must have the skills and knowledge or data literacy. In the next segment of “5 Factors to Get Right with Data Informed Decision Making”, we will consider the importance of the “right” format in order to ensure that teams are able to analyze and interpret data meaningfully for decision making.
Tip: Is your school or division considering a data management platform? Prior to selecting the system, identify your needs and evaluate available tools relative to those needs. Consider using this two step tool from the Center on Multi-Tiered System of Supports at American Institutes for Research to assist with this process.
References
Ellen B. Mandinach (2012): A Perfect Time for Data Use: Using Data-Driven Decision Making to Inform Practice, Educational Psychologist, 47:2, 71-85
Flannery, K. B., McGrath Kato, M., Horner, R. H. (September, 2019). Using Outcome Data to Implement Multi-tiered Behavior Support (PBIS) in High Schools. Eugene, OR: OSEP TA Center on PBIS, University of Oregon. Retrieved from www.pbis.org
Selecting an MTSS data system. Selecting an MTSS Data System | NCII. (n.d.). Retrieved January 20, 2023, from https://intensiveintervention.org/resource/selecting-mtss-data-system
