Data-Based Decision Making

As teachers we have a lot of experience gathering information about our students and using it to make decisions. For instance, we review grades to assess student progress on academic tasks and revise our lesson plans to ensure that students are successful at least 70% of the time in initial practice sessions and 90% or more during independent activities. When we observe students are more disruptive at certain times of the day, we look closely at the setting to find clues that explain this misbehavior. Sometimes we find that a task requiring more concentration needs to be scheduled in the morning instead of after lunch when students are more easily distracted. Decisions that are based on data help us to create the ideal learning conditions for our students. Many of our decisions can be made without collecting data systematically. However, at times a strategic data-based decision making process may be needed. A formal data-based decision making process can help us identify important variables related to our teaching faster and more efficiently than relying on our memory to recall important events. This is especially true for teachers who often deal with many challenges that require their attention simultaneously.

Quick, time-efficient strategies can be used when a teacher wants to collect data on one or more students. For instance, placing a handful of paper clips in one pocket and transferring one paper clip to another pocket every time a behavior is observed is a relatively simple strategy that can be used during instruction. Another strategy may include analyzing assignment scores after class by summarizing these data. In more complicated situations, assistance from paraprofessionals or other school personnel who can spend time observing and recording information may be necessary for data collection purposes. Some schools have organized student improvement teams so that teachers can ask other school personnel to conduct observations while they are teaching class. Observers in these schools include school psychologists, behavior support specialists, paraprofessionals, counselors, fellow colleagues, or administrators. Each school can create strategies to enhance data-based decision making systems by tailoring these strategies to their particular school's characteristics and needs. For more information about embedding data-based decision making into school systems, consult the School-wide Positive Behavior Support Module. School-wide Positive Behavior Support tool-coming soon

Data-based decision making can be used in many different ways with individual students or classrooms. Questions about student academic performance and both problematic and appropriate behavior can be answered by collecting information systematically. Examples of questions to investigate might include the following:

• Is the lesson plan too difficult for most students?
• Are students experiencing enough success on independent work tasks?
• Does the number of homework assignments completed increase when students receive reinforcement (extra free time, bonus points, etc.)?
• Are the new classroom management strategies related to decreases in student misbehavior?
• Are there times of day or specific activities when the students are more likely to be on-task?
• Is there a difference in academic scores on in-class exams when students complete a review activity that summarizes major points the day before?
• Has the student engaged in more social interactions with her friends and this month since the peer-mentoring plan started?

In addition to student behavior, you can collect data on your own behavior as well. Master teachers are constantly reviewing their performance and trying new strategies. Self-monitoring and self-management strategies can help you improve your instructional and classroom management skills. Data can be collected on your own behavior by using a small counting device that you can click every time you engage in a behavior. For instance, you could keep track of the frequency of verbal praise given to students. Other adults can come observe your class and collect observational data while you are busy teaching. Teaching students to collect data is a valuable learning activity and asking for their feedback about lessons provides valuable information that can be used to improve your teaching. Types of questions to investigate about your own work might include the following:

• Am I providing reinforcement to students on a ratio of 4 positives for every demand or correction?
• Which lesson did the students prefer and find more interesting?
• Am I moving around the room and using proximity to decrease problem behavior?
• Am I using pre-correction on a regular basis in math class?
• Am I attending to students when they raise their hand to answer a question?
• How often do I praise students for correct answers when they yell out the answer instead of raising their hand?
• Am I providing examples throughout the lecture?
• Are my classrooms expectations clear to the students? How many of them can tell me these class rules?
• How predictable is my classroom schedule? How often do I let the students know ahead of time that we will be changing activities?
• How much time do I allow for students to work independently? Is this too much or too little?

The decision to measure something is made when you are concerned about some event (for example, individual student or class academic progress, student conduct in class) or when you want to evaluate a new strategy or intervention. The first step is to define the behavior of interest. It is always very important to create a clear definition, especially when asking someone else to assist in collecting data. If the behavior is defined well, it will include:

• A brief description of the behavior,
• Information about what the behavior looks like (topography), and possibly what it does not look like (so as to not get confused with similar behaviors),
• Details about the frequency, length or duration of the behavior, and/or
• Information about the behavior's intensity.

If the concern is about an academic behavior, it is important to establish a certain criterion that describes the behavior of concern. For example, you may be interested in any score that is below 50% on a student's test. You can then count the number of times a student scores below 50% on the tests (note that this excludes all other scores, such as homework assignments). Good definitions will be written in such a way that someone who hasn't seen the behavior will be able to understand and observe it. 

Behavior

Good definition

Bad definition

On task

During class time, Jack is on task every time he is looking at the teacher when she is talking, when he is answering her, or when he is looking at a paper on his desk.

Jack is on task when he is working.

On time

Mimi is on time to class if her feet are in the classroom at the time that the bell rings. If Mimi is at the door when the bell rings (i.e. her feet are partly out of the doorway) she is not on time.

Mimi is on time if she gets to class by the time that the bell rings.

Non-compliant

James is non-compliant every time that the teacher asks him to begin working on an assignment and he does not begin looking at it within five seconds of the request.

James is non-compliant when he does not do what the teacher says.

Rudeness

Clark is exhibiting rudeness when a teacher is talking to him and Charlie rolls his eyeballs upwards while closing his eyes. This excludes any time that a teacher is not directly addressing him.

Charlie is rude to the teacher.

Once the target behavior has been defined, you can choose a measurement strategy that best fits the type of behavior you are observing.

Some measurement systems provide an exact measure of a behavior's occurrence, while others provide a general estimate or proportion of the behavior's occurrence. The type of measurement system chosen depends upon a number of different issues including what the behavior looks like (topography), the frequency of the behavior, and the time and energy the person observing can dedicate to observing and recording the data. In general, elaborate and complicated measurement methods for collecting data can yield greater accuracy. Nevertheless, this is only true when the method is used as intended. We can achieve greater accuracy with a method that is not as rigorous but is a better fit for the time, energy, and r of those responsible for using it. In addition if data are not used on a regular basis to make decisions, the quality of the recording often decreases and the process is viewed as a waste of time.

Each measurement strategy has advantages and disadvantages. Some measurement strategies are more accurate but are difficult to implement. In some cases, you may choose to use variety of methods to measure different aspects of the same behavior depending upon how much time you have to invest. Follow the guidelines below to identify the best measurement strategy for different types of behaviors. Please note that the methods have been placed in order of least to most difficult to implement. The links in this section will take you to a description of different types of measurement tools within the Data-based Decision-making Module.

Choose a measurement strategy by asking the following questions:

Does the behavior generate a product (for example, a written assignment, a clean table, or papers on the floor)? If the answer is yes, click here for Permanent Product recording.

Can you easily count every time that the behavior occurs (for example, raising your hand)? Can you easily identify when the behavior starts and when it ends? Would this behavior be easy to count (or does it occur so frequently that it would become complicated to track)? If the behavior is easy to count, click here for Event Recording.

If the answer is yes, ask these additional questions:

• When the behavior occurs, does it last for a while (for example, writing)? If the answer is yes, click here for Momentary Sample recording.
• Does the behavior happen so quickly that it is hard to catch (for example, swearing or making gestures)? If the answer is yes, click this link for Partial Interval recording.
• Is it important to know that the behavior continues without interruption (for example, studying)? If the answer is yes, click here for Whole Interval recording.

If the answer is yes, click here for Latency (Time to respond).

If you are collecting data in a team setting, the next step is to identify:

• Who will be recording the behavior
• Exactly when and where they are to be recording or observing the behavior, and
• How often observations will occur.

The person observing should be in fairly close proximity to the student they are observing to make sure all instances of the behavior can be seen. However, the person observing should also try to be as discrete as possible while observing the student, so as to not influence the occurrence of the behavior. It can be helpful to discuss types of student behavior or times when a behavior will not be counted. For instance, a teacher and paraprofessional may be collecting data on how often a student places his head on his desk. The teacher may not think that it is a problem for the student to put his head on his desk during breaks. However, if the paraprofessional counts all occurrences of the student's behavior while the teacher only records occurrences during lessons, there may be problems later interpreting the data. The number of times the student's head was on his desk is in part reflective of the way in which each observer recorded the data.

Deciding how often observations will occur depends upon a number of factors including the time available to collect data, the concern or strategy being evaluated, and the type of behavior. A common mistake is to assume that behavior must be collected all the time, every day. Creating a highly intensive data collection schedule is not usually necessary. However, it is crucial for everyone measuring the behavior to be clear on when the behavior is to be measured so that everyone's numbers are comparable. It is also important to think about how long observations will be conducted. You will need to observe long enough to identify a clear pattern of behavior over time and to feel confident that you have gathered enough information to answer your questions.

The way to summarize data efficiently for decision making is to create a visual summary using a graph. A graph is a visual representation of the occurrence of behavior over time. After instances of behavior are recorded on a measurement form, the information is summarized and then transferred to a graph. Graphing the data you are collecting helps to organize the information and identify important patterns related to behavior. The information collected before any interventions have been implemented is referred to as a baseline. The information collected during the implementation of interventions is referred to as intervention data. Baseline data are compared to intervention data to determine whether the intervention you are conducting is resulting in positive outcomes (for example, increases in academic achievement, increases in positive social skills, and decreases in problem behavior).

Developed by: Rachel Freeman, Ph.D., Marie Tieghi-Benet, M.S., University of Kansas