Can Standardized Test Data be Formative?

Posted by on Mar 27, 2010

Very often we refer to state standardized testing data as summative.  It is used to determine if a student, and an institution, meet AYP.  We apply the data, much as the somewhat tired analogy goes, as a learning autopsy.  We identify problems and areas of health, and perhaps even the cause of learning death, but we say it’s too late at that point for us to use it to change anything for that student.  The problem, most people say, is that we don’t receive the results in a timely manner, and thus, can only use it to reflect back upon.

I wonder if we can change that.

Because at the beginning of a given school year, you typically have at least several years of data on each of your students.  You have how they performed on the test last year, and the year before, and depending on what grade level you teach, you might even have the data for quite a few years.

What if we approached our standardized testing data this way?

Instead of basing your instructional decisions for this year on what a different group of kids did last year, what if you looked at the students you have at the beginning of the year and used their historical data?

This might shift our perspective from summative to formative.

I often see schools and districts use the performance data from the previous year to make instructional decisions for the next year.  For example, students perform poorly on vocabulary one year.  So, the teacher or perhaps even entire grade level, makes the determination to focus on vocabulary as a weakness for improvement for the next year.  The problem is, what if the class you have this year is actually very strong in vocabulary but really need help with comprehension?  Or what if only several students are very strong in vocabulary but really need help with making connections?  What if we looked at what each student needs individually based on how they have done over the years?

I wonder how much this would change.

I honestly don’t know how much valuable information can be found and used in a formative capacity in state standardized testing.  I’ve a feeling, though, there might be more there than we realize.

Thanks to DrWurm for the use of the Flickr image.

20 Comments

  1. Matt Townsley
    March 27, 2010

    I think there’s some potential to your idea, Ben. Have you heard of RTI (response to intervention)?

    A few places to begin reading, if you’re not familiar with it.
    http://www.rti4success.org/
    http://wrightslaw.com/info/rti.index.htm
    http://en.wikipedia.org/wiki/Response_to_intervention

    Using standardized tests to begin the RTI makes sense, particularly in the context of helping struggling learners.

    Here in Iowa, we’re calling our flavor of RTI, “instructional decision making” or IDM.

    http://www.iowa.gov/educate/index.php?option=com_content&task=view&id=801&Itemid=1305

    Often times, at least in my district, the assessment that kicks off the IDM process is not the state-mandated standardized test, but as you are suggesting, why couldn’t it be? Is this the type of use you are envisioning for standardized tests?

    Reply
  2. Frank Pearse
    March 27, 2010

    Ben,

    I see the logic in your thought but am worried about the implications of using these test results in this way. I believe that there is a huge risk that teachers will move {farther} away from worrying about how their students are learning {and thinking} and instead focus on how well they can have their students perform on THE TEST! If the use of the standardized tests in this was is to have students learn to think for themselves then there will definatly be some merit, however if the focus is to have the student perform better on the tests the following year I believe that true learning will be negatively impacted.

    Fortunatly I work in a system that is not completly drived by standardized tests. Student receive about 5 hours of provincial testing in both grade 4 and 7 and then not again until grade 10. and the tests are not linked to funding in any way.

    Frank.

    Reply
  3. Ben Grey
    March 27, 2010

    Matt-

    Thanks for all the great links. Yes, we presently use the RtI process. Although, most of our interventions are based on our NWEA MAP test data rather than our standardized testing data. We had an independent study conducted that showed MAP as a very strong predictor of performance for students on our Illinois State Standardized tests, so we feel the use of MAP not only serves to give us important growth data, but it also provides quite an accurate predication of ISAT performance as well.

    Yes, I wonder if we can’t find more instructional value to the information we glean from our standardized tests. I believe too often we misapply the results of this data, or simply allow it to serve as information for accountability exclusively.

    Frank-

    Absolutely I am worried about the same thing. My recommendation for using the data in this way is not to present a scenario for focusing exclusively on student performance on the assessment, but rather, to allow the standardized test data to truly inform instruction based on student need. It seems we seldom use the data in this way. It could certainly prove another valuable data point as we work to individualize the way we engage students in the process of learning.

    Thanks both of you for the great comments. Lots to consider in all of this. Especially as we continue to see things moving and shifting in our standards and assessments on a national level.

    Reply
  4. Frank Pearse
    March 27, 2010

    Matt,

    Good to hear. Not sure if your state has something like this, but if you click the link below you can see our ‘implications for instruction’ report. This can be used to inform our instruction in areas where large portions of our kids did not do well…definitely a starting point.

    My School

    Reply
  5. Tim Furman
    March 27, 2010

    The idea of measuring the growth of the actual students a teacher faces in a given year appears to be part of President Obama’s proposed changes to NCLB, at least according to this Washington Post article:

    http://bit.ly/ba5hyd

    The divorce between data sets and student sets is for me one of the puzzling aspects of assessment that schools face when trying to fight of the inevitable failure to make AYP. However, I’m under the impression that some states are much more data-focused than others, and that in some of these places, teachers are given very rich data-yields on their current individual students at the beginning of the year. But even in these places, schools are judged in the end as to how the current kids did compared to the previous kids, rather than as to how the current kids improved over the year.

    I would wager that even among educators, it is not widely understood that this is how AYP works in the current model. I’m only just beginning to understand what the President is proposing, but it’s actually quite sweeping, for better or worse. We’ll see.

    Reply
  6. Heather Mason
    March 27, 2010

    We can. In fact, we have to. In our district, we are required to look at our past year’s data and determine what goals we want for this year. We use FCAT combined with FAIR (a state mandated reading test) to make placement decisions for students. And as teachers we are encouraged to use FCAT data to determine what lessons to focus on with students. So not only can it be formative, it is already being used that way.

    Two problems:
    1. In some subjects, the information taught one year isn’t necessarily the standards for the next. It is just an indicator of how well a student learned (or how well the teacher taught) the subject during one given year. Many subjects aren’t even tested; in our state technology, communication outside of essays, art, music, history, geography, problem solving, government among others are left out.

    2. Standardized testing is a political process not an academic one. The tests are in place to assess teachers, administrators and schools much more than they are to assess students. A child’s performance on one day of testing doesn’t guarantee a true assessment of a child. Many students of mine have test anxiety due to the high-pressure nature of the FCAT. Many know the material but have trouble focusing for the duration of the test (our reading test is one 80 minutes session, 10 minute break, second 80 minute session). These students may understand the material, but still perform poorly on the test.

    If we really want to get the most out of standardized testing, we need to stop looking at it as the end all be all of assessment….and that doesn’t look like it’s happening any time soon. In fact, here in Florida there is a bill in our state Congress that will base over half of a teacher’s evaluation on test scores and than base pay solely on that evaluation. The breeds fear, not good formative assessment.

    Reply
  7. Dean Shareski
    March 27, 2010

    Our province has attempted to do just a you suggest.
    http://www.education.gov.sk.ca/Assessment-for-Learning

    Unfortunately, they’ve simply recreated traditional standardized testing and been slow to get results in teachers’ hands. They do them in May which is ludicrous unless you specifically work with the teacher from the next year who will be able to adjust instruction accordingly.

    We’ve lobbied our province to make these changes and are some hopes but in the end, unless you make a distinct effort to make it about helping students learn more, not just raise scores, you’ll fail everytime.

    You’re right, with some tweaking it could be useful but it requires not only a change of structure and format but of attitude. That’s the hard part.

    Reply
  8. Jon Becker
    March 27, 2010

    While you’re at it, why not give the 4th grade test that is to be given at the end of the 4th grade year at the beginning of 4th grade (in addition to the end of the year)? No point on focusing on standards for which students already demonstrate proficiency, right?

    Reply
  9. Ben Grey
    March 27, 2010

    Tim-
    Yes, the proposed change in Obama’s plan does focus much more on student growth, which I do think is a step in the right direction. I agree with you on the AYP comparison usage of the data. The way we compare one cohort to a previous year, is in my opinion, a very bad application of the data.

    Heather-
    Point number 2 is very well taken. We must understand this is but one point of datum in a student’s progress. In fact, most state standardized tests measure very finite, discrete learning skills and disregard an entire portion of who a student really is. I wrote a bit about that idea here. http://bengrey.com/blog/2010/03/airplanes-and-education/

    Dean-
    It isn’t actually terribly problematic that they do the tests in May. It just requires us to look at the situation in a bit different manner. Let’s say you’re a 7th grade teacher. At the beginning of the school year, instead of looking back at the 7th graders you had last year and seeing how they did on the standardized test, I wonder what would happen if you looked at the new 7th grade class you have coming in, and you looked at their 6th grade tests. Maybe even their 5th, 4th, and 3rd as well. I believe our paradigm is to look at the results from the students we had the year before. I don’t think that serves to inform us all that well. We have plenty of data on the students we get in the fall, but how often do we stop to look at that and allow it to inform the learning experiences we’re offering for our students? You’re right, it would require a change of attitude, and that would likely be the highest hurdle to leap.

    Jon-
    I’ve been wondering that same thing myself. I wonder if that’s why we’re starting to hear about NWEA becoming the potential national assessment as it has the capacity to do this. Most present state tests don’t given their nature and the cost of evaluation.

    Reply
  10. Tami Thompson
    March 27, 2010

    I think there is a problem in the time gap between when the test is taken and when the data are available. For example, in Colorado, 4th graders take their CSAPs in March, yet testing information is not available until first part of September, usually. That’s a six-month gap, hopefully with growth and learning occurring during that time. As a 4th grade teacher, I also have limited longitudinal information on the same types of tests because different tests are given for primary (K-3) and secondary (3-6) students. Poor third grade, they’re tested to death with both primary & secondary tests given.

    Reply
  11. Chad L.
    March 30, 2010

    Ben, our district not only gives the state mandated test, but we also give the NWEA MAP test 3 times a year for all students, kindergarten through 5th grade. I actually think that data is more valuable to the teachers because, like you say, it focuses on the students the teacher actually has, not the ones from previous years. It sure is a lot of testing for our students, but it does provide a lot of data.

    Reply
  12. Heather Mason
    March 30, 2010

    Many people are listing the other tests given along with the mandated state testing. We also give other tests throughout the year, some of which are better, others the same.

    I’m wondering…when it comes to these formatted, standardized tests, is it possible to test too much? And if so, where do we draw the line between too much and just enough?

    Reply
  13. Testing, Testing, 1,2,3… « Teacher in Transition
    March 30, 2010

    […] yesterday I read this post by Ben Grey. He posed the idea that maybe test scores have value as formative data rather […]

    Reply
  14. Ben Grey
    April 1, 2010

    Tami- Yes, that is one of the larger issues for the primary teacher who begin the standardized testing with their students. There isn’t a whole lot of longitudinal data to look at, and getting the results back the next year when their students are gone doesn’t serve to inform them of much.

    Chad- I agree that it does become a lot of testing. However, if we can actually use the data to make decisions that are best for our students, then I think the sacrifice is worth it. Unfortunately, that isn’t always happening in many school districts.

    Heather- I think that is a very valid question, and I love Grant Wiggins response to it. At ASCD, he said about the question of teachers saying we don’t have time to assess so much, “If you say you don’t have time for this, you assume that the teaching is more important than the learning. Feedback is the key to reaching goals. Saying there’s no time is to confuse causing learning for mentioning stuff.”

    Reply
  15. Russ Goerend
    April 2, 2010

    The way I understand it, formative assessment should be a quick assessment to check for understanding of what is currently being taught. The standardized tests I’ve encountered as a teacher don’t fit that definition. The closest I’ve found is the NWEA MAP tests our students take, although getting those tests to fit that description takes some squeezing. The tests take an hour-plus to take, and while the results are immediate, they don’t directly reflect what is currently being taught in individual classrooms.

    I could be wrong about (or oversimplifying) that definition, or I could not be aware of standardized tests that fit that definition (or both), though.

    Reply
    • Ben Grey
      April 2, 2010

      Russ- Thanks for the comment. I’ll try to clarify the best that I can, though admittedly there are many who can explain this much more thoroughly. Check out this article for more, or read anything by Rick Wormeli or Grant Wiggins that you can. They’re great with this.

      Formative and summative assessments are not determined by their length. They are determined by their nature. A summative assessment is a final assessment to see what a student has learned based on a specific set of standards. Typically, standardized assessments are thought of as summative because the results inform how a student has performed in a given year on a given set of standards, and it also informs, to some degree, how a program or school is performing with their curriculum. Although, that assumption can be terribly flawed. That’s why I wrote this post. To see if we can actually use those data to move us into a formative assessment mindset.

      Formative assessments are ongoing checks to see how a student is doing. The difference with a formative assessment is that is typically doesn’t count for a grade, and both students and teachers can do something with the data immediately after receiving it. Like the article above references, students going out on the road with a driving instructor is a type of formative assessment.

      So, it’s really how we approach the assessment and what we do with the information it yields that determines if something is formative or summative.

      I hope this helps. Feel free to push back or challenge or disagree with any of this.

      Reply
  16. Frank Pearse
    April 2, 2010

    Ben,

    Thanks for the continued thought (to both you and your readers). Your post, and resulting discussion, has lead me to finally getting aroudn to posting about our (British Columbia’s) standardaized tests (the F.S.A.’s) and how we can use (are using) these in a way that is not dissimilar to what you describe. I would be interested in your, or your readers thoughts:

    http://principalofthematter.com/?p=127

    Cheers,

    Frank.

    Reply
  17. Russ Goerend
    April 23, 2010

    Ben,
    I should have used “quick.” What I meant was “timely” (quickly after instruction). Formative assessment, in essence: this was taught, have they learned it. Over-simplified?

    If I knew exactly what was on the test, I could work to make my instruction fit the timing of the test, but I don’t. That leads to another issue I have with this idea, standarized tests are teacher-created. I’m not saying all formative assessments have to be teacher-created, but I do think assessments are better that way. I also think formative assessments should be focused. I don’t the cumulative nature of standardized tests to fit that idea.

    I realize I’m being a downer. I just know the time spent on high-stakes tests could be better spent learning. The amount of time spent taking and analyzing those tests relative to their use is out of whack.

    Have to go change the baby. I’m going to post this, even though it’s not fully thought through. Please keep pushing me. I know I seem resistant, but I’m open-minded.

    Reply
  18. Russ Goerend
    April 23, 2010

    By “should” of course I meant “shouldn’t” in that first line.

    Reply
  19. Thoughts on Assessment 3: Writing the obit on summative assessment | Constructing Meaning
    July 8, 2010

    […] “Can Standardized Test Data be Formative?” by Ben Grey (Twitter) at The Edge of Tomorrow […]

    Reply

Leave a Reply