On-Demand Performance Assessments & the Norming Meeting

One year that I taught in New York City, all 3-5 teachers were required to score the problem solving tasks on the state math tests.  I remember sitting around a table with my grade-level colleagues, at a different school, learning how to objectively score students’ responses to the problems.  I quickly learned this was not an easy task since what one person thought was a 3 (grade-level proficiency) was another person’s 4 (above grade-level proficiency).  Coming to a consensus on what was a 1, 2, 3, or 4 should’ve been easy since there was a scoring rubric, but it wasn’t.  (Hence the reason we spent a half-day learning just how to score them properly.) In order to ensure teachers’ own expectations didn’t affect each student’s score, each test taker’s short answer problems had to be scored by two teachers.

Earlier this month, I took Annie Taranto‘s “Staff Development in Writing: Methods That Maximize Growth” course at the TCRWP Writing Institute.  We spent about an hour talking about norming meetnigs with regard to assessing on-demand writing at the beginning of the school year.  As a group, we used Writing Pathways Grades K-5: Performance Assessments and Learning Progressions by Lucy Calkins (Heinemann, 2013) to anchor our discussion. Most of the people who took Annie’s section in August were literacy specialists, with a few classroom teachers sprinkled in.  We did a mock norming meeting during one of our class sessions.  It took us awhile to agree on whether or not the opinion piece we were examining was a grade 3 or grade 4 level based on the piece of writing and the levels in the Learning Progression for Opinion Writing.  We fell into a predictable pitfall that many folks fall into when they do this kind of work: we slipped into side conversations with the people sitting near us, discussing our personal thoughts about the writing.  Annie had to bring us back, reminding us, “We’re slipping into side conversations. We all talk together.”  This was really important since everyone in the room needs to hear what their colleagues are thinking in order to come to a consensus when norming student writing.  Therefore, when you hold a norming meeting with colleagues, make sure someone (hopefully it’s your staff developer or literacy coach) says things like, “We are talking as a group.”  When Annie said things like that it rerouted us and brought us back together.

Here is some additional information and some helpful tips Annie shared about norming meetings:
  • We don’t look at writing and see the same things as our colleagues.  Everyone needs to sit down to talk about what they saw, what they are looking for, what they are expecting.  There needs to be a common understanding about what kids’ writing should be scored as.  (Helps parents understand why some writing is considered a 4 by one teacher and a 3 by another teacher.)
  • We need to standardize our expectations.
  • One of the things teachers need to think a lot about is that we don’t always look at our own students’ work objectively.  Sometimes we score kids lower or higher based on our own expectations.  Instead teachers should think about where their students are in relation to grade level and on the learning progression (pg. 82 of Writing Pathways).
  • There’s information beginning on pg. 25 of Writing Pathways about norming meetings.  Essentially, Annie summarized the steps as follows:  *First, you and your grade-level colleagues score one student’s writing as a group.  Together you talk-out the student’s writing to come to a consensus about how the student’s work ranks on the learning progression.  Second, everyone on the team rates another student’s writing individually.  (A grade level might need to do several students’ levels depending on how much of this work teachers have done in the past).  Then, everyone cross-checks to see if the group ended up with the same conclusion. Then, you look at the grade level anchor paper (e.g., pg. 106 of Writing Pathways) to see if it matches up with the student.  That is, read the anchor piece for the grade level to determine if the student’s piece resembles that student’s writing.  Finally, you devise a plan that leads to action and instruction.
    • * Two more tips for the first step:
      • The bottom line for step one is that you want to figure out which level best matches a student right now and why.  Don’t split hairs over something.  You’re looking for the overall level.
      • Make sure every individual part does not take 20-30 minutes to discuss.  You don’t want to get into a full-blown discussion about whether something is a fourth grade transition or a fifth grade lead.  You don’t want to spend oodles of time debating whether something is structured like it’s a first or second grade piece.  If you go box-by-box, then you keep your initial discussion of a student’s work to a five minute conversation.  Remember you have 15 things to get to!
  • Annie doesn’t believe teachers should assess their own students’ on-demand writing pieces.  (And I agree!)  Reason being: one is more objective when grading other students’ writing.
    • If teachers don’t score their own students’ writing, they must talk to the colleague who score their students’ on-demand pieces so they will understand why the writing was scored in  a given way — and so they’ll know how to move their own students forward.
  • Doing this kind of work helps teachers figure out how to teach students (and to set conferring goals and data-based, whole class minilessons that will move students forward.
  • The learning progressions are on pages 82 (opinion writing), 124 (information writing), and 178 (narrative writing) of Writing Pathways.  The progressions are not rubrics.  For instance, Annie advised us to score students where they mostly fall (grade-level wise).  For instance, you don’t score a student on a second grade level on their lead, a third grade level on their transitions, a fourth grade level on their elaboration, a second grade level on their spelling, etc.  Instead, your ask yourself, which column (aka: grade level) are students mostly in?
    • Say, “I think this student looks most like (a grade ___), because…”

I’m preparing to do this kind of work with a local school this fall.  While I’ve  administered and reviewed many on-demand pieces of writing, I’ve never participated in a norming meeting until like the one Annie had us go through until two weeks ago. I certainly feel more prepared to this this important work with other educators as a result of what Annie taught me so that on-demand assessments can really drive writing instruction.

Have you had successful norming meetings with your colleagues?  Please share your secrets for how you keep meetings, on-track and focused.  Also, if you have used Writing Pathways Grades K-5: Performance Assessments and Learning Progressions please share your tips for using the book.