A few months ago I received an email from a curriculum director in a district where I sometimes do a bit of consulting: Did I have any data regarding my impact in schools? Did I know where I could find such data? The district needed to prove my impact in order to get the grant funding it was hoping for.
This email sparked a new area of inquiry for me: How do I measure my impact? How do I know if I’ve been effective as a coach?
I still consider myself at the beginning of this inquiry, but through trial and error, I’ve already learned a few things.
GOALS
Assessing my impact this school year would have been a lot easier if I had set much clearer goals for my work from the beginning. Much of the work I’ve done this year was based on teachers’ personal professional goals, which was a great start, I think. However, in the future I plan to work with each school community to set school-wide and system wide goals that are just as explicit and clearly written as individual teacher goals.
This year, and in years’ past, educators and I spoke often about our shared goals for literacy instruction, but they weren’t written down anywhere, and there wasn’t a clear plan from the start for how to assess progress toward those goals, other than surveys and evaluations now and then. Next year I plan to make some changes.
STUDENT-CENTERED DATA
As a coach, I’m focused on whole schools, as well individual classrooms and students. One of my plans for next year is to set out to collect student data to demonstrate the growth of literacy instruction in each school–across the school. For example, an engagement inventory, like the one in Jen Serravallo’s book, is a simple tool that I can use as coach to observe entire classrooms at work and quickly assess how we’re doing with engagement on a whole-school level.
I often use checklists to help me assess how we’re doing in different areas of classroom environments: strategy charts, writing centers, student writing on display, seating arrangement, etc. These checklists contain valuable school-wide data that could represent growth as a school over time in different areas. We also use checklists and rubrics for looking at student-writing. This data is another way to be able to look across the whole school to assess how student work is coming along.
I also keep notes of all my classroom visits. These notes are mainly to help me remember each visit from week to week, but I can also look back at these notes for patterns and signs of growth.
I might also use photographs to document growth over time. I often wish that I had before and after photos of classrooms I work with. Photos could be used to document classroom libraries, writing centers, charts, students at work, and more.
SURVEY DATA
At the start of this year, I began using Google Forms to gather feedback after each workshop and course that I taught, and it has turned out to be invaluable. With all the responses automatically feeding into a spreadsheet, I can keep better track of all the information, sort it different ways, and use different tools for finding key words, patterns, and trends in the responses.
What I liked about the surveys this year was that I had two or three questions that were always the same: 1) How did it go? 2) What could have been better? 3) What will the impact of this day/week/course be on your classroom? Asking those three questions allowed me to look across all the responses from all the workshops to look for patterns and common themes. For example, people nearly always loved participating in classroom labsite demonstrations, but were less unanimous when it came to having time to plan together with same-grade level colleagues.
I’ve also stopped using a simple “rate the day” on the scale of 1-5. What I’ve found, after a decade of doing this work, is that that scores are almost always 4-5, and I beat myself up over not getting 100% 5’s. Maybe this sort of question is helpful at a larger conference with many presenters, but not so helpful for me, individually. The open-ended feedback is far more revealing and useful, and I find that when a survey includes a scale, people tend to give less written feedback – they feel they’ve already rated the day, so why go to the trouble of also writing about it?
QUANTITATIVE DATA
Which brings me back to that curriculum director and his request for data. It would be wonderful if I could simply share all the feedback responses from surveys across the years. In fact, I have done that in the past, and hopefully it helps with grant applications. However, I know that many grant committees are looking not only for written recommendations and teacher evaluations of my work, but for objective, quantitative, “hard data” to support the work.
I need not explain here why standardized test scores, as a measure of success in literacy instruction is problematic. For a million complicated reasons, standardized tests are not great indicators of children’s authentic growth as readers and writers, and tend to be unreliable evidence of the quality of teaching that students received in any given year–which is the thing I ‘m most concerned with.
Yet states and other institutions are looking for scientific data that my coaching is having an impact, leaving me with a challenge. No scientists will be researching me specifically, any time soon, so I have begun to search for studies of the practices I support in schools, studies of the model of professional learning that I provide, and studies of other literacy coaches like me.
ONGOING INQUIRY
This is my inquiry project for the summer and next year: finding authentic ways to assess my impact in schools. I’m already thinking ahead to next year and years to come.
Are you a literacy coach? How do you assess your impact? Share your thoughts below.
TWO WRITING TEACHERS 
Hi Beth!
Such amazing ideas! I look forward to learning from you with each of your weekly posts!
I worked for many years under the brilliant leadership of Daniela Anello. As co-literacy coaches we assessed our impact with three pillars 1) stufent data 2) Improvement in teacher plans 3) Teacher survey data and Admin evaluation/feedback. 4) We also worked with teachers to develop individual literacy inquiry questions to drive their individual goal setting. For example, if we were in a coaching cycle on small group instruction we recorded student data at the beginning of the 6 weeks and the end, compared plans from beginning to the end and feedback. Lastly, last year I added in walk through with teachers to gather data on consistency across classrooms.
Hope this helps!
Laura Oldroyd
Literacy Coach
LikeLike
Oh my gosh! I remember when Daniela was my intern in NYC all those years ago! What a small world! You’re so lucky to have worked closely with her – she’s wonderful!
LikeLike
It’s so important for us to think about how we can assess our impact. Thanks for putting this out there. I always love learning from you, Beth. You’re always doing such smart thinking!
LikeLike
Beth,
I plan to write about this . . . just now I’m in the midst of the grandson’s birthday and the middle Florida favorite nephew’s graduation. We have a Wildly Important Goal around PD and Coaching in the Core and PD and Coaching in Interventions and then match it to student / performance by buildings in K-3.
First look at my summative data is here: https://docs.google.com/spreadsheets/d/10JBM_TYo_FQhGRJ5a7hYO31mHveBFjRGuYz9KLs3Ans/edit?usp=sharing
LikeLike
Thanks for connecting us to Diane Sweeney’s book. I will definitely check it out over the summer. I’m also a district K-5 coach across 6 buildings. It is hard to assess effectiveness when each building has different needs. I’m hoping to start next year with a goal in each building (wouldn’t it be nice if it could be the same across a few?) and have a self-reflection time every few months to check in on my work.
LikeLike
Currently, I am a literacy coach in a K-5 building. Next year, our building will be just K-3. Although, I’m looking forward to narrowing my focus, I will certainly miss the upper grades’ intellectual thinking! I survey my staff at the beginning of every year to get a sense of teacher need. I also get ideas from the year before–things that we can improve for the following year. The end of the year is always a whirlwind–that I forget to send out a survey to assess my impact! I’m definitely doing that this year! Thank you for the reminder. The coaches in our district just started a study group on Diane Sweeney’s brand new book: Student-Centered Coaching: the Moves. It looks like it has some valuable ideas! Thank you for your blog! I always love reading them and get inspired! Thank you!
LikeLike
I love Diane Sweeney’s work around student-centered coaching, and for the last two of years, I have been evaluated on the measurable impact I have on students’ growth as writers. It’s a lot of pressure, but it changes the spin of the work I do.
LikeLike
Thanks for the tip on the Diane Sweeney book! Like everyone else who shared, I also coach in a K-5 district. The topic of coaching effectiveness has come up recently, as we are working toward developing better ways to assess our methods. We actually work with Jen Serravallo and have embraced the Independent Reading Assessment, The Reading Strategies book and the Writing Strategies book (next year’s focus) . Through her reading hierarchy, our teachers are digging into reading data, so utilizing “student-centered’ data is key to our instructional and coaching decisions. I also administer and compile staff surveys, which then drives my coaching goals. For example, since last year’s surveys indicated teachers were unfamiliar with inquiry methods, I planned grade level professional learning opportunities utilizing various methods. My supervisor and I also initiated staff surveys frequently throughout the year to measure effectiveness. The next step is to tie that data to student growth. Like Elizabeth, I worry about using standardized test data, and would rather drill down into the facets of literacy growth. I would also love to hear from others who have found methods that work.
LikeLike
I bought Diane Sweeney’s book, Student-Centered Coaching, because I learned at a Coaching Forum that she dives into this topic and includes some resources for measuring impact in the appendix of that book. I haven’t read it yet (it’s on my summer reading list). She also has a website that references two studies on the impact of coaching in her resources and materials section. Looks promising. I plan to check that out!
LikeLike
Love Diane Sweeney’s work and recently read Student Centered Coaching with a study group of other coaches!
LikeLike
Diane Sweeney has done some work around assessing your impact.
LikeLike
I created a survey that I ask teachers, teams, disciplinary groups to complete. It is private so teachers do not need to “hold back”. The survey asks teachers to be specific whenever possible. It has been helpful.
LikeLike
I have yet to figure out how to assess my impact. I am just completing my first full year as a coach. I will check out the Serravallo book and wait to see how this topic enfolds in the coming months.
LikeLike
Thanks for this post! As an instructional coach, I also find it challenging to measure my effectiveness. There were a lot of similarities in the data we both collect. You have given me a lot to think about in regards to completing the engagement survey for teachers, as well as using open-ended responses for feedback after PD sessions. Thanks again for sharing your experience – data can be a struggle!
LikeLike
Thanks for sharing! I am a literacy coach, and I could completely relate to your blog. I feel that I don’t keep track over the year, but on on check-out slips for the staff throughout the year. I also reflect on our school literacy data from classroom on-demand writing samples, but I feel it is not reflecting the input the staff is putting in.
LikeLike
I think it is important to carry our concerns and work from the last year to the next one. My written reflection at the end of the year or semester includes the goals and carry-over plans for the following time. Creating a checklist or survey based on these goals creates continuity and accountability for professional learning and classroom carry-over.
LikeLike
LOL. Thanks!! I’m loving the information you share about instructional coaching/professional learning. I’m a professional developer and would love to connect with other people in this type of position. Are there good blogs, groups or other resources you use/like that I could start following or being a part of?
LikeLike
I am the ELA/SS coach in my district K-6. I am spread very thin but I need to reflect and assess my own progress and impact on my schools. I’m going to join the journey with you! I have a bunch of checklists as well I could use. Thank you for the spark!
LikeLike
The engagement survey sounds interesting. Which book were you referring to?
LikeLike
Caryn, it’s from Teaching Reading in Small Groups. Click the link and it will take you to the publisher. : )
LikeLike
Correction! Jen’s engagement inventory is from Literacy Teacher’s Playbook!
LikeLike
Thanks for articulating your thinking about Literacy coaching. I am eager to hear more about this journey of yours. I’m always looking for authentic forms of data reporting.
LikeLike
As an instructional coach, I am always questioning my effectiveness on student achievement. I would love to see more articles and conversation for the literacy coaching and professional development realm.
LikeLiked by 1 person
Like Caryn, I am a professional developer. Diane Sweeney developed Student-Centered Coaching which introduces a new way of looking at and delivering school-based coaching by putting the needs of students’ front-and-center. Student-Centered Coaching focuses on specific goals for student learning, rather than on changing or fixing teachers, a coach can navigate directly towards a measurable impact and increased student achievement. Diane Sweeney’s Results Based Coaching Tool measures student growth, shows implementation of instructional practices, and the work of the coach. I’ve worked with coaches and teachers using her tools and strategies with great results! Diane Sweeney has three books addressing her work, as well as conference opportunities, and a blog. Check it out at https://dianesweeney.com/.
LikeLiked by 1 person