by: Vivienne Porritt
Posted August 20 2019
You are planning an improvement project and you want to have evidence of the impact you have achieved. It’s familiar situation. Let me ask a question – when will you evaluate impact and how will you do it?
Middle and senior leaders often struggle in evaluating the impact of school improvement and are looking for practical and simple ways to do this. In working with school leaders, I’ve developed a practical approach that’s simple in concept yet rigorous in the difference it can make. This hinges on when we evaluate impact and clarity over what is being evaluated.
When do we evaluate impact?
Traditional impact evaluation tends to be when you and your team have finished whatever improvement project you initiated. Usually this happens at the end of the project or initiative when you feel and hope you have made a difference. At best, you want to celebrate impact and share how something has been improved especially student outcomes. At worst, you want to tick a box, prove you have done what was asked of you1. I want to ‘move thinking from ‘proving’ to ‘improving’2
Clarity over what is being evaluated
Steven Covey’s exhortation that we should “Begin with the end in mind”3 is important as all planning about the potential impact of an improvement project ‘should be undertaken before….activity starts’. 4
The light bulb moment for me was reading the work of Thomas Guskey who states impact evaluation should be ‘an integral part of discussions during the earliest stages of … development planning when … goals are defined, and activities specified.5
So, planning an improvement project should be focused on the difference improvement will make for teachers /organisation /students (the change that will happen) rather than activity (what will happen). In my experience this rarely happens and so it’s difficult and often impossible to show evidence of change as what the change will look like in practice hasn’t been identified with enough rigour.
- Step 1: Initial planning should focus on clarity about what we can improve, what we want to achieve in practice and the evidence that will demonstrate such impact. The more time spent planning, the easier it will be to evaluate whether you have achieved the desired difference.
In applying Guskey’s research whilst working with school leaders, it became clear that another step was needed. We need to be very clear about the original practice we want to improve, our baseline. This is important as it helps us to assess how much of a difference we make when bringing about improvement.
- Step 2: Establish current practice or baseline so that you can evidence the quality and depth of the subsequent impact on adult practice and young people’s learning.
Another significant benefit to such clarity is to enable an effective match between the baseline practice and appropriate actions that will best effect such change.
- Step 3: Be clear on evidence of current practice and evidence of what improved practice will look like. Then, and only then, decide on the strategies and actions you will employ to bring about such change.
Step 3 is hard as educators seem hard wired to get on with practical actions rather than take time to plan what will help you know whether you make the most difference possible. To me, impact evaluation is most definitely not about proving you have met your appraisal objective. The experience of one project leader I worked with helps here. She learned that the purpose of ‘evaluating impact was to improve … activity; it was not an end in itself.’6
- Step 4: Evaluate the improved practice and depth of impact compared to your baseline
Impact evaluation is therefore about designing a way to improve the quality of the intended outcomes and to evidence and celebrate the depth of such impact. This requires a shift in mindset from focusing on an input model where we can prove that what we have done has worked, to an impact model where improvements make a bigger difference than we thought possible. And that feels much better than ticking a box: rather it enables you to be a catalyst for change!
Further reading
How to lead and support innovative professional development (https://library.teachingtimes.com/articles/pdt_how-to_lead-support.htm)
Making a difference: practical ways to evaluate the impact of professional development. (https://tdtrust.org/making-a-difference-a-practical-approach-to-evaluating-the-impact-of-professional-development)
Middle leaders as catalysts for change. (https://www.oxfordowl.co.uk/for-school/pd-bp/conferences/great-teaching-cardiff/vivienne-porritt) Peter Earley, and Vivienne Porritt, (eds) (2010) Effective Practices in Continuing Professional Development: Lessons from schools. London: Institute of Education Press
References
- Porritt, V. (2010) How do we know we are making a difference: evaluating the impact of professional development? Presentation for National Union of Teachers conference.
- Porritt, V. (2013) Baseline/Impact approach, UCL Institute of Education. Unpublished.
- Covey, S. (1989) The 7 Habits of Highly Effective People, Free Press.
- Porritt, V. (2009). Evaluating the impact of professional development. Education Journal, 116, 9–10.
- Guskey, T. (2000) Evaluating professional development. New York: Corwin Press.
- Earley, P., & Porritt, V. (2013) Evaluating the impact of professional development: the need for a student-focused approach, Professional Development in Education (https://www.tandfonline.com/doi/abs/10.1080/19415257.2013.798741)
