Five Levels of Professional Development Evaluation for Leadership Development
How do we know if PD is having an impact on student learning?
Content is adapted from an article by Tom Guskey.
We can use five hierarchical levels to evaluate PD effectiveness-with levels going from simple to complex. Each level builds on the one that comes before it. Successful implementation at one level is usually necessary for successful implementation at the levels that follow.
To determine the effectiveness of leadership PD we can measure:
Level 1
Evaluation: Participants' Reactions
Format: Participants' reactions are usually collected through a questionnaire handed out at the end of a PD session or activity. This is the most common form of PD evaluation and is the easiest to gather and analyze. This format often collects the "happiness quotient" or the degree to which a participant enjoys or is satisfied with the PD.
Level 1 Evaluation helps answer questions like:
Did participants like the PD activity?
Did they feel their time was well spent?
Were activities meaningful?
Was the leader or instructor knowledgeable and helpful?
Do participants believe that what they learned will be useful?
Level 2
Evaluation: Participants' Learning
Format: This can be a pencil-and-paper assessment, a simulation, skill demonstration, oral/written reflections, portfolio entry showing what the participant actually learned in the PD. Specific criteria and indicators of successful learning must be outlined before the PD.
Level 2 Evaluation helps answer questions like:
What did the participant learn from the PD?
What new skill or attitude has the participant gained from the PD?
What can the participant do now as a result of the PD?
Level 3
Evaluation: Organizational Support and Change
Format: This information is more complicated to gather. To do this, one may analyze records or meeting minutes. Questionnaires and structured interviews may be used to determine organizational support and change.
Level 3 Evaluation helps answer questions like:
Are district and school policies compatible with implementation efforts brought about by PD?
Did the PD affect organizational climate and procedures?
Was change the leader made as a result of the PD encouraged and supported at all levels?
Was the change brought about as a result of the PD aligned with district/school mission?
Was administrative support public and overt?
Were resources sufficient?
Level 4
Evaluation: Participants' Use of Knowledge and Skills
Format: Information is gathered based on clear indicators that reveal the degree and quality of implementation resulting from the PD. One may use questionnaires, structured interviews with participants/supervisors/peers/subordinates, personal reflections, portfolios, and direct observations. Measurement occurs after the PD and over several intervals since these measures show progression of learning and implementation.
Level 4 Evaluation helps answer questions like:
Are participants using what they learned?
Are participants using what they learned well--at a high level?
Level 5
Evaluation: Student Learning Outcomes
Format: Measures of student learning (assessment scores, portfolio evaluations, grades, cognitive indicators, affective (attitudes and dispositions) and skills and behaviors. Examples include assessments of students' self-concepts, study habits, school attendance, homework completion rates, or classroom behaviors. School-wide indicators such as enrollment in advanced classes, memberships in honor societies, participation in school-related activities, disciplinary actions, and retention or dropout rates also might be considered. The major source of such information is student and school records. Results from questionnaires and structured interviews with students, parents, teachers, and/or administrators also could be included.
Level 5 Evaluation helps answer questions like:
What was the impact on students?
Did the learning from the PD program benefit students in any way?
What is the PD program's overall impact?
Guskey notes that evaluation at any of these five levels can be done well or poorly. The information gathered at each level is important and can help improve PD, but many have discovered, tracking effectiveness at one level tells nothing about impact at the next. Although success at an early level may be necessary for positive results at the next level, it is clearly not sufficient. That is why each level is important. Sadly, the bulk of professional development today is evaluated only at Level 1, if at all. Of the rest, the majority are measured only at Level 2.
