What is success? How do we know when we’ve achieved it? When speaking of success in terms of education there must be several “yard sticks” in play in order to get a fair and accurate reading on the pulse of our schools locally, statewide, and nationally.
“Ranking schools is a hollow, misleading exercise that can do more damage than good,” say John Gulla (Edward Ford Foundation) and Olaf Jorgenson (Almaden Country School, CA) in this article in Independent School. But they agree that educators can’t “reflexively dismiss all efforts to apply some elements of quantitative analysis to our work.” They list nine measures of school effectiveness – three traditional and six more recent. Selecting judiciously which of these to use, say Gulla and Jorgenson, might give a good sense of a school’s quality:
• Parent satisfaction – This includes the level of parent demand, a low rate of attrition, and positive ratings on a well-constructed parent survey. As important as giving a survey is how well the results are used. “If a measurement matters at all,” says information expert Douglas Hubbard, “it is because it must have some conceivable effect on decisions and behavior. If we can’t identify a decision that could be affected by a proposed measurement and how it could change those decisions, then the measurement simply has no value.”
• Standardized testing – Standardized multiple-choice tests can be seen as measuring (in the words of John Austin of King’s Academy in Jordan) “the educational equivalent of factory work” – better suited to the Industrial Age that faded decades ago. Far more valuable, say Gulla and Jorgenson, are SAT II, AP, and IB tests with their open-ended and creative-response questions. The key data to watch for are trends over time and the value a school adds to students’ entering achievement levels.
• Accreditation – A thorough audit can provide valuable insights on a school’s effectiveness as measured against its mission and aspirations, in the context of its resources and capacities. But a lot depends on the quality of the visiting team and the school’s self-study.
• Value-added assessment – The College and Work Readiness Assessment (CWRA+) is designed to measure how much a school contributes to a student’s 21st-century skills from freshman to senior year – critical thinking, analytic reasoning, problem solving, and ability to communicate in writing. This assessment is a “compelling alternative” to the limited range of the SAT I, say Gulla and Jorgenson.
• Student surveys – Well-crafted questionnaires can quantify students’ level of engagement, providing important insights on how students feel about their school’s purpose, relevance, rigor, and challenge, as well as relationships, support, and connectedness. The High School Survey of Student Engagement (HSSSE – see following article) can provide information that is “[p]otentially far more institutionally useful than data from standardized achievement testing,” say Gulla and Jorgenson.
• Benchmarking – The key factor in a meaningful analysis of schools is comparing apples to apples and not rank-ordering, say the authors. “Healthy benchmarking collectives… are characterized by member school commitment to collegiality and use of data to benefit the greater good,” they say.
• Data dashboards – These corporate-inspired tools for displaying a school’s “vital signs” in a user-friendly format can include enrollment and re-enrollment data, budget information, student and staff attendance, course selections, test results, college placements, post-graduate achievement, and more. “Still, much of what we do in schools cannot be easily measured and is not always subject to quantitative analysis,” say Bulla and Jorgenson. “Variables including curiosity, resilience, self-control, and determination – what we sometimes refer to as personality traits or simply ‘character’ – are even more critical than measures of IQ or academic achievement in determining how and why children succeed.”
• Longitudinal alumni surveys – “Great schools are measured not by the accomplishments of their students, but by the lives led by their alumni,” said Michael Chun, past president of Kamehameha School in Hawaii. John Austin agrees: “Are they active and involved in their communities? Have they put their own educations to work in the service of others? Are they doing what Howard Gardner and his team at Harvard call ‘good work’ – work that is excellent in quality, socially responsible, and meaningful to its practitioners?” Insights on questions like these can be gleaned by periodically surveying graduates and asking them for critical reflections on the school and how well it prepared them – or didn’t prepare them – for occupations, relationships, collaboration, leadership, self-advocacy, and coping.
• A senior survey – Harvard professor Richard Light is developing an interview protocol for graduating seniors designed to elicit information about both student life and the academic program.
• The Mission Skills Assessment – This assessment is designed to measure middle-school students’ curiosity, teamwork, resilience, ethics, and time management. Comparing data with other schools might provide valuable insights on the most effective practices in developing these vital life skills.
This is an exhaustive – and exhausting – list of possible measures of school effectiveness, conclude Gulla and Jorgenson. The trick is to select the measures that most accurately and efficiently measure what a school wants to do. “As school leaders, we must be courageous, resisting the significant pressures to commodify education as a neatly defined, measured set of metrics and program outcomes,” they say. “At the same time, we do our students and ourselves no favors if we reject all efforts to measure the value we know we deliver.”
“Measuring Our Success: How to Gauge the ‘Value Added’ by an Independent School Education” by John Gulla and Olaf Jorgenson in Independent School, Spring 2014 (Vol. 73, #3, p. 28-36), www.nais.org