How should we grade our schools?

Last week I got two questions via social media that ask variations of the same thing: How can we understand how our schools are performing relative to other schools in our area or across the country?

One questioner simply requested: "Please grade APS as a school district."

The other asked, "APS High Schools (Yorktown, W-L, and Wakefield) trail nearby Fairfax schools (Langley, McLean, Marshall, Woodson, Oakton) in SAT scores and all school ranking services (Niche, Great Schools, US News). What do you think APS should do within the next 3 years to remedy that deficit?"

School ranking services like Great Schools and Niche pull data from sources like the Virginia School Quality Profiles. (Under federal education law, each state must offer some type of publicly-available "school report cards" and report certain indicators to the US Department of Education.) In Virginia, we can learn a lot about our schools from their School Quality Profiles, including how each school's students perform on the Standards of Learning assessments, how many students are completing advanced coursework and CTE credentials, the percentage of students going on to college, per-pupil spending, the number and type of disciplinary actions taken, teachers' education and experience, and kindergarten readiness.

When I look at APS high schools relative to nearby high schools in Fairfax County, using the Virginia School Quality Profile data here's what I'm seeing:

​These data show what I would expect: schools that have a greater percentage of students from more affluent families generally have more students graduating with advanced diplomas and going on to four-year colleges and universities. I believe this has more to do with the resources these students and their families are able to access than something exemplary happening in certain school classrooms. Middle- and upper-class families are able to foot the bill for many "extras" including:

• Private tutoring and out-of-school enrichment classes and camps
• Private evaluations of, and therapies for, learning disabilities and physical and mental health conditions that can affect performance in school
• Extra SAT/ACT prep courses and private college admissions counseling
• Quality early childhood care and preschool.

​Do these things influence how a student does in school? You bet. Does this mean that it doesn't matter what schools do because the outcomes are predetermined? Absolutely not--but it does mean that indicators of school performance, like those shown in the chart above, are far more interesting when we spot schools that are bucking the trend. For example, the high school in Harlem that has zero dropouts, an 87% college enrollment rate, and a student population that is 77% economically disadvantaged. Or the high school in Oakland whose students are 100% English learners, most having immigrated to the US within the past four years, and 91% economically disadvantaged. 99% of these students are taking classes required for admission into the state college (UC/CSU) system. 

I also want to note that it matters who is asking the question, "How good is this school?" What's a good fit for one student may not be great for another. For example, if I'm a parent of a student with disabilities, I'm going to want to filter the data in the Virginia School Quality Profiles so that I can focus on how well each school engages and teaches students with disabilities. If I do that, I'll see the following:

These data suggest that it might be Yorktown, not Langley, that's the better performing school for students with disabilities: a greater percentage of these students are earning advanced diplomas at Yorktown and achievement on the English Reading and Writing SOL tests are on average higher. (Note that you can also filter Virginia School Quality Profiles to focus on other groups including students who are English learners and students of various races and ethnicities. In my "not-a-professional-data-scientist" review, none of these correlated as closely with overall school performance as did family income.)

Finally, I hope that legislators, education leaders and families will ask for and use more holistic and detailed measures of student success. My favorite "school report card" system is the New York City Department of Education's School Quality Snapshot dashboard. There I can find a wealth of data on each school, including things I can't track in Virginia like:

• Percentage of 9th and 10th graders who have earned enough credits to be on track for graduation
• Graduation rates as a function of 8th grade achievement test results
• Percentage of students who have enrolled in different types of colleges and universities (two-year, four-year, in-state, out-of-state, public, private)
• School culture survey data from staff, students and families about rigor of instruction, effective educational leadership, staff collaboration, safe and supportive environment, school-family relationships, and trust
• Movement of students with disabilities into less restrictive environments (inclusion)
• Teacher demographics and attendance.

​The NYC School Quality Snapshots are something the district chose to do on top of their state-mandated reporting. How cool would it be if APS and other school districts offered something similar?

The ability to expand our definition of "success," transparently report our strengths and challenges, hold ourselves accountable for steady progress, spot the outlier schools that are exceeding expectations in some way and work to scale what they're doing right... that sounds like a great school system to me.
​