You Earned a Ticket!

Which school do you want to support?

Lesson 1.6

Progress:
Are Schools Improving?

Are schools getting better? The answer may surprise you.

hero image

The last few lessons discussed the evidence that too many of California's students are learning too little in school, especially relative to those in other states and nations. Despite the scale of the challenges, there is also good news.

Two decades ago, it was very difficult to credibly identify high-achievement schools in high-poverty and high-minority settings. Today, it is easy to find them. These schools prove that children’s destinies are not coldly and totally predetermined by poverty and ethnicity. These exceptional schools even have an acronym in education reform speak: “Beating the Odds,” or BTO schools.

Yes, schools are improving

Over the long run, there is strong evidence that educational achievement is slowly improving for students in all subgroups, all over America, including California. Test scores are rising. Graduation rates are improving. College-going rates have risen. Disciplinary cases like expulsions and suspensions are becoming more rare.

As described in Ed100 Lesson 1.1, America's best tool for directly measuring educational progress is the NAEP test. (The acronym NAEP stands for National Assessment of Educational Progress). Its purpose for existence is to enable meaningful statistical comparisons over time and place. Over the long haul, NAEP scores have been rising, with big gaps among racial and ethnic groups:

Poverty plays a huge role in the measurable gaps in student learning. Over time, scores have been rising for students from both richer and poorer families, according to the NAEP results.

Scores on the NAEP exams can help describe long-arc changes in the academic capabilities of America's students. They can quantify differences in academic readiness between groups of students. On their own, though, the scores are thoroughly abstract. Reactions to the raw scores generally falls somewhere in the boredom triangle between "uh-huh," "so what?" and "I'm sorry, were you talking to me?"

Cut-scores: Magnifying Little Changes

To give the scores some meaning, for each test the NAEP governing board defines a "cut-score" that it labels as "proficient." Students above the cut-score are proficient. Those below it aren't. The proficiency cut-score for each grade level is set about ten scale points above the grade beneath it, which is the source of a widely-used (but disputed) approximation: a ten-point difference in a NAEP score is equivalent to a year of education.

Cut-scores can help make conversation about test results easier, but they can be misused, as we describe in Lesson 9.6. The main problem is that they tend to exaggerate changes. Students know about cut-scores from grades: a score of 79 isn't so different from a score of 80, but one is a B and the other is a C. When a normal distribution curve shifts across a fixed cut score, even tiny shifts can seem like a big deal, depending on where the cut-points have been placed.

Here's another example. For a decade, California used a set of state tests known as the CST, bundling the results together to create a score known as the Academic Performance Index (API). For each test, a score in the right "tail" of the bell curve was fixed in place as the "proficiency" target.

Over time, student scores on the CST improved a bit, and the whole bell curve shifted to the right. Because the proficiency benchmark was fixed in place, small gains could produce the impression of eye-popping improvement. For example, over the ten year period the proficiency rate of socioeconomically disadvantaged students more than doubled on the CST.

Source: CA Department of Education Source: CA Department of Education Source: CA Department of Education.

Improvement is Normal. And Difficult.

Statistical exaggeration notwithstanding, the evidence suggests that California's kids are learning more... just like kids all over America and the world. Slow, widespread improvement in education results is normal. Despite recent progress, California remains at or near the bottom of the pile in national and international comparisons. For California's students to thrive, they have to do more than keep up.

Grading schools with a single "API score" glossed over a lot of detail. In 2017 California replaced the API with the California School Dashboard.

In search of improvement, from about 2013 California began a set of sweeping systemic changes in public education.

  • Academic standards were overhauled with the adoption of the Common Core State Standards, discussed in Lesson 6.1.
  • To support these new standards, the state rolled out new state tests discussed in Lesson 6.5.
  • The state dumped its academic accountability system, the Academic Performance Index (API), in favor of a more sophisticated performance dashboard approach explained in Lesson 9.7.
  • The state replaced a highly inequitable system of school funding with a more equitable one by adopting the Local Control Funding Formula, explained in Lesson 8.5.

In 2018, the Getting Down to Facts II project assembled research to examine all this interrelated change. The early evidence is both encouraging and sobering. Based on a national comparison titled A Portrait of Educational Outcomes in California "...once students begin school, their academic achievement increases slightly faster in California than the average nationwide. Other than in the richest districts, growth rates in California are slightly greater than the nation."

Now some not-so-good news: According to the report, the results in California haven't been so great for students from less-well-off families. "Though California had been making progress in the last decade among low, average, and high SES [socioeconomic status] districts, data from 2015 show a reversal of that progress in poor and middle-class districts from 2013 to 2015. Educational opportunities in small towns and rural communities are particularly lacking.

Background: What's the Score?

For many years, California bundled test results in various subjects and at different grade levels into a single annual school-wide statistic called the Academic Performance Index (API). The API system was abandoned when the state adopted the Common Core Standards and began using the CAASPP tests instead of the STAR tests.

Grading schools with a single number obviously glossed over a lot of detail. After years of debate, in early 2017 California shifted to a new, broader way of evaluating success for schools and districts, the California School Dashboard. The Dashboard measures schools in a variety of ways to show both current conditions as well as momentum. For more on this topic please see our Ed100 blog series on the California School Dashboard. Visit the Dashboard to find your school's results.

Developing the Dashboard proved difficult not only because of disagreements about what should be included, but limitations in what data the state has the capacity to collect. California's education data systems are notoriously weak. Because it includes multiple measures, the scorecard makes it difficult to rank schools in a simplistic way.

The impetus to score and rank schools originated, in part, from federal requirements under the No Child Left Behind (NCLB) Act. With the 2015 passage of the Every Student Succeeds Act (ESSA), the federal requirement was softened: states became obligated only to identify the lowest-performing 5% of schools.

Beating the Odds

Since the passage of No Child Left Behind, and continuing today, a key function of school accountability systems (like the California School Dashboard today, or the API system before it) has been to calculate differences in results for "subgroups." For example, if there are a significant number of students of a specific ethnicity in a school or district, the Dashboard shows the results for that subgroup. Differences among subgroup scores can suggest where a school or district ought to focus its attention. Like it or not, these differences also deliver a bit of "shame" motivation for schools to remember all of their kids when evaluating their success.

But shaming isn't the only reason to look at the subgroup scores: they can also show reasons for pride, and for learning, because some schools "beat the odds."

Schools that beat the odds are important, because they show that the kids are not the problem. They prove beyond doubt that, given the right support, every child can learn. They provide evidence that investing in kids really can make a difference, even where it seems hard. They convert the doubters' claim that "those kids can't learn" to "those kids won't learn unless...

The difference is important.

What are the ideas, approaches, programs, interventions, investments, and inspirations that can lift California’s student achievement from the bottom of the developed world, boost economic growth and make a crucial difference in children’s lives and America’s future?

Answering that question is the focus of the lessons ahead. But first, let's back up a little. Education isn't all about scores, or even about economic competition. What is it for? And has our view of education's purpose changed over time?

Updated May 17 2017.
Updated extensively Oct 2018.

Review

Which one of the following statements is NOT true?

Answer the question correctly and earn a ticket.
Learn More

Questions & Comments

To comment or reply, please sign in .

user avatar
Carol Kocivar June 18, 2018 at 7:57 am
Does a recession have an impact on student progress?
Two recent studies take a look at the impact of the Great Recession and student achievement.
They come to similar conclusions: Cuts in education funding matter. And they seem to matter most for low income students.

The Impact of the Great Recession on Student Achievement: Evidence from Population Data
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3026151

Do School Spending Cuts Matter? Evidence from the Great Recession
https://works.bepress.com/c_kirabo_jackson/35/download/
user avatar
Steven Davis June 11, 2018 at 4:00 pm
Please add data here and throughout on special needs kids!
user avatar
David Siegrist1 November 15, 2017 at 8:29 am
And Asian stydents’ academic performance?
user avatar
Lisette October 3, 2017 at 1:05 pm
Could it be that schools are so concentrated on teaching our children how to pass these standardized tests rather than concentrate on their long-term goal of a child's education?
user avatar
May 3, 2017 at 11:29 am
Too much restriction and policy on how school should use the funding to education the students.
user avatar
Jeff Camp - Founder April 28, 2015 at 1:43 pm
The charts in this page are visited again in Lesson 9.6, where we make an important point: "By the nature of statistics, if a cut-off point stays put while a curve moves, metrics can exaggerate the scale of the change if the cut point is anywhere close to the steep part of the curve. Education statistics are loaded with metrics derived from cut-off scores."
user avatar
tonyammarquez April 28, 2015 at 1:39 am
I'd like to know who is in charge of evaluating and\orregulating the tests that are referred to in Ed100?
user avatar
Jeff Camp - Founder April 28, 2015 at 10:59 am
Thanks, Tony. Ed100 refers to many tests (there are a lot of them, explored in lesson 9.3!) In this lesson (1.6) the focus is on the NAEP tests. The NAEP tests are overseen by an appointed, independent governing body of 26 members. http://nces.ed.gov/nationsreportcard/about/ The members I have met have been passionately committed to the work of giving America a meaningful, useful barometer of student learning throughout the country, and have understood the statistical challenges of doing this work properly. These tests are NOT the ones that everyone has to take -- they are administered only on a "sample" basis as a tool for understanding what's really happening.
user avatar
sylviambee April 7, 2015 at 11:38 am
My son is academically advanced compared to me at that age. Of course there are some similarities and differences that contribute to this matter. My son attends public school, I attended private, Catholic schools for 9 years. I earned a BA before my son was born, and completed a Masters program before he was 11. My mom earned her BA when I was 13. Both my parents were immigrants and had to learn English, only my son's father was an immigrant and had to learn English. My parents were required to volunteer at my school, but usually outside the classroom. My husband and I volunteered inside and outside of the classroom. I was a stay at home mom when my son was in grades 2-6, my parents worked full-time. I had babysitters and Was a latch-key kid by age 8. My son has always been picked up from school, and only until this past year does he walk to a nearby after-school site. There are really so many factors leading to the reasons why my son is academically advanced compared to my experiences.
user avatar
jenzteam February 27, 2015 at 6:57 am
I'm not a huge fan of test scores, however I do acknowledge that they are needed to track progress. That said, I also believe that students should also be given opportunities to demonstrate their knowledge in other ways besides a standardized test. The focus of districts is always on scores, not on the overall education experience and how it is preparing kids for the real world. I don't TEST my employees - I watch how they interact, work collaboratively, and contribute to the overall health of the company. I look at their interpersonal relationships with their colleagues and their commitment to developing their skills. Educational systems would benefit from using a business model if they want to prepare students for real world work environments.
user avatar
tonyammarquez April 28, 2015 at 1:43 am
I'm glad you said that very good point!
user avatar
Sherry Schnell January 22, 2015 at 9:31 am
A big part of helping high-poverty students succeed is to eliminate (or ameliorate) the impact of poverty on these children - i.e. stable housing and electricity, nutritious and dependable food, quality childcare outside of school hours, reliable transportation to school, etc. Is this really the job of the school or is it the responsibility of other public institutions? In my opinion, schools should be responsible for "educating" and we need to step up other programs to deal with poverty.
user avatar
Jeff Camp - Founder January 22, 2015 at 9:22 pm
Thanks, Sherry -- most of the topics that you touch on here are examined in more depth in chapter 2, which focuses on students and what they bring with them (for good or ill) that affects their learning. Efforts to integrate school with "wraparound services" are explored in lesson 5.7, on "community schools".
user avatar
Lauren Dutton March 10, 2011 at 7:06 am
Indeed, throughout California there are shining examples of outstanding "beating the odds" public schools. Located directly in some of our state's most under-served communiies, they are doing whatever it takes to ensure that their students are on the path to success in college and in life. In turn, these schools are demonstrating that ALL students can succeed at high levels and deserve our highest expectations. It is up to us as adults to figure out how to ensure that these are not isolated examples but that we learn from and scale these approaches.

There are many examplars, but as the post mentions KIPP is one strong example with 12 schools in the Bay Area and Los Angeles. Check out the comprehensive Mathematica study on their national results here: http://www.kipp.org/mathematica
©2003-2018 Jeff Camp
Design by SimpleSend

Sharing is caring!

Password Reset

Change your mind? Sign In.

Search all lesson and blog content here.

Sign In

Not a member? Join now.

or via email

Share via Email

Join Ed100

Already Joined Ed100? Sign In.

or via email