Some childhood gaps are welcome news, like that gap-tooth smile. Achievement gaps? Not so much.

What Is an Achievement Gap?

California's student body is officially organized into groups by grade level, but it can also be seen as consisting of "subgroups" based on race, economic wellbeing, English language readiness and gender. Frustratingly stable patterns predict which subgroups of students tend to do better in school and which tend to do worse. These measurable differences, known as "achievement gaps," are usually quantified using standardized test scores. Gaps in students' academic readiness can also be measured in other ways, such as graduation rates, college-going rates and course grades, but in the education system, test scores are the currency of the realm.

There are some good reasons to measure achievement gaps using standardized test scores. They are widely used and understood. They can reveal small differences or trends. They are comparatively quick and inexpensive. They also seem to correlate powerfully with less convenient measures. Students with higher scores are more likely to persist in school, graduate, go to college and succeed there.

Poverty strongly correlates with test scores. Students who qualify for free or reduced-price lunch service (a common measure of poverty) tend to score lower on tests than the kids who don’t. Race correlates, too: on average, black and Latino students tend to score lower on standardized tests than white and Asian students do.

How are Gaps Measured?

Describing patterns of difference turns out to be more difficult than it might seem, even using test scores. After a student takes a standardized test like the CAASPP, California's state tests, a complex process grades the test using a "scaled score". It's more than just a simple measurement of the number of questions that you get right or wrong. Some questions are harder than others, so they are worth more than easier ones. Some questions are literally worth nothing because they are experimental.

A test score without context is meaningless, so test designers identify performance tiers, assigning cut-points that indicate whether students have met expectations. California's system separates scores into four tiers.

Let's suppose that you and your twin sister each take a test, say the grade 5 CAASPP test in mathematics. Your sister scores 2,527 scaled points. You score 2,528 -- one point higher.

Your parents receive two envelopes in the mail with the score reports. Tearing open the first envelope, with warmth in their heart they learn that your score indicates "Standard Met." They open your sister's envelope, and the warmth fades. Alas, your sister's score falls short -- it lands in the "Standard Nearly Met" range.

Of course, the one-point "gap" between these hypothetical scores is insignificant from a statistical perspective. If you both retook the test, your scores would probably be similar but not the same. Like free throws in basketball, there's skill involved, but a bit of randomness, too.

Randomness becomes less important when small samples like this are combined in larger numbers. The close calls on either side of the line tend to cancel one another out. Unhappily, the statistics show that there are some big gaps.

How Big are the Gaps?

They're big. Here's a comparison of CAASPP scores by group for English Language Arts:

The gaps are big for math, too:

Are Scores Improving?

Starting in 2002, the CST showed rising scores for 10 years, for all groups.

Yes. Over time, the trajectory of California students' scores on standardized tests has been upward. Slowly but significantly, annual successions of students at each grade level are out-scoring their predecessors.

Scores on the CAASPP tests have risen, but the tests have only been around for a few years. It's easier to see the long-term trend by looking at the test results on the "CST" tests that preceded it. For all subgroups, scores rose for ten years.

Source: CA Department of Education. Click for additional detail

Are We Sure?

Yes, pretty sure. According to California's state tests, scores have been rising over time for all groups of students. The improvement is corroborated by rising graduation rates. The federal NAEP test shows long-term improvement, too.

Have Achievement Gaps Shrunk?

Are scores rising in a way that reduces achievement gaps? Perhaps not-- or at least not lately. According to the Public Policy Institute of California, "achievement gaps persist and have widened in some cases." What can be done about it? EdSource launched an online conversation about it in 2015, inviting views about what should be changed. Spoiler alert: it does't seem there's a ready supply of easy answers.

Test scores are going up /
and that is great
good news /
But not enough /
to disrupt /
the gaps
between the groups

When looking at any chart that compares "proficiency" rates, squint a little. Remember, individual scores are pesky, detailed things. A little bump can push a score over the cutoff line between one achievement category and the next. Cutoff points are defined by fiat -- they are supposed to represent a standard of achievement, not a relative position within a curve.

By the nature of statistics, if a cut-off point stays put while a curve moves, metrics that rely on the cut-off point will have a tendency to exaggerate the scale of some changes more than others. If the cut point is anywhere close to the steep part of the curve, a little bit of change can cast an exaggerated shadow. Education statistics are loaded with metrics derived from cut-off scores.

Bell curves have two tails, and some critics argue that reducing gaps by boosting the learning of kids at the bottom of the learning curve isn't enough. They argue that long-term American competitiveness is at risk if challenging kids at the "top of the curve" isn’t also encouraged as part of a school’s mission.

The success of California’s schools, districts, and public education system as a whole are often judged by how effectively they address these predictable gaps in achievement. The next lesson discusses the ways these systems’ successes are measured.

Updated September 2017