Standardized Test Analysis

Background

I was a private school kid.

My younger sister was too.

My sister and I as kids.

To be clear, though, my family wasn’t rich.

We were working-class, but my parents thought the sacrifice was necessary to get my sister and me ahead.

Unfortunately, that’s the kind of calculus a lot of parents have to do these days. They sacrifice financially for the good of their children.

That never quite sat right with me then, and it doesn’t sit right with me now.

The question is, though, did my parents have to do that? 

Was I better off with private school education, or would I have been fine with a public school one?

That’s an issue many in California have had to ask themselves throughout the years. With the pandemic, however, a little less so. 

That’s because, at the start of the pandemic, the University of California temporarily stopped using standardized test scores as part of its admissions process. 

That decision has faced some pushback.

Wall Street Journal

On one hand, many question how can schools ensure the best students are admitted without a standard way to evaluate them all.

Sizable numbers of first-generation, low-income and “underrepresented minorities,” such as Black, Hispanic and Native American students, have earned admissions to the University of California due “solely by virtue of their SAT scores.”

Others however claim that they’re unfair and harmful to diversity.

Colleges that have gone "test optional" enroll — and graduate — a higher proportion of low-income and first generation-students, and more students from diverse backgrounds, the researchers found in the study, Defining Access: How Test-Optional Works.

VIEW ON GITHUB

Datasets:

Libraries:

matplotlib, numpy, pandas, seaborn, and sklearn.

Problem Statement

Is standardized testing an equitable measure for student performance?

The Data

To answer this question I drew upon three data sources.

  1. California SAT and ACT scores from 2019 and 2020.
  2. California school district racial demographic data that I compiled myself via DataQuest. DataQuest is the California Department of Education’s web-based data reporting system for publicly reporting information about California students, teachers, and schools.
  3. 2021 Study of the Most & Least Equitable School Districts in California.

So, are standardized tests unfair?

Yes.

But not to the extent one might think.

Initially, I thought this would be a slam dunk, but while there is evidence, it wasn’t as strong as I originally thought it would be.

Let’s take a look at the percentage of students with ACT composite scores over 21:

2020 California ACT results.

There’s a slight relationship between income and strong scores, but it’s not as strong as you’d imagine it to be if it had an overwhelming impact.

It’s a similar story, though the relationship between income and strong scores is a little strong here:

2020 California SAT results.

These test results are from 2020.

How about participation rates?

Well, there is some connection with the ACT: 

It's not very strong, but you can see a trend.

As income rises, so do participation rates.

But the SAT isn’t as clear cut.

The only trend here seems to be there isn't one.

There seems to be a tiny relationship, but not one you’d draw any grand conclusions from.

Still, on the whole, the evidence seems pretty clear that there is a relationship between income and standardized test success.

So, we should scrap standardized testing, right?

Well, that depends?

Simply scrapping standardized tests won’t make colleges more diverse.

A study of 180 liberal arts colleges found that test-optional policies did not increase the numbers of low-income and minority students.

On the other hand:

So, while the data leads me to believe that standardized tests unjustly favour certain parts of the population, if the only thing you do to address that is remove them, campuses will look no different than they do today.

Back to Projects Page