Problem

Background

In 2017, Oregon passed HB2355, mandating the development and implementation of a standardized, statewide collection of officer-initiated traffic and pedestrian stop data. From there, the Oregon Criminal Justice Commission (CJC) is tasked with the annual analysis of the collected data to determine if any disparities exist. This OKB summary is simply an overview of the five analytical techniques included in the CJC plan for Oregon stops analysis. To this end, this is not a replacement for the CJC’s white paper, STOP Program Research Brief (October 2018). The full research brief contains an in-depth background on the development of the analysis plan, including the strengths and weaknesses of the numerous analytical techniques available for examining stop data.

Research

Primary Analytical Challenges

1. Stops are dynamic interactions, with multiple decision points at which racially disparate treatment may or may not occur.
2. There are three competing explanations for racial disparities found in stop data: racially based policing, differences in driving behavior and/or offending by race, and exposure to law enforcement varying by race (this includes a where and how much a group drives).

Criteria For Inclusion

CJC used three criteria to choose the analytical techniques used in the analysis of Oregon stops data.

1. Technique must be evidence-based and conform to best practices for the analysis of stops data.
2. Technique must provide finding that are widely accessible, falling between robust, complex models and simpler, easy-to-communicate models.
3. Technique must use data that are reasonably easy to obtain and update. This is measured by ease of obtaining the data, costs associated with data, and frequency with which data need to be updated.
4. Technique must be selected after consideration of both its strengths and weaknesses.

Outcome

Stop Program Research Question 1: Are There Disparities Between The Stop Rates Of Different Races/Ethnicities?

Aggregate Stop Rates: Census Residential Benchmarking

What is it? This technique compares the stop rates of a particular area with the makeup of the residential population of the same area. In the STOP Program, the driver’s zip code is the data element that will help analysts determine whether a driver is a resident of a particular jurisdiction

Strengths: It is easy to obtain and update census data. This type of analysis is also widely understood, and often expected, by stakeholders. It is also relatively easy to construct benchmarks and make comparisons at multiple levels – state, county, and municipal.

Weaknesses: This type of population data does not necessarily reflect accurately the driving population at risk of encountering law enforcement.

Aggregate Stop Rates: Estimated Driving Population

What is it? This benchmark uses additional census information to “construct the demographic features” of the population commuting from other areas.

Strengths: As with any census benchmark analysis, the benchmark data are easy and inexpensive to gather. Additionally, the results of these analyses are accessible to stakeholders.

Weaknesses: This method is an imperfect measure of the driving population of a jurisdiction, as it does not account for out-of-area drivers that are in a town for shopping or recreation, including tourists. As with all census-based benchmarking, the census datasets include significant sample error. Oregon is a state where work commutes through multiple jurisdictions are commonplace and many towns welcome thousands of tourists every year, highlighting the weaknesses of benchmark analyses. For this reason, benchmark analyses are simply the beginning of a conversation between agencies and citizens.

Detailed Breakdowns Of Stop Rates: Veil Of Darkness

What is it? The Veil of Darkness (VOD) is “a comparison of traffic stops made in the daytime versus after dark, when race is not easily viewed” (CJC, 2018). The assumption of this analysis is that if no racial bias is occurring, then the daytime race distribution of stops (when driver’s race is most likely to be seen) should be similar to the race distribution at night (when the driver’s race is less likely to be seen). Since a group’s driving quantity could vary over a 24-hour period, CJC will focus on stops roughly between the hours of 5pm and 9pm, which switch between daytime and nighttime lighting depending on the time of year. This ensures that travel patterns, driving behavior, and exposure to police are similar between daytime and nighttime data.

Strengths: The two main strengths of the VOD are first, it does not require a benchmark and second, it addresses multiple variables in the calculation, such as clock time, day of week, and stop location. It is the closest to a “gold standard” analysis available.

Weaknesses: Recent studies demonstrated that driving behavior, which varies by race, might differ at night, which would add another explanation for disparities in stop rates.

Stop Program Research Question 2: Are There Disparities In The Post-Stop Outcomes Of Different Races/Ethnicities?

Propensity Score Matching

What is it? This type of analysis compares a group of interest to a “similarly situated” comparison group. Stops in each group exhibit the same distribution of observed stop features (time of day, location, reason for stop, driver age, driver gender, etc.) such that the only difference should be driver race.

Strengths: Analysts can control for several factors, other than race, which may influence the outcome of a stop.

Weaknesses: The data available to analyze is limited to the observed data collected by police officers, therefore any differences not captured by the observed data could affect results.

Outcome Tests

What is it? This type of test, “[seeks] to determine whether…an officer is more likely to search a minority driver, even if the likelihood of a successful search is lower than it would be for a non-minority driver.”

Strengths: Outcome tests provide easily interpretable results and account for differences in offending behavior.

Weaknesses: The primary weaknesses of outcome tests are associated with assumptions made about officers. These tests assume that officers have complete discretion over the decision to search a stopped individual and that the demographics of officers (age, experience, work experience, etc.) have no effect on an officer’s response to a given situation.

References And Resources

STOP Program Research Brief

Understanding Race Data from Vehicle Stops: A Stakeholder’s Guide

By the Numbers: A Guide for Analyzing Race Data from Vehicle Stops

Testing for Racial Profiling in Traffic Stops from Behind a Veil of Darkness