A Brief Response to John Pfaff’s Review of “Mass Incarceration Nation”

Original Review: John Pfaff
Publisher: (Mass Incarceration Nation) Cambridge University Press, 2022. 234 pages.
Reply: Jeffrey Bellin | January 2024

I offer this response to John Pfaff’s review of Mass Incarceration Nation: How the United States Became Addicted to Prisons and Jails and How It Can Recover (Cambridge 2023) for two reasons. (1) Professor Pfaff leaves out important context, failing to mention that his review is part of a long-running exchange about the causes of mass incarceration that began when I wrote a critical review of his book on this topic. (2) Pfaff claims to have identified errors in the book and, in light of these errors, urges readers to approach the book’s findings with caution. Yet, as I point out below, Pfaff’s claims of error are wrong and easily refuted.

There is food for thought in Pfaff’s review which includes distinct points of emphasis drawn from his own work, but the ultimate takeaway should be the opposite of what he intends. Rather than “approach[ing] the book’s empirical claims with significant caution,” readers can be reassured that an empirical scholar pored over the book looking for errors and found none.[1]

The Relevant Context

Mass Incarceration Nation provides a “precise diagnosis of the causes of Mass Incarceration” as a necessary step toward undoing the phenomenon.[2] It reveals that there are two overlapping systems filling America’s jails and prisons, a “criminal justice system” that offers a retributive response to crimes like homicide and sexual violence, and a “criminal legal system” that targets behaviors like drug abuse, weapons possession, and drunk driving in a (largely unsuccessful) effort to deter them. This dichotomy informs one of the book’s key findings. While part of the story of the rise of mass incarceration is longer sentences for “criminal justice system” offenses, an equally important part involves a pronounced turn toward incarceration for “criminal legal system” offenses (ostensibly) to achieve policy goals.

The book also shows that, contrary to the claims of other commentators, mass incarceration in the United States has no single cause. The phenomenon was the product of remarkable consensus, involving not just bipartisan legislative cooperation, but also substantial contributions from police, prosecutors, judges, and other officials. Drawing on my own experience working in the criminal courts and voluminous data, the book “catalog[ues] the ways that the system’s mechanics – primarily police, prosecutors, and judges but also probation and parole officials – worked together to increase the number of people flowing into the system and the punishments they received.”[3]

Anticipating that Mass Incarceration Nation would be closely scrutinized, I emphasized transparency in the book’s introduction, which explains:

“Wherever possible in the pages that follow, I test my arguments against the historical data and lay out that data for readers to draw their own conclusions. To enhance the transparency of the presentation and allow skeptics to check my claims, I rely as much as possible on the most widely accepted, official, public data sources” (p. 8).

True to this commitment, Mass Incarceration Nation presents its findings alongside the data that support them. While the narrative often summarizes to make larger points, charts and graphs accompany those points illustrating precisely what the data show. And since all the data is publicly available, any errors are easy to uncover.

This approach was designed to avoid missteps that I had seen in other writing on this topic, including in Fordham law professor, John Pfaff’s 2017 book, Locked In: The True Causes of Mass Incarceration-and How to Achieve Real Reform. Locked In criticized Michelle Alexander and others who told a “Standard Story” about mass incarceration.[4] But Locked In’s novel contribution was its identification of the “one thing” that, Pfaff contended, explained “most of” the rise of American mass incarceration: “the rate at which prosecutors filed felony charges against arrestees.”[5] Pfaff wrote in Locked In that he derived this insight from data from the National Center for State Courts and, specifically, a “dataset, sitting in plain view on an NCSC server but apparently overlooked by all the studies before my own.”[6] He later clarified, however, that his finding was derived from an unpublished “internal spreadsheet” emailed to Pfaff by a NCSC analyst.[7] Relying on this spreadsheet, Pfaff concluded that, with respect to incarceration, “The only thing that really grew over time was the rate at which prosecutors filed felony charges against arrestees.”[8]

I published a review of Locked In in 2018.[9] It criticized the book’s prosecutors-only thesis on two broad grounds. With respect to Pfaff’s empirical finding, I pointed out that Pfaff’s claim that nothing else changed over the period of mass incarceration conflicted with evidence that other things, such as sentence lengths, changed over time.[10] In addition, after consulting with the NCSC, I highlighted problems with the data Pfaff relied on, including a major revision in how the NCSC collected court data in 2003 that inflated filing numbers in the middle of Pfaff’s (already limited) 1994-2008 data range.[11] (Sociologist Katherine Beckett later echoed and supplemented these critiques.)[12]

Second, my review explained that while prosecutors are important, they are part of a complex ecosystem, controlled by legislators and influenced by police who initiate cases, and judges who preside over court proceedings and sentences. I explained that “While prosecutors can unilaterally open exits, it takes a village to incarcerate someone; and when it comes to incarceration, the criminal justice village is full of figures with as much or more power than prosecutors.”[13]

Pfaff published a response in the Michigan Law Review that he claimed was necessary “to correct a significant error [Bellin] makes when discussing some of my data.”[14] Despite acknowledging some missteps,[15] the theme of Pfaff’s response was that my critique was “simply wrong.”[16] The next public entry in this saga was an essay I was invited to contribute to a California Law Review Symposium debating the abolition of prosecutors. There, I criticized proposals premised on Pfaff’s claim that mass incarceration was caused solely by prosecutors, for failing to address the contributions of legislators, police, and judges.[17] That essay drew on the research I had done for Mass Incarceration Nation. And while the book does not focus on Pfaff’s claims specifically, it resolves our underlying debate. Among other things, the book lays out the evidence that, while prosecutors played a role in the rise of mass incarceration, their contribution unfolded in parallel with those of other official actors – all of whom, the data show, substantially contributed to mass incarceration by embracing penal severity after the 1970s.

Pfaff’s review of Mass Incarceration Nation published in Criminal Law and Criminal Justice Books in October 2023 is the latest chapter in our discourse. But Pfaff’s review does not disclose any of the history summarized above. Readers are not told, for example, that the author of the book Pfaff was reviewing wrote a critical review of Pfaff’s own book about mass incarceration. More significantly, Pfaff does not reference our substantive disagreement about whether mass incarceration was caused by a single change in prosecutor filing practices (Pfaff’s view) or a combination of changes initiated by legislators, police, prosecutors, judges, and other officials (my view). Instead, he pivots to critiquing “criminal legal system data” generally, “demand-side-only analysis,” and recklessly lobbing allegations of errors.

The Alleged “Errors”

Most of Pfaff’s review concerns the standard differences of opinion that one can expect on any complex topic. And while I am happy to debate these points in another forum, I focus here on the one paragraph in Pfaff’s review that prompted this response. That paragraph begins,

“I think it is important to note that multiple times the book makes empirical assertions that are simply untrue, or discusses the numbers in ways that obscure important patterns.”

It ends with:

“Taken together, these sorts of errors unfortunately suggest that readers should approach the book’s empirical claims with significant caution.”

Between those sentences, Pfaff offers five examples, each of which I quote and then refute below.

“Similarly, the book asserts that arrests did not fall over the 1990s even as crime did, a claim that is clearly incorrect (all sorts of arrests fell by a lot).”

I begin with this charge because the change in arrests over time is a theme of Mass Incarceration Nation. There, I point out the large jump in arrests as mass incarceration gained steam, with police making 4 million more annual arrests in 1990 than they had in 1980.[18] I also show that while crime plummeted in the 1990s, arrests did not fall in tandem with the crime drop. Instead, as I explain, police kept arrests near or even higher than 1990s levels by pivoting to “criminal legal system” arrests for easy-to-detect-and-prove crimes, like drugs, drunk driving, and weapons possession. To flesh out this point, I offer lots of data-driven examples, such as: “comparing 1990 to 2006, annual robbery arrests fell by 42,000 and forcible rape arrests fell by 15,000, while drug arrests increased by 800,000!”[19]

In a footnote, Pfaff quotes the sentences he is critiquing which state:

“Less crime after 1990 did mean fewer arrests for crimes like murder, rape, and robbery. But it didn’t mean fewer arrests overall” (p. 97).

As noted above, Pfaff says this is “incorrect” because “all sorts of arrests fell by a lot.” It’s a strange critique because the sentences Pfaff highlights mention arrests falling for three major crimes. And those sentences are immediately preceded by a chart (Table 10.1, p. 96, reproduced below) that illustrates the precise data on arrests:

Table 10.1 Arrests over time

Arrests	Total	Murder	Forcible	Rape
1980	10,458,260	20,040	31,380
1990	14,217,170	22,990	39,160
2000	13,985,979	13,227	27,469
2010	13,122,113	11,201	20,088

The chart (and subsequent discussion) show that annual arrests decreased slightly (about 1.6% between 1990 and 2000), while arrests for murder and rape fell dramatically.[20] A later chart shows that overall arrests exceeded the 1990 total in 2006 (reaching a total of 14,382,852).[21] Pfaff does not dispute any of these numbers (or any numbers in the book), and – as the preceding discussion shows – he is simply wrong to suggest that the sentences he highlights (quoted above) or the surrounding discussion are misleading or “incorrect.”

This is also a place where one would expect Pfaff to address our substantive disagreement. Mass Incarceration Nation identifies a major police-directed contribution to mass incarceration – a sharp rise in arrests up to 1990 and then (when crime fell in the 90s) a change in the mix of police arrests that kept arrest numbers high. This presents a direct challenge to Pfaff’s prosecutors-only claim. But Pfaff skips over any discussion of how these arrest numbers impact our substantive debate, choosing instead to make an easily refuted claim that an isolated comment in the book is “clearly incorrect.”

“The book also sometimes makes empirical claims that are not technically incorrect, but which gloss over important trends in a way that makes them feel deceptive. For example, at one point the book argues that arrests and convictions for aggravated assault moved roughly in lock-step, by comparing where they were at two points in time (1986 and 2006). The claim is technically correct—both arrests and convictions were higher in 2006 than 1986—but the trends in each differ significantly within that period.”

The discussion Pfaff critiques here is not only “technically correct” but contains all the purportedly missing context. Here’s the passage from the book that Pfaff cites to support this critique:

“A similar although less distinct trend can be seen in aggravated assaults in Figure 11.3. As police arrested more people for aggravated assault (top line), prosecutors charged more aggravated assaults and courts reported more aggravated assault convictions (bottom line). The large gap between the two lines reflects greater disagreement between police arrests and prosecutor charging and judicial convictions. Yet the general correspondence remains. Comparing 1986 to 2006, aggravated assault arrests went up by almost 100,000 and convictions increased by about 60,000” (p. 112).

Contra Pfaff, the passage doesn’t say that arrests and convictions moved “roughly in lock-step.” And it doesn’t “gloss over” any complexity. The passage says that, as compared to drug and homicide crimes (addressed in a previous passage), the aggravated assault trend is “less distinct,” revealing “greater disagreement” between police and other actors, but that over the entire span, there is a “general correspondence” as both arrests and convictions increased substantially. And again, Pfaff ignores the graph (Figure 11.3) that accompanies the discussion. That graph – referenced repeatedly in the passage Pfaff critiques – illustrates the precise relationship between annual arrests and convictions over the entire period, not just “at two points in time.”

“The book similarly ignores everything between the end-points when talking about drug arrests and prosecutions,²³ [and] when looking at arrest data from the LAPD.²⁴”

Both claims are fleshed out only in the two footnotes, where Pfaff quotes the offending language and then reveals the flaw in his critique. He supports his claims that the book “ignores” important details by reference to the book’s presentation of those details in another part of the discussion “on that page” or in an accompanying figure. Here are the footnotes:

“[FN 23] Bellin, Mass Incarceration, at 111: ‘As police arrested more people for selling drugs, prosecutors prosecuted more folks, resulting in more convictions.’ Yet like with aggravated assaults, the graph (Figure 11.2) shows that drug arrests rose until about 1990 and then slowly declined, while prosecutions rose over the whole period (albeit more quickly prior to 1990 than after).”

“[FN 24] Bellin, Mass Incarceration, at 105: ‘Overall LAPD arrest totals declined from 1980 and 2010,’ which is nominally true, although Figure 10.2 on that page makes it clear that arrests rose from 1980 to 1990, and then declined through 2000 and 2010.”

There is not much to add to Pfaff’s unintentional self-critique on these points. As Pfaff notes in these footnotes, the figures that accompany the discussion that (he says) “ignores everything between the end-points” portray the entire trends. The discussion does too. Pfaff quotes only the beginning of the sentence about the LAPD. The portion of the sentence Pfaff omits (italicized below) highlights the changes within the endpoints that Pfaff claims are missing. The full sentence from the book reads:

“Overall LAPD arrest totals declined between 1980 and 2010, but that decline masks an important change in the mix of arrests, led by a dramatic midtrend spike in aggravated assault and drug arrests” (p. 105).[22]

“There are also times where the book misstates the findings of the papers it cites, like the time it refers to a paper about the impact of mandatory arrests in domestic violence cases to support a claim about arrests in general, even though those are two very different policies.”

Pfaff offers only this one example to support his claim that the book misstates the findings of “papers.” Pfaff says the book “misstates the findings” of a “2020 meta-analysis” on domestic violence by leaving out that the study concerns “mandatory arrests” not “arrests in general.”

The passage in Mass Incarceration Nation where Pfaff claims I misstated the study’s findings is almost entirely a block quote from the study’s authors summarizing their findings. Here is the passage:

“A 2020 meta-analysis of the many studies on the effectiveness of arrests in reducing domestic violence found little evidence to support the status quo. The authors summarized their findings as follows:

‘The purpose of the current meta-analysis was to synthesize the results of the Minneapolis DV experiment, the replications, and other studies using experimental and quasi-experimental research designs to examine whether arrest for DV is effective in reducing repeat offending. The results of the meta-analysis suggest that the mandatory arrest policies adopted by police departments in the 1980s and 1990s did not produce the desired deterrent effect sought by law enforcement agencies and advocates alike’” (p. 177).

Not only does the passage primarily quote the study’s authors, but the quoted summary specifically references the focus on “mandatory arrest policies” – which is the important detail Pfaff (incorrectly) claims was left out. And notice that the authors describe their findings as relating to arrests generally, saying that they studied “whether arrest for DV is effective in reducing repeat offending.” That’s because Pfaff is also wrong that the authors only looked at jurisdictions with mandatory arrest policies. The authors sought “to determine whether arrest for DV is an appropriate law enforcement intervention,” and so included any qualifying study that used “arrest as the major independent variable,”[23] even if the study took place in a jurisdiction without a mandatory arrest policy.[24]

“it claims that people’s perceptions of crime in the 1990s did not respond meaningfully to falling crime rates, even though Gallup’s measure of fear of crime was cut in half over that decade”

Pfaff identifies the following sentence as the source of this “error”:

“Crime has decreased steadily since the 1990s, but most Americans continue to perceive crime as rising” (p. 37).

To support his claim that this sentence is wrong, Pfaff points to a period at the end of the 1990s when Gallup polling showed a decreased fear of crime. But the statement Pfaff critiques (“most Americans continue”) speaks to a broader, more recent time horizon. And while Gallup polling reflects decreased fear in the late 1990s, that changes over the next 20 years as decreased crime coincides with increased fear.[25] In fact, in the 34 Gallup polls between 1972 and 2022, “less crime” beats “more crime” only 4 times (1998, 2000, 2001, 2018). Thus, Pfaff’s claim of error is wrong.

And again, it is revealing how tangential Pfaff’s critique is to any substantive disagreement. He criticizes the quoted sentence for understating the degree to which crime matters to public opinion. But the importance of crime to public opinion is one of the book’s themes. The sentence he critiques is in a paragraph that concludes: “In the 1970s and 1980s, with crime actually rising and the public feeling its effects, the nation reached a consensus – new measures were needed to fight crime.”[26] And that paragraph comes right after Chapter 5 “The Crime Surge” which identified a spike in violent crime as “the spark that lit the fire that became Mass Incarceration.[27] That’s why Part IV “The Road to Recovery,” includes a chapter on reducing serious crime, explaining “less crime means fewer people to arrest and convict and imprison” and “reduces populist pressures for increased severity.”[28] And Pfaff ultimately acknowledges that “there is definitely some inescapable truth to that point.” There is no substantive disagreement here, just a sentence pulled out of context and (incorrectly) criticized as incorrect.

***

This response is intended to reassure readers, and potential readers, of Mass Incarceration Nation that contrary to Pfaff’s claims, he has not identified any “empirical assertions that are simply untrue” or instances where the book “misstates the findings of the papers it cites.” Every example Pfaff offers to support these criticisms collapses under slight scrutiny.

Jeffrey Bellin is the Mills E. Godwin, Jr., Professor of Law at William & Mary Law School

[1] For other reviews, see Robert Sanger, The Champion, July 2023 (“Everyone involved in the criminal law, at all levels, should read Professor Bellin’s carefully researched and well-thought-out work.”); Graciela Perez, Criminal Justice Review, July 2023 (“Mass Incarceration Nation is a highly informative and balanced book, offering a wide-ranging scope on the causes of mass incarceration and proposing practical solutions to reduce the prison population.”); SpearIt, America’s Punishment Addiction Explained, Jotwell, Nov. 2023 (“Despite its nuance and academic structure, the work is accessible and written in clear language, which makes it as useful for law students and stakeholders in the criminal system as it is interesting for the lay reader.”).

[2] Jeffrey Bellin, Mass Incarceration Nation 7 (Cambridge 2023).

[3] Id. at 93.

[4] Locked In at 5. This argument built on earlier critiques. See, e.g., James Forman, Jr., Racial Critiques of Mass Incarceration: Beyond the New Jim Crow, 87 N.Y.U. L. Rev. 21, 23 (2012) (criticizing Alexander’s account for overlooking that “violent offenders make up a plurality of the prison population”).

[5] John F. Pfaff, Locked In 73 (2017) (“It’s important to be wary of ‘one thing explains it all’ theories, especially for a phenomenon as complex as prison growth. These results, however, certainly support a claim of ‘one thing explains most of it.”’); id. at 72 (“‘The only thing that really grew over time was the rate at which prosecutors filed felony charges against arrestees”).

[6] Locked In at 71.

[7] John F. Pfaff, Prosecutors Matter: A Response to Bellin’s Review of Locked in, 116 Mich. L. Rev. Online 165, 166 (2018).

[8] Locked In at 72-73.

[9] Jeffrey Bellin, Reassessing Prosecutorial Power Through the Lens of Mass Incarceration, 116 Mich. L. Rev. 835 (2018) (“Locked In Review”). The review first became publicly available on SSRN in March 2017, see https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2930116.

[10] Locked In Review at 839.

[11] Id. at 842.

[12] See Katherine Beckett, Mass Incarceration and Its Discontents, 47 Contemp. Soc. 11, 16 (2018) (“Pfaff’s analysis is undermined by methodological flaws, logical errors, and conceptual limitations.”). Beckett obtained the spreadsheet (which “was not readily available on the NCSC server”) and observed “many errors contained in the data.” Beckett quotes an NCSC analyst who flagged substantial “2003 caseloads jump” for Florida and Texas, as well as “‘footnotes for many data elements’” on the spreadsheet that “‘denote a data quality issue’” whose “‘extent is unknown.’” Id. at 19 & n.14.

[13] Locked In Review at 837.

[14] John F. Pfaff, Prosecutors Matter: A Response to Bellin’s Review of Locked in, 116 Mich. L. Rev. Online 165, 165 (2018) (“Response”).

[15] See e.g., Response at 170 (“With hindsight, I should have omitted Texas.”). Pfaff’s Response suggests that he knew that the NCSC data-gathering change occurred in 2003 (in the middle of his 1994-2008 date range) but does not explain why he did not flag that complication, and instead wrote that the NCSC “changed the way it gathered the data in 1994.” Compare Locked In at 257 n.50 (“The NCSC gathered data on felony filings before 1994, but it changed the way it gathered the data in 1994, making it hard to compare across years.”); John F. Pfaff, The Micro and Macro Causes of Prison Growth, 28 Ga. St. U. L. Rev. 1239, 1250 (2012) (“the NCSC revised how it gathered the data in 1994”); with Response at 166 (“Bellin points out, correctly, that starting in 2003 the NCSC changed how it gathered its felony-filing data, leading to a discrete jump in the number of felony filings.”); see also Beckett at 19 & n.13 (“Neither in his book nor in the article on which it is based does Pfaff discuss these filing data and their limitations in any detail.”).

[16] Response at 165, 166.

[17] See Jeffrey Bellin, A World Without Prosecutors, 13 Cal. L. Rev. Online 1, 2–3 & fn. 11 (2022).

[18] Mass Incarceration Nation at 97.

[19] Id. at 119.

[20] See also id. at 6 (“These numbers stayed high even as crime dropped because the system pivoted to commonly occurring, easily detected, and readily provable offenses.”); id. at 95 (“More police did mean more arrests. But arrest numbers went up and stayed high even after crime fell.”).

[21] Id. at 115 (Table 11.1.).

[22] Pfaff slightly misquotes the sentence fragment he critiques by inserting a “from” where there is a “between.” But it is clear this is the sentence he is referring to since he cites page 105 and there are no other similar sentences on that page.

[23] Susan J. Hoppea, Yan Zhanga, Brittany E. Hayes, Matthew A. Bills, Mandatory Arrest for Domestic Violence and Repeat Offending: A Metaanalysis, 53 Aggression and Violent Behavior 1, 3 (2020) (emphasis added).

[24] For example, the meta-analysis included: Richard M. Tolman & Arlene Weisz, Coordinated Community Intervention for Domestic Violence: The Effects of Arrest and Prosecution on Recidivism of Woman Abuse Perpetrators, 41 Crime & Delinquency 481, 485 (1995) (evaluating arrests in jurisdiction with “pro-arrest” policy); and J. David Hirschel, Ira W. Hutchison. & Charles W. Dean, The Failure of Arrest to Deter Spouse Abuse, 29 J. Res. Crime & Delinquency 7, 13 (1992) (measuring the effectiveness of varying arrest decision in scenarios where “police were empowered but not required to make an arrest”). Since most of the studies were experimental, arrests were often “mandated” or prohibited by the experimental protocols. By randomizing arrests, the researchers measured the effectiveness of arrests, not just mandatory arrest policies.

[25] Compare Mass Incarceration Nation at 180 (graphs of crime rates) and Gallup polling data, available at https://news.gallup.com/poll/1603/crime.aspx

[26] Mass Incarceration Nation at 37.

[27] Id. at 33 (“Rising crime was the spark that lit the fire that became Mass Incarceration. Changing laws, policies, and attitudes that arose in the wake of that spark explain the rest.”).

[28] Id. at 179.