I have written here before, most recently in “Things the President Doesn’t Know About Racial Disparities”  (Aug. 5, 2016), about the way the federal government, including the President, base many civil rights law enforcement policies on an understanding of statistics that is the exact opposite of reality. In particular, policies involving fair lending, school discipline, and criminal justice are based on the belief that relaxing standards or otherwise reducing the frequency of adverse outcomes will tend to reduce (a) percentage differences in rates of experiencing the outcomes and (b) the proportion racial minorities and other disadvantaged groups make up of persons experiencing the outcomes. In fact, however, reducing the frequency of an outcome tends to increase both (a) and (b). 

I illustrate the pertinent statistical pattern by showing how lowering a test cutoff, while tending to reduce percentage differences in pass rates, tends to increase percentage differences in failure rates. Lowering a cutoff will also tend to increase the proportion the lower-scoring group makes up of both persons who pass the test and persons who fail the test. 

This pattern is not so easy even for people with mathematical backgrounds to understand. But neither it is debatable.  One can find tabular illustrations in my articles “The Mismeasure of Health Disparities,”Journal of Public Health Management and Practice (2016), and “Race and Mortality Revisited,” Society(2014), as well as in my amicus curiae brief in Texas Department of Housing and Community Development v. The Inclusive Communities Project, Inc. (2014) and my presentation titled “The Mismeasure of Disparate Impact” at the Federalist Society’s Fourth Annual Executive Branch Conference in May of this year.  Graphical illustrations may be found in my 2015 University California, Irvine methods workshop titled “The Mismeasure of Discrimination.”  

Indications of how longstanding is the confusion over this subject may be found in my “The Perils of Provocative Statistics,” Public Interest (1991). One might say, however, that failure to understand the pattern has caused confusion in the analysis of racial and other demographic differences for as long as people have been talking about such things.  Thus, while the patterns by which measures tend to change as the frequency of an outcome changes are things that everyone discussing demographic differences should know, virtually no one in fact knows them. Indeed, as  noted, the government’s  understanding is the opposite of reality. And a common consequence of the government’s failure of understanding is that by following government guidance to generally reduce adverse outcomes, entities covered by civil rights laws increase the chances that the government will sue them for discrimination. In the case of the enforcement of fair lending laws that situation has persisted for more than two decades. 

Two manifestations of the misunderstanding may be found in recent matters pertaining to criminal justice outcomes in Baltimore, Maryland and voter ID issues involving Texas and North Carolina.  

On August 10, 2016 the Department of Justice (DOJ) issued a report titled “Investigation of the Baltimore Police Department,” following the August 9, 2016 execution of an Agreement in Principle whereby Baltimore agreed to reach, by November 1, 2016, a consent decree addressing racial disparity issues identified in the report.  The report reflects the same misunderstanding of statistics reflected in a March 2015 DOJ report on police and court practices in Ferguson, Missouri and a federal court complaint DOJ filed against the Ferguson in February 2016. In both documents DOJ maintained that the aggressiveness of police practices and the harshness of court procedures in Ferguson caused African Americans to make up higher proportions of persons experiencing adverse interactions with the police and courts than would be case with less aggressive police practices and less harsh court procedures. But reducing the aggressive of police practices and the harshness of court procedures would tend to increase, not reduce, those proportions.  

A consent decree entered in April 2016 put Ferguson under a variety of obligations regarding monitoring data on disparities in criminal justice and police employment outcomes, as well as obligations to seek less discriminatory alternatives to practices causing racial disparities. As I explained in a submission to the court in the case (though to no avail), the decree places Ferguson in an untenable situation, given that what are commonly perceived to be less discriminatory alternatives tend to increase racial disparities according to the way the government commonly measures them. 

The DOJ report on Baltimore reflects the same mistaken belief reflected in DOJ actions regarding Ferguson that reducing the frequency of adverse criminal justice outcomes will tend to reduce the proportions African Americans make up of persons experiencing them.  

Baltimore may differ somewhat from Ferguson because overall racial differences in Baltimore are much influenced by arrests and searches in largely African American neighborhoods. Reducing police presence or aggressiveness of enforcement in such neighborhoods may well reduce overall racial differences, regardless of how one measures them.  Also, if bias influences racial differences and modifications to practices reduce bias, all measures of racial differences will decrease. 

But it is difficult to craft a decree that makes any sense with an understanding of the usual effects of reducing the frequency of outcomes on measures of racial disparities that is as far off the mark as the understanding of the DOJ.

Meanwhile, on July 20, 2016, in Veasey v. Abbott, the Fifth Circuit struck down a voter identification requirement in Texas, and on July 29, 2016, in  North Carolina NAACP  v. McCrory, the Fourth Circuit invalidated  various North Carolina voting policies, because of the manner in which the requirement and policies adversely affected African Americans. 

The Veasey decision nicely illustrates the statistical issue. The court relied on the large percentage racial difference in rates of failing to have the necessary voter ID, at one point noting that the racial disparities identified by the district court were “stark.” The court of appeals also noted the difficulties individuals would have in securing a type of ID that the state accepted from voting purposes. As with other decisions on voting requirements, however, the decision reflected no understanding that relaxing the requirements for securing an acceptable ID (that is, making it easier) will tend to increase percentage differences in rates of failing to secure an acceptable ID. 

Both cases involve issues of discriminatory intent as well as disparate impact. Along with some other considerations, arguments are made that courts should infer a discriminatory intent on the basis of the adoption or continuation of practices in light of the practices’ expected or observed racial impact. In such circumstances, the perceived magnitude of the disparity may well have a role, as it did in the DOJ’s Ferguson complaint that argued that the city’s continuation of practices in light of the magnitude of the racial impact was evidence of discriminatory purpose. As explained, however, the perceived magnitude or a racial disparity, and its relationship to particular practices, are not always what they seem.  At any rate, it is a rather more complicated issue than most understand it to be.

The State of Texas just filed its petition for certiorari and North Carolina has indicated an intention to do so. Given apparent conflicts among the circuits, there seems a good chance that the Court will grant the petitions. That may provide an opportunity for the Court to come to understand some things about racial disparities that, like other government entities and the overwhelming majority of the social science community, the Court has so far failed even to glimpse.