I have discussed in many places – most comprehensively in “Race and Mortality Revisited,” Society (July/Aug. 2014), and comments for the Commission on Evidence-Based Policymaking (Nov. 14, 2016) – that there have been virtually no sound analyses of demographic differences involving outcome rates because of the failure of such analyses to consider the ways the measures employed tend to be affected by the prevalence of an outcome. And I have suggested in the above items, and stated explicitly in a July 17, 2017 letter to the Departments of Education, Health and Human Services, and Justice, that the federal government should fund no research into demographic differences that fails to consider the extent to which observed patterns are simply functions of differences in the prevalence of an outcome and the extent to which the patterns reflect something meaningful about underlying processes. In fact, since studies that fail to consider such things are invariably misleading, funds devoted to such research would be better used to pay researchers not to perform it.
The fundamental failings of research into demographic differences involving outcome rates are not limited to the United States. They are universal around the world and have existed ever since societies have studied those differences.
But until recently the most profound ignorance about the analysis of demographic differences, and the most pernicious consequences of that ignorance, have been largely limited to the United States. I refer to the many federal civil rights law enforcement activities where analyses of demographic differences are not merely oblivious to the effects of the prevalence of an outcome on the measures they employ, but are based on a belief about those effects that is the opposite of reality. That is, federal civil rights enforcement policies relating to lending, school discipline, employment, voter rights, and criminal justice have been based on the belief that generally reducing adverse outcomes will tend to reduce (a) relative racial and other demographic differences in rates of experiencing the outcomes and (b) the proportions more susceptible groups make up of persons experiencing the outcomes.
In fact, generally reducing any outcome tends to increase, not reduce, both (a) and (b) as to the outcome. That is, reducing an outcome and thereby increasingly restricting it to those most susceptible to it, while tending to reduce relative differences in rates of avoiding the outcome (i.e., experiencing the opposite outcome), will tend to increase relative differences in the outcome itself; correspondingly, reducing the outcome, while tending to increase the proportions groups more susceptible to the outcome make up of persons avoiding the outcome, will tend also to increase the proportions such groups make up of persons experiencing the outcome itself.
This is a matter I have discussed here many times and that was the subject of a July 24, 2017, Federalist Society teleforum titled “Are Existing Civil Rights Policies Based on a Statistical Understanding That Is the Opposite of Reality?” I discussed it here most recently in “The Pernicious Misunderstanding of Effects or Policies on Racial Differences in Criminal Justice Outcomes” (Oct. 12. 2017) with regard to a September 2017 Sentencing Project document reflecting the mistaken view that generally reducing incarceration rates should tend to reduce relative racial differences in incarceration rates.
Among the many anomalies arising from government’s mistaken belief about the effects of policies on measures of racial disparity that it employs are that lenders and public schools that comply with government encouragements to relax lending and school discipline standards tend to increase the chances that the government will sue them for discrimination. And, as explained in the recent post, in the criminal justice context the government’s promotions of policies that are supposed to reduce (a) and (b), but that in fact tend to increase (a) and (b), can lead those who believe that racial disparities are driven substantially by bias to believe that bias must be increasing. See also my September 26, 2017 post titled “An Important Interview in Baltimore” regarding the unwarranted expectations being created in Baltimore as to the likely effects of the consent decree covering Baltimore Police Department practices on the measures of racial disparity the government has typically employed.
Similar perceptual problems exists in public school districts across the country that Dear Colleague letters from the Departments of Education, Health and Human Services, and Justice have led to believe that generally reducing adverse school discipline outcomes will tend to reduce (a) and (b) as to those outcomes. See my August 24, 2017 post titled “Innumeracy at the Department of Education and the Congressional Committees Overseeing It.” Hence, the aforementioned July 17 letter to those agencies urged them to issue another Dear Colleague letter explaining that prior guidance was incorrect.
Meanwhile, the following events are unfolding in the United Kingdom. On September 8, 2017, after extensive study that included consultation with the United States Department of Justice, a special commission led by Member of Parliament David Lammy issued the “Lammy Review: An independent review into the treatment of, and outcomes for, Black, Asian, and Minority Ethnic individuals in the Criminal Justice System.” The document, much like reports the Department of Justice issued regarding Ferguson, Missouri in 2015, and Baltimore and Chicago in 2016 and 2017, appraises criminal justice disparities on the basis of (a) relative racial differences in rates of experiencing adverse outcomes and (b) the proportions racial minorities make up of persons experiencing the outcomes (compared with the groups’ representation in the population). And it does so while reflecting the mistaken belief that generally reducing those outcomes ought to reduce, rather than increase, (a) and (b) as to the outcomes.
The third paragraph of the press release announcing issuance of the report, in the manner of the Sentencing Project document discussed in the October 12 post, highlights that the proportion racial minorities made up of young offenders in custody “rose from 25% to 41% between 2006 and 2016, despite the overall number of young offenders falling to record lows.” Thus, like the Sentencing Project document, the Lammy report, which regards racial bias to play a significant role in racial differences in adverse criminal justice outcomes, gives the impression that forces causing racial differences in rates of incarceration have been increasing. But it is impossible to determine whether those forces are increasing or decreasing in the UK, just as it is impossible to do so in the United States, without understanding that generally reducing an outcome tends to increase, not reduce, the proportions more susceptible groups make up of persons experiencing the outcome.
In a section titled “From data to insight” the Lammy report sets out the following as it second recommendation:
The government should match the rigorous standards set in the US for analysis of ethnicity and the CJS [Criminal Justice System]. Specifically, the analysis commissioned for this review – learning from the US approach – must be repeated biennially, to understand more about the impact of decisions at each stage of the CJS.
The recommendation is presented in the context of analyses of relative demographic difference in criminal justice outcomes at each stage of the process. Thus, it specifically pertains merely to the matters as to which there have yet to be sound analyses of demographic differences by the United States government or any other entity or person as a result of the failure to consider the ways the measures employed tend to be affected by the prevalence of an outcome. But the approbation of the rigor of analyses of demographic differences in the United States occurs in the context of the report’s wholesale acceptance of the manifestly incorrect notion promoted by the Department of Justice that generally reducing criminal justice outcomes will tend to reduce the aforementioned (a) and (b) as to those outcomes.
In that regard, several aspects of the Lammy report warrant note. The report finds that black offenders are more likely to reoffend than other offenders and it cites approvingly the practice of expunging the records of offenders who do not reoffend. Yet the fact that black offenders are more likely to become multiple offenders than other offenders necessarily means that expunging records of first offenders who do no reoffend will cause the black proportion of persons with criminal records to increase. This point may be compared to the illustrations in Table 2 to 5 of the July 17 letter to the three federal agencies of the way that giving students reprimands instead of their first suspension will tend to increase the proportions groups with generally higher suspension rates make up of persons with one or more suspensions.
The same may be said of the Lammy report’s recommendation, highlighted in the press release as a “particularly radical proposal,” of deferring prosecution of low level offenders and dropping charges where the offenders successfully complete targeted rehabilitation. Whether radical or not, and whether a good or bad idea, the approach will tend to increase the proportion racial minorities make up of persons ultimately prosecuted.
A September 7, 2017 Independent article on the report discusses that Mr. Lammy noted, on the basis of the facts that in the UK blacks are 3 percent of population and 12 percent of persons in prison while in the US blacks are 13 percent of the population and 35 percent of persons in prison, that racial disproportionality is greater in the UK than the in the US. While there are some mathematical issues in Mr. Lammy’s quantification of the UK disproportionality, one can derive from the referenced figures that in the UK the black rate of incarceration is 4.4 times the rate of other persons while in the US the black rates is only 3.6 times the rate of other persons.
One cannot appraise the significance of the larger ratio in the UK, however, without understanding that such ratio will tend to be greater in the UK simply because overall rates of incarceration are much lower in the UK than in the US. But the UK civil servants and researchers that produced the Lammy report are no more likely to figure that out than legislators in New Jersey, or education officials in Loudoun County, Virginia or Massachusetts are to recognize the association between their comparatively large relative racial differences in incarceration or suspension rates and the fact that they have comparatively low overall rates for those outcomes.
Indeed, as discussed in "Race and Mortality Revisited" (at 340), one of the more remarkable longstanding misinterpretations of data on group differences involves the significance observers have attributed to the fact that relative socioeconomic differences in adverse health outcomes are greater among British civil servants than in the UK population at large. They have invariably done so while failing to recognize implications of the fact that British civil servants are healthier than the UK population at large. See also my 2006 Chance editorial (at 50) and my 2014 Minneapolis Star Tribune commentary regarding the failure of Norway and Sweden to recognize the connection between their comparatively large relative socioeconomic differences in mortality and the fact that that they are comparatively healthy countries. See generally "Race and Mortality Revisited" (at 339-341), “Race and Mortality,” Society (Jan./Feb. 2000), and the abstract of my November 2015 University of Massachusetts Medical School Seminar titled “The Mismeasure of Health Disparities in Massachusetts and Less Affluent Places” regarding the universal failure to recognize that relative demographic differences in adverse outcomes tend to be comparatively large, while relative differences in the corresponding favorable outcomes tend to be comparatively small, in circumstances where the adverse outcomes are comparatively uncommon.
On October 10, 2017, the UK government published a “Race Disparity Audit,” described as a summary of findings from the government’s Ethnicity Facts and Figures website. This document is simply the usual reportage one might find from any government or any private research entity that employs various measures of disparity without consideration of the ways the measures tend to be affected by the prevalence of an outcome or recognition that some measures that it employs may contradict patterns identified by other measures. Thus, the Race Disparity Audit promotes much misunderstanding, though, in contrast to the Lammy report, it seems not to affirmatively recommend actions that will tend to increase the disparities it describes.
According to an October 9, 2017 Independent article discussing release of the Race Disparity Audit, Prime Minister Theresa May indicated that, as a part of a broad initiative to attack racial disparities in the UK, the Ministry of Justice will be adopting recommendations of the Lammy report. The article also stated that a new “external review” (presumably meaning an independent review akin to the Lammy Review) will be conducted of racial disproportionality in school suspensions and expulsions. That is another area where the UK may seek to learn from the United States. But, as things now stand, what the UK might learn from the United States in that area is unlikely to further its understanding.
I mentioned above that my July 17 letter to the three agencies urged them to inform school administrators that prior guidance regarding effects of relaxing discipline standards on measures of racial disparity was incorrect. I made that recommendation in the context of a specific obligation of the agencies arising from their prior erroneous guidance and with regard to a matter that could be addressed immediately. But the government has long been furthering the unsound analyses of demographic difference even when it has not been promoting beliefs that are the exact opposite of reality. And if the government ever comes to fully understand these issues – something I have suggested, while by no means certain to happen, may be more likely to happen in the current administration than the last – it should recognize far reaching obligations to the public and the research community with respect to correcting the harms arising from the government’s failure to understand things a government should understand. It now has similar obligations to the UK.