On January 4, 2017 – fifteen days before the change in administrations and fourteen days before the effective date of the outgoing administration’s regulation on measuring significant racial/ethnic disproportionality in special education programs that would eventually be the subject of Council of Parent Attorneys and Advocates, Inc. (COPAA) v. DeVos – I posed here the question “Will Trump Have the First Numerate Administration?”  A lower court ruling in COPAA v. DeVos barring the Department of Education from postponing compliance with the  regulation is now on appeal to the U.S. Court of Appeals for the District of Columbia Circuit, with the government’s brief due on September 18, 2019.  That brief may answer the question posed here more than two-and-a-half years ago.

[Because of the length of this post, a PDF version is available here.]

The January 2017 post principally addressed the longstanding belief underlying many federal civil rights enforcement policies that relaxing a standard or otherwise reducing an adverse outcome like rejection of a mortgage application, suspension from school, incarceration, or lack of a voter ID would tend to reduce the relative (percentage) difference between rates at which advantaged and disadvantaged groups experienced the outcome.  The relative difference in such outcomes is commonly presented in terms of the ratio of a disadvantaged group’s rate to an advantaged group’s rate, termed the “risk ratio” in the referenced Department of Education regulation that mandates its use for determining significant racial/ethnic disproportionality under the Individuals with Disabilities Education Act.  By way of example, it would be on the basis of a risk ratio that the U.S. Commission on Right’s July 2017 report on discipline disparities (see Appendix hereto) states (at 148) that even after substantial reductions in out-of-school suspensions between the 2011-12 and 2013-14 school years, black K12 students remained 3.8 times as likely to receive out-of-school suspensions as white students.  In terms of the relative difference that the 3.8 risk ratio represents, the black rate was 280 percent greater than the white rate.

But, as I explained in the post, generally reducing an adverse outcome, while tending to reduce relative differences between rates at which disadvantaged and advantaged groups avoid the outcome (i.e., experience the corresponding favorable outcome), tends to increase relative differences between the rates at which the groups experiencing the outcome itself.  For example, test score data show that lowering a test cutoff, while tending to reduce relative differences between the pass rates of higher- and lower-scoring groups, tends to increase relative differences between the groups’ failure rates.  Income and credit score data show that lowering an income or credit scored requirement for receipt of some loan product, while tending to reduce relative differences between rates at which blacks and whites meet the requirement, tends to increase relative differences between rates at which blacks and whites fail to meet the requirement. 

The existence of these patterns is not debatable.  Nevertheless, possibly every data analyst in the Department of Education is unaware that lowering cutoffs for meeting academic proficiency standards (or improving test performance) will tend to increase relative racial and other demographic differences in rates of failing to meet the standards.  Probably every member of the Federal Reserve Board and virtually all of Federal Reserve economists assume without question that relaxing a lending standard will tend to reduce relative racial differences in loan rejection rates.  And, even though the National Center for Health Statistics (NCHS) long ago recognized that as health and healthcare improve, while relative differences in the (increasing) favorable health and healthcare outcomes tend to decrease, relative differences in the corresponding (decreasing) adverse health and healthcare outcomes tend to increase (as discussed here and here), possibly no data analyst in any other arm of the federal government is aware that is even possible for relative differences in favorable outcomes and relative differences in corresponding adverse outcomes to change in opposite directions as the frequency of an outcome changes.  Most pertinent here, a high proportion (often 100 percent) of data analysts in all agencies continue to assume that reducing an adverse outcome will tend to reduce relative racial and other demographic differences in rates of experiencing the outcome, which is the opposite of what NCHS found and, more important, the opposite of reality.

There are reasons why generally reducing a particular adverse outcome may not increase, and may even reduce, relative differences in rates of experiencing the category of adverse outcomes of which the particular outcome is a part, as I have discussed here, for example.  But never has anyone even proffered a reason why generally reducing an outcome would be expected to reduce relative differences in rates of experiencing the outcome.  Rather, in the hundreds or thousands of instances in recent decades where scholars or commentators have observed that “despite” a decline in some adverse outcome, relative racial or other differences in rates of experiencing the outcome persisted or increased –  or where legislators or government officials implemented policies to reduce such differences by reducing the outcomes – this profoundly false expectation has simply been taken for granted. 

The government’s leading the public and affected entities to believe that actions will tend to reduce a measure of racial disparity when in fact the actions tend to increase the measure has a number of pernicious consequences.  One anomaly that had long existed in January 2017 was that lenders that complied with government encouragements to relax lending standards in order to reduce the ratio of minority mortgage rejection rates to white mortgage rejections rates increased the chances that the government would sue them for discrimination.  Many lenders in fact paid dearly for the government’s (and their own) mistaken understandings of the effects of lowering lending standards on that ratio.  In the school discipline context, numerous school districts have labored under agreements requiring the districts both to generally reduce suspensions and to reduce the ratio of the black suspension rate to the white suspension rate, while all involved regard the requirements as complementary rather than conflicting.  Very likely, school superintendents and other officials have lost their jobs because, after implementing policies that were supposed to reduce relative racial differences in suspension rates, the differences increased.  It is also likely that individual decisionmakers have faced accusations of discrimination simply because they assiduously followed guidance to reduce suspensions or exhibited special skill in controlling student conduct with comparatively infrequent resort to suspensions. 

But the most harmful consequence of this regime of near universal innumeracy involves the way that the government’s misleading of the public promotes racial mistrust.  For one thing, as a result of the government’s encouragements to relax various standards, observers that found it difficult to believe, for example, that factors other than discrimination could explain why the black suspension rate was three times the white rate would find nondiscriminatory explanations even more difficult to believe when the black rate becomes four times the white rate.  Further, when measures of racial disparity increase in the face of actions that the government has led the public to believe should reduce those measures, observers who believe that racial bias plays a substantial role in observed differences will reasonably believe that bias must have increased.  That includes racial minorities whose life choices within and without school will often be adversely influenced by beliefs not only that they face substantial bias in many settings, but that bias is constantly increasing. 

The adverse consequences of the government’s misleading the public about the effects of policies on measures of racial disparities are exacerbated when the government itself promotes the belief that things like severalfold differences in public school suspensions could not be occurring unless racial bias was playing a substantial role, as the Departments of Education and Justice had been doing as of January 2017.      

But one reason why the Trump administration might be better able to understand this issue than prior administrations was that explaining the matter to the public would facilitate the administration’s doing a number of things that it was likely to want to do for other reasons.  For example, it would make it easier for the Department of Justice to curtail what the administration was likely to regard as inordinate federal oversight of state and local police practices if the agency were to explain that the remedies commonly implemented  in order to reduce seemingly large measures of racial disparity in adverse criminal justice outcomes (including any adverse interactions with the police) had actually been increasing those measures.  It would make it easier for the Departments of Education and Justice to rescind the January 2014 joint Dear Colleague letter that had called for the relaxing of discipline standard in order to reduce the ratio of the black suspension rate to the white suspension rate if the agencies were to explain that relaxing standards actually tended to increase the ratio.  And it would make it easier for the administration to modify prior administration expansions of the disparate impact doctrine if it were to explain that the relaxing of standards that is commonly deemed to reduce a disparate impact actually tended to increase the measure of disparate impact the government employed in most settings.  But the administration would have to understand the issue first, something no prior administration, and very few statisticians and social scientists, had so far been able to do.

Since 1987, I have explained this subject in a great many places.  I first discussed it here in a long May 5, 2016 post regarding constitutional issues arising from the fact that relaxing standards tended to increase one standard measure of disparate impact at the same time that it reduced another standard measure of disparate impact.  In the post, I also explained with respect to another way that the government often appraised racial disparities that, contrary to the government’s longstanding belief, reducing an adverse outcome tends to increase, not reduce, the proportion a disadvantaged group makes up of persons experiencing the outcome.  On May 16, 2016,  I then gave a presentation titled “The Mismeasure of Disparate Impact” on the subject at the Federalist Society Fourth Annual Executive Branch Review Conference, and, in posts of August 5 and October 3, 2016, I discussed prevalent misperceptions regarding the effects of policies on measures of racial differences in adverse criminal justice outcomes and failure to have a voter ID.

Over the year following the inauguration I discussed this subject quite a few times here with reference to one topical matter or another, while questioning whether the Trump administration would actually prove to be any more numerate than its predecessors.  Using these materials, I presented a July 24, 2017 Federalist Society teleforum on the subject (as discussed in a July 21, 2017 post titled “The Government’s Uncertain Path to Numeracy”).  And I repeatedly emphasized two things.  First, the government’s mistaken understanding of the effects of reducing an adverse outcome on the measures of racial disparity that the government commonly employed was but part of larger and essentially universal failure to understand the ways that measures tend to be affected by the frequency of an outcome that had undermined virtually all analyses of demographic difference involving favorable and corresponding adverse outcomes in the law and the social and medical sciences.  Second, once understanding that it had been mistaken in its belief regarding the effects of reducing an adverse outcome on measures of racial disparity, the government had an obligation to explain to the public and various directly affected entities that the government had been misleading them on this matter for decades.  See especially “United States Exports Its Most Profound Ignorance About Racial Disparities to the United Kingdom” (Nov. 2, 2017), which at the outset addresses the former issue and at the close addresses the Department of Justice’s obligation toward the United Kingdom due to the agency’s role in leading a parliamentary commission to believe mistakenly that reducing adverse criminal justice outcomes would tend to reduce, rather than increase, the measures of racial disparity the commission proceeded to employ. 

I also stressed both of these matters in oral and written testimony at a December 8, 2017 hearing on school discipline held by the U.S. Commission on Civil Rights and orally and in written materials at a March 22, 2018 meeting with Department of Education staff.    

 

One obvious occasion for the Departments of Education and Justice to fulfill their obligations regarding the misleading guidance on the effects of reducing adverse outcomes on measures of racial disparity would be in rescinding the January 2014 joint Dear Colleague letter on school discipline, which the agencies did by a notice issued on December 18, 2018.  Rather than explaining that the premise of the 2014 Dear Colleague letter regarding the effect of generally reducing school suspensions on measures of racial disparity was the opposite of reality, however, the notice said nothing about the issue.  The notice thus effectively bolstered the mistaken belief underlying the 2014 guidance even as it rescinded the guidance that had been premised on that belief.  And the many jurisdictions around the country that regarded relaxing discipline standards as a sensible course precisely because they believe that doing so tends to reduce certain measures of racial disparity would continue to follow that course without questioning the soundness of that belief. 

Similarly, nothing else the administration did in scaling back government monitoring of demographic differences showed any awareness that premises informing prior administration activities were mistaken.  Even the Department of Housing and Urban Development’s proposed rule on disparate impact liability under the Fair Housing Act (posted on August 19, 2019 with comments due by October 18, 2019) that purports to clarify a range of issues, including regarding less discriminatory alternatives to challenged practices, shows no awareness whatever that relaxing a standard, while tending to reduce relative differences in favorable outcomes, tends to increase relative differences in the corresponding adverse outcomes.

The Department of Education’s Disproportionality and Delay Regulations at Issue in COPAA v. DeVos

The Individual with Disabilities Education Act (IDEA) requires that states and school districts receiving federal funds under the act must determine whether there exists significant racial/ethnic disproportionality in the identification of children with disabilities or types of disabilities, placement of students with disabilities in particular educational settings, and disciplining of students with disabilities.  When such disproportionality is found, states and school districts must examine their procedures in an effort to determine the causes of such disproportionality and revise their procedures accordingly – things that, as a practical matter, involve adding circumspection to decision-making processes and other actions that tend to generally reduce putative adverse outcomes.  (I use “putative” because assignment to a special education program is actually an adverse outcome only when it is unwarranted.)  Upon finding disproportionality, school districts must also set aside a proportion of certain funds “to provide comprehensive coordinated early intervening services to serve children in the local educational agency, particularly children in those groups that were significantly overidentified.”      

 

IDEA left the manner of determining significant disproportionality to the Department of Education, which until recently had left the matter to the states.  But in 2013 a Government Accountability Office report suggested that the Department should provide greater guidance for determining significant racial/ethnic disproportionality.  The Department then began the process that eventually led to the  Disproportionality Regulation at issue in COPAA v. DeVos with its requirement that significant disproportionality be measured in terms of the in terms of the ratio of the rate at which students from a particular racial/ethnic group experience an outcome to the rate at which all other students experience the outcome (terming it the “risk ratio”).  In 2014, I had commented on the proposed regulation, discussing the unsoundness of the ratio of two rates for measuring demographic differences given the way that the ratio tended to be affected by the frequency of an outcome.  But at the time of the January 4, 2017 post, I was not aware that Department of Education had adopted the regulation in December 2016, doing so just in time for it to take effect before the change in administrations. 

Apparently concerned about excessive federal monitoring of racial differences in educational outcomes in circumstances where there was little reason to believe that racial discrimination was playing a significant role, as well about the possibility that racial minorities were being under-identified for requiring special education services, in February 2018 the Department of Education under Secretary Betsy Devos began the process of postponing compliance with the Disproportionality Regulation from July 2018 (as specified in regulation itself) until July 2020.  According to the proposed regulation, during the postponement period the agency would reconsider the Disproportionality Regulation’s mandate that states and school districts employ the risk ratio to measure significant racial/ethnic disproportionality under IDEA.

There are very sound reasons for the Department of Education to reconsider the Disproportionality Regulation and, indeed, for abrogating it – reasons that ought also to be bases for a numerate court to find the regulation to be arbitrary and capricious.  I discuss some of them on my COPAA v. DeVos web page and in an August 9, 2019 letter to Department of Justice counsel handling the COPAA v. DeVos appeal (which letter I discuss further below).  In addition to the general unsoundness of a ratio of two rates as a measure of association, reasons include problems in comparing one group’s rate with the rate for all other persons (rather than the rate of another racial/ethnic group) and problems in analyzing differing rates of experiencing intermediate outcomes like out-of-school suspensions of ten days or less.   Such reasons, clearly articulated by the Department of Education, not only would have insulated the decision to postpone compliance from successful challenge under the Administrative Procedure Act, but should have obviated challenges from entities concerned about the sound monitoring of demographic differences in educational outcomes.  Understanding the matter, such entities would want to support the Department of Education’s reconsideration, not only of the Disproportionality Regulation, but of everything the agency has been doing with respect to the monitoring of demographic differences in educational outcomes.  See my August 24, 2017 post “Innumeracy at the Department of Education and the Congressional Committees Overseeing It.”

The agency could probably have satisfactorily justified reconsidering the Disproportionality Regulation by simply stating (a) that the regulation was adopted at a time when the agency mistakenly believed that generally reducing adverse outcomes by adding circumspection to decision-making processes – as IDEA encourages school districts to do and as IDEA mandates upon findings of significant disproportionality – would tend to reduce risk ratios for such outcomes; and (b) that, now understanding that that opposite is the case, the agency needs to consider the regulation in light of its new understanding.

Still unaware even that generally reducing an outcome tends to increase risk ratios for the outcome, however, in the Delay Regulation adopted in July 2018, the Department of Education proffered none of the referenced reasons for the postponing compliance with the Disproportionality Regulation.  Moreover, in attempting to the justify the postponement, the agency further promoted the mistaken belief that reducing an outcome would tend to reduce relative differences in rates of experiencing the outcome.

The Delay Regulation largely justified postponing compliance with the Disproportionality Regulation on the basis that there was reason to believe that requiring the use of risk ratios to measure significant racial/ethnic disproportionality in special education programs would (a) incentivize school districts to unjustifiably limit the number of special education identifications in order to reduce risk ratios (referencing a Texas statute limiting special education identifications to 8.5 percent of students in a district), and (b) incentivize school districts to take race/ethnic-conscious actions in order to reduce risk ratios.

Both arguments were misleading, though in somewhat different respects.  The first argument is misleading by suggesting that school districts would believe that generally reducing special education identifications will tend to reduce risk ratios because generally reducing special education identifications in fact will tend to reduce risk ratios, something that is patently not the case.  It is true, of course, that the Disproportionality Regulation’s requirement that school districts measure significant disproportionality in terms of risk ratios will incentivize school districts to limit identifications in order to reduce the ratios, because, like the Departments of Education and Justice, virtually all school administrators mistakenly believe that limiting identifications will tend to reduce risk ratios.  But the remedy for problems arising from that mistaken belief would be to correct the belief.  This is something the Department of Education could easily do and in fact has a pressing obligation do to.  Instead, however, by making the argument the agency bolstered the mistaken belief.

The argument that requiring the use of risk ratios to measure significant racial/ethnic disproportionality will incentivize school districts to take race/ethnic-conscious actions in order to reduce the ratios is also misleading, though for a different reason.  Attaching adverse consequences like restrictions on the use of funds to findings of racial/ethnic differences of a certain magnitude will incentivize affected entities to take race/ethnic-conscious actions to diminish chances of such findings, and such actions will tend to reduce all measures of racial/ethnic disparity (including risk ratios).  But the incentive, which is inherent in IDEA’s requirements regarding significant disproportionality, would apply to all measures of racial/ethnic difference, including a sound one. 

There is nothing special about the risk ratio with respect to this incentive except in the following respects.  Due to the longstanding promotion by the government and the social science community of the mistaken belief that generally reducing an outcome will tend to reduce relative racial/ethnic differences in rates of experiencing the outcome, the principal non-race/ethnic-conscious actions aimed at reducing risk ratios – mainly, some activity or modification of policies that generally reduces putative adverse outcomes – have been actions that actually tend to increase those ratios.  Texas, for example, would tend to have comparatively high risk ratios precisely because of its statutory provision limiting special education identifications.  The high risk ratios that result from generally reducing adverse outcomes create additional incentives for school districts to take race/ethnic-conscious actions to reduce the ratios.  But, again, the solution respecting incentives peculiar to the risk ratio is simply for Department of Education to explain that generally reducing an adverse outcome tends to increase, not reduce, risk ratios for the outcome. 

The use of risk ratios to measures demographic differences can also cause perceptual problems because the high risk ratios that tend to exist when outcome rates are very low can be regarded as extremely disturbing by those who do not understand the relationship between the rareness of the outcome and the size of the risk ratio.  But that, too, is a problem of understanding that can be corrected.

Within days of the adoption of the Delay Regulation, the Council for Parent Attorneys and Advocates, Inc. (COPAA) a disability rights group located in Towson, Maryland, brought COPAA v. DeVos in the U.S. District Court for the District of Columbia.  Asserting standing on the basis of its own interest in monitoring demographic differences, COPAA maintained that the Delay Regulation constituted arbitrary and capricious agency action because the regulation failed to provide a sound justification for the postponement.  Representing the Department of Education in the case, the Department of Justice defended the Delay Regulation on the basis of the same arguments advanced in the Delay Regulation itself, which is to say that the agency sought to lead the court to believe that generally reducing special education identifications or other putative adverse outcomes would tend to reduce risk ratios for the outcomes when in fact the opposite is the case.  In a March 7, 2019 ruling, the district court agreed with plaintiff COPAA and vacated the Delay Regulation.

The government will have difficulty now advancing in the court of appeals justifications for the Delay Regulation that were not advanced in the regulation itself (or even in the district court).  Understanding such arguments, however, the court of appeals should want to avoid requiring the continued implementation of an essentially absurd monitoring scheme.  In any case, however, the Department of Justice is obligated to explain to the court of appeals that it misled the district court and to avoid similarly misleading the court of appeals.      

That is what I urged the Department to do in the aforementioned August 9, 2019 letter (which also provides (at 5-6) some of the most compelling evidence that generally reducing suspensions in fact usually increases relative racial differences in suspension rates and one of the easier to understand illustrations of why that happens).  I also urged the agency to employ whatever resources are necessary to ensure that it understands the issue, and, once understanding it, to address with counsel for plaintiff COPAA whether, in light of things previously unknown to COPAA, it is in COPAA’s interest to continue to pursue the matter.  That letter was written when the government’s brief was due on August 19, 2019.  But on August 12, 2019, the government moved to extend the due date to September 18, 2019.  That may give it time actually to understand the issue and to recognize an obligation to explain to the court of appeals the way that the government misled the district court.  Assuming that the government comes to understand the issue as it bears on COPAA v. DeVos, it ought also to recognize its obligations to explain, with respect to varied matters, that the government has long been misleading the public and other affected entities regarding the effects of policies on measures of racial disparity. 

Possibly, that process might even lead to the government’s recognizing that the billions it spends on research about one demographic difference or another yields virtually nothing of value and a great deal that is misleading because the research never endeavors to determine whether an observed patterns is anything other than the usual consequence of a general change in the frequency of an outcome. 

But if the government does not explain to the court of appeals that it misled the district court, win or lose in the court of appeals, the government might find itself faced with explaining to the Supreme Court that it misled both lower courts.

Appendix – The U.S. Commission on Civil Rights’ Beyond Suspensions Report

In the U.S. Commission on Civil Rights’ July 2019 report Beyond Suspensions: Examining School Discipline Policies & Connections to the School to Prison Pipeline for Students of Color with Disabilities that I mentioned at the outset of this post, the Commission showed itself to be unwilling or unable to understand my testimony explaining that generally reducing suspensions tends to increase the ratio of the black suspension rate to the white suspension rate.  For the Commission promoted the opposite understanding throughout the document.  And in the single instance where it cited a ratio of the black suspension rate to the white suspension rate as evidence of the continuing need for programs of the type recommended in the 2014 Dear Colleague letter  –  the 3.8 figure that remained even after substantial reductions in suspensions between the 2011-12 and 2013-14 school years (at page 148) ­­­– the report failed to mention (or the drafters did not know) that the ratio was an increase from the 3.57 ratio in the earlier school year. 

Further, at least by means of the statement in the third paragraph of the transmittal letter that “[s]tudents of color as a whole, as well as by individual racial group, do not commit more disciplinable offenses than their white peers,” the report implied, not simply that racial bias was a substantial cause of racial differences in discipline rates, but that racial bias was the entire cause.  See my post “The Misunderstood Relationship Between Racial Differences in Conduct and Racial Differences in School Discipline and Criminal Justice Outcomes” (Dec. 20, 2017) regarding the facial implausibility of the claim that there are no racial/ethnic differences in student behavior, given the evident overrepresentation of certain racial/ethnic groups in categories defined by background characteristics and achievement levels where disciplinary problems are more common.  Thus, the document might be deemed an extreme instance of a governmental entity’s misleading the public in a way that promotes racial mistrust.  It is also one where, with respect to promoting the mistaken belief that generally reducing suspensions will tend to reduce the ratio of the black suspension rate to the white suspension rate, more than in any other circumstance, those promoting the belief had reason to know it was not true.  A comparable point might be made about the Commission’s November 2018 report Police Use of Force: An Examination of Modern Policing Practices, which contains statements like (at 33) “even though officers use force in less than 2 percent of all civilian interactions, the use of force against black people is . . . more than three times [as common as against] white people,” without evident awareness that reducing the use of force will tend to further increase the ratio underlying the described racial difference.    

The Beyond Suspensions report also displayed extreme carelessness in what was probably its most provocative finding – that (at 67) “[d]uring the 2015–16 school year, 32 percent of black students with disabilities were suspended once, and almost 40 percent were suspended repeatedly.”  The statement –  which was picked up almost verbatim in a U.S. News account of the report that has already been reproduced on the site of Penn State Social Science Research Institute, which now appears even in an account of the report in India, and which may well be cited again and again – means that almost 72 percent of black students with disabilities were suspended one or more times.  Irrespective of the types of measurement issues I commonly discuss, that unquestionably is a shocking figure and would be a cause for much concern it were a true figure.  In fact, however, as a careful reader of the report will divine, the 40 percent figure is (a) the proportion black students with disabilities made up of students with disabilities who were suspended repeatedly, not (b) the proportion of black students with disabilities who were suspended repeatedly.  By way of example for those who do not find the difference between (a) and (b) to be evident, students with disabilities made up 71 percent of students who were physically restrained during the 2015-16 school year (as the report mentions at page 57 while implying incorrectly that reductions in the use of restraints should reduce, rather than increase, the figure); but only about 1.4 percent of students with disabilities were subjected to restraints.  No doubt that 32 percent figure for single suspensions reflects the same mistake.

By my reading, data from the 2013-14 school year indicate that about 20 percent of black students with disabilities were suspended one or more times and about 10 percent of such students were suspended multiple times.  These figures are probably down a bit for the 2015-16 school year.

By emails of August 8 and August 12, 2019, I urged the Commission to correct this and certain other errors.  As with the Department of Justice’s fulfilling its obligation to explain to the court of appeals in COPAA v. DeVos the way that it misled the district court, however, whether the Commission will make these corrections remains to be seen.