The U.S. has imposed emissions and fuel efficiency standards on automakers since 1975. Despite changing minutiae, the gist remains unchanged nearly 50 years later: a vehicle must travel a minimum distance per gallon of gas, while emitting less than some maximum amount of specified compounds. Regulators don’t amend the Corporate Average Fuel Economy program whenever automakers file a patent because good regulations do not require incessant revisions. As regulators reconcile advances in Artificial Intelligence and Machine Learning (AI/ML) with laws like the Equal Credit Opportunity Act of 1974 (ECOA), they would do well to consider this approach.

At first, this analogy may seem amiss. Anyone can observe an engine or exhaust pipe, but AI/ML models are harder to “see.” The Consumer Financial Protection Bureau (CFPB) warned of “certain complex algorithms, sometimes referred to as uninterpretable or ‘black-box’ models, that make it difficult—if not impossible—to accurately identify the specific reasons for denying credit or taking other adverse actions.” An article in The University of Chicago Law Review noted that “the complexity of machine-learning pricing limits the ability to scrutinize the process that led to a pricing rule.” The Yale Journal of Law and Technology claimed that “[c]redit scorers have trade secrecy on their side; at present, consumers and regulators have no practical way to dig into the models to understand what drives lending decisions.” A cursory Google search could unearth a thousand similar quotations.

These concerns over AI/ML in lending miss two important points. First, new algorithms do not necessitate changes to existing laws and regulations. Second, although rapid changes in AI/ML might seem like an urgent challenge to regulators, the best course of action may well be forbearance.

The ECOA prohibits creditors from discriminating “on the basis of race, color, religion, national origin, sex or marital status, or age.” Furthermore, “each applicant against whom adverse action is taken shall be entitled to a statement of reasons for such actions from the creditor.” Via Regulation B, the CFPB has enforced the ECOA since passage of the Dodd-Frank Act in 2010. The CFPB’s Official Interpretations clarify otherwise ambiguous sections of Regulation B. For instance, the Official Interpretation of Paragraph 9(b)(2) grants lenders significant latitude for compliance related to adverse action notifications:

  • “The regulation does not mandate that a specific number of reasons be disclosed.”
  • “A creditor need not describe how or why a factor adversely affected an applicant.”
  • “The regulation does not require that any one method be used for selecting reasons for a credit denial or other adverse action that is based on a credit scoring system.”

Similarly, Regulation B’s requirement for “an empirically derived, demonstrably and statistically sound, credit scoring system” leave much to the lender’s discretion. It neither endorses nor prohibits any method, noting only that such systems must be “developed and validated using accepted statistical principles and methodology.” Neither the Official Interpretation nor its Supplement mention AI/ML. Their silence implies that nothing about these methods changed the law. CFPB Director Rohit Chopra made this point explicit in 2022: “Companies are not absolved of their legal responsibilities when they let a black-box model make lending decisions. The law gives every applicant the right to a specific explanation if their application for credit was denied, and that right is not diminished simply because a company uses a complex algorithm that it doesn’t understand.”

Unfortunately, Director Chopra’s statement elided the salient fact of credit scores: for most consumers, every model is a complex algorithm that they do not understand. Consider a lender making the following disclosures:

  • “We expanded our data set with reject inference.”
  • “We used gradient descent to obtain the coefficients.”
  • “We binned our continuous variables using weight of evidence.”

For those unfamiliar with credit scoring, these three statements are impenetrable, yet they are commonplace to the most mundane risk-rating scorecards. Yet over ninety-nine percent of people seeking loans will never have the luxury of studying these or a dozen other subjects germane to their credit scores. The ECOA has stood for decades in this environment because, at some level of detail, almost every model becomes a black box to almost every customer. As the 2019 CFPB Fair Lending Report stated, “The existing regulatory framework has built-in flexibility that can be compatible with AI algorithms.”

Credit scores estimate a borrower’s probability of default. Of course, simple problems may not have simple solutions. Just obtaining data has long presented myriad challenges. In-house data sets may have too few defaults to build bespoke models for specific business segments, but a commercial off-the-shelf model could rely on data unrepresentative of a lender’s customers. Historical inequities such as red lining may still bias and distort our inferences today. The ubiquity of online data raises new questions about what exactly constitutes “private.” But a member of the Federal Reserve Board of Governors recently reminded his audience that all of these problems, old and new, are separate and distinct from concerns over AI/ML: “Just as with conventional models, problems with the input data can lead to cascading problems down the line.”

However they obtain that imperfect and limited data, developers all have the same goal: to maximize the model’s out-of-sample performance (i.e., accuracy for use on future applicants). Their model must do two things: rank order applicants by their actual riskiness and estimate a number of defaults that accords with those actually observed. Novel algorithms that cannot improve on those metrics serve no purpose. The pursuit of profits renders banks agnostic about methodology; simplicity is no vice and opacity no virtue.

Regulators also have no interest in the model per se, but rather on its inputs, marginal effects, and group outcomes. For the purposes of regulatory compliance, how developers choose their inputs makes no difference. Rather, they must answer two questions:

  • Are the model inputs protected characteristics or close proxies thereof? In other words, do they serve a legitimate business necessity? The growth of available data complicates this question, as practically all variables show some difference across groups. But the question of where to draw the line has nothing to do with the model.
  • Do they enable clear adverse action codes? An American Express customer saw his credit limit reduced because “other customers who had used their card at establishments where [he] recently shopped have a poor repayment history with American Express." The CFPB likely had such instances in mind when they reiterated that “a creditor will not be in compliance with the law by disclosing reasons that are overly broad, vague, or otherwise fail to inform the applicant of the specific and principal reason(s) for an adverse action.” As with protected characteristics, the choice of model neither exacerbates nor ameliorates this concern.

Understanding a variable’s marginal effect empowers consumers to improve their risk rating in the future. This means that variables should demonstrate “monotonicity.” That is, their effect must always be either positive or negative, even if having a peak effect would improve model fit. As one paper put it, “Monotonicity and other constraints are essential to both interpretability and legal requirements for lending models; monotonicity constraints ensure that as risk factors increase, the estimated risk should also increase.” Much like proxies and adverse action codes, concerns about marginal effects are distinct from the choice of model. The law demands a clear explanation of marginal effects to enable the ECOA’s “twin goals of consumer protection and education.” Even the most intractable of models still allow lenders to meet both of these goals.

Lastly, regarding group outcomes, “The ECOA has two principal theories of liability: disparate treatment and disparate impact . . . Disparate impact occurs when a creditor employs facially neutral policies or practices that have an adverse effect or impact on a member of a protected class unless it meets a legitimate business need that cannot reasonably be achieved by means that are less disparate in their impact.” Again, the choice of model has no relevance for assessing disparate impact.

AI/ML models do not alter the basic incentives and obligations of lenders, consumers, or regulators. For this reason, their use does not call for any significant change to our regulatory regime.

Imposing novel regulations on AI/ML methodologies offers uncertain benefits at best. Meanwhile, the costs of stifling innovation in underwriting would likely fall on lenders and (especially) their least wealthy customers.

Since no equation will ever perfectly forecast creditworthiness, limiting the search for better estimators could obstruct and delay future improvements. Recall that lenders only use AI/ML methods when they believe it improves out-of-sample performance; so it’s not surprising that researchers in Argentina have found that “credit scoring techniques, based on big data and machine learning, have so far outperformed credit bureau ratings in terms of predicting loss rates of small businesses.” AI/ML models also displayed superior performance against traditional models for fintech companies operating in China. Or consider the XGBoost algorithm: only available since 2014, it “has the best out-of-sample performance among a number of . . . standard prediction methods.” on U.S. credit card applications. So long as markets reward better models, regulatory forbearance could pay dividends in the form of more efficient capital allocation by banks.

Traditional models may have worked well enough for those who score well. But to increase access to credit for everyone else, lenders should consider alternative data and novel methodologies. Whoever solves this puzzle will win a huge market: Oliver Wyman estimated that “Nineteen percent of American adults (49 million consumers) don't have conventional credit scores . . . Another 21 million have some limited information in their mainstream credit file, but not enough to generate a conventional credit score.” Fortunately for them, one study found that “even simple, easily accessible variables from a digital footprint match the information content of credit bureau scores. A digital footprint complements rather than substitutes for credit bureau information and affects access to credit and reduces default rates.” A paper from the Bank of International Settlements found that “non-traditional information represents the main reason why Model I [AI/ML] performs better than Model II [conventional].” Researchers at MIT combined “customer transactions and credit bureau . . . [to] construct out-of-sample forecasts that significantly improve the classification rates of credit-card-holder delinquencies and defaults.” The resulting cost savings ranged from 6 to 25% of total losses. But further improvements are not foreordained. Today, a climate of ambiguity could forestall improvements, as “[l]enders have a strong incentive to maintain the status quo in light of regulatory uncertainty and risks raised by fair lending enforcement actions.” Well-intentioned regulators might all too easily reduce credit for the very people they meant to support.

The general public’s awareness of AI/ML methods has surged in recent years. From the introduction of “deep learning” in 2012 to the launch of ChatGPT in 2022, the sense of accelerating progress has grown ubiquitous. But these advances in technology do not require new regulations. At a recent symposium, Acting Comptroller of the Currency Michael Hsu said that his agency did not require any new laws related to AI/ML. Many leading institutions agree with Hsu that the established law can and should apply:

  • The Office of Management and Budget reminded executive agencies that “Federal agencies must avoid regulatory or non-regulatory actions that needlessly hamper AI innovation and growth.”
  • Former Vice Chair of the Federal Reserve Lael Brainard warned that “[r]egulation and supervision need to be thoughtfully designed so that they ensure risks are appropriately mitigated but do not stand in the way of responsible innovations.”
  • The BPI warned against “prescriptive changes [that] may have unintended consequences,” arguing that lenders and borrowers alike are best served by regulations that apply equally to all methodologies.

The Clean Air Act remains intact after countless innovations, from radar-based cruise control to hybrid drive trains. The ECOA might follow a similar trajectory, if regulators remember that legal compliance and technical innovation can and should coexist.

Note from the Editor: The Federalist Society takes no positions on particular legal and public policy matters. Any expressions of opinion are those of the author. We welcome responses to the views presented here. To join the debate, please email us at [email protected].