But the Court’s majority did not address how the First Amendment’s Free Speech Clause interacts with public-accommodations laws. As the Supreme Court predicted decades ago, this conflict has sharpened as state and local governments have simultaneously expanded the definition of “public accommodation” and broadened the classes of persons protected by public-accommodations laws.[2]

This issue is now squarely before the Court in 303 Creative LLC v. Elenis, where the Court must decide “[w]hether applying a public-accommodation law to compel an artist to speak or stay silent violates the Free Speech Clause of the First Amendment.”[3] The case involves a custom website designer who is challenging Colorado’s law—the same one that applied to Mr. Phillips—because it requires her to create wedding websites celebrating same-sex weddings if she does so for opposite-sex weddings.

When public-accommodations laws—or any laws for that matter—regulate speech, courts apply heightened scrutiny. This generally requires courts to “make[] a normative judgment about the ends” and then decide if “the government can and should serve the end through a better-drafted law.”[4] But in the public-accommodations context, how should courts balance the government’s generally legitimate interest in ending discrimination with the constitutional protections for free expression and religious liberty?

Professor Netta Barak-Corren is a legal scholar, professor, and cognitive scientist who developed a study to attempt to answer that question.[5] Her study—the Masterpiece Study—tried to examine whether the Supreme Court’s decision in Masterpiece increased discrimination by the wedding services industry against same-sex couples.[6] She did this by sending fictitious requests from same-sex and opposite-sex couples to creative professionals—photographers, bakers, and florists—before and after the Masterpiece decision. She then measured whether these professionals’ responsiveness to same-sex wedding requests changed after the decision. She claims there was a statistically significant increase in discrimination against same-sex couples seeking wedding photographs, cakes, or floral arrangements. She calls this the Masterpiece Effect.

Professor Barak-Corren then reasons that this evidence may justify state and local governments refusing to grant religious exemptions to their public-accommodations laws.[7] She argues that the Masterpiece Effect is relevant for First Amendment claims and defenses and establishes the government’s interest in denying religious exemptions to public-accommodations laws.[8]

We evaluate this study now because the Supreme Court will evaluate the interaction between public-accommodations laws and the First Amendment in 303 Creative LLC v. Elenis. In that case, several amicus briefs—including one signed by twenty-one states and the District of Columbia—cite the Masterpiece Study to argue that Colorado’s interest in regulating the website designer is sufficient to overcome any First Amendment interests.[9] So the study and its conclusions are directly implicated in the case.

The study has gained attention elsewhere too. Media reports cite the Masterpiece Study.[10] And a local government relied on Professor Barak-Corren as an expert to defend its law.[11] Although her report was ultimately excluded, the case is on appeal.[12] And the study still raises important questions. What is the role of experts in balancing governmental interests and constitutional freedoms? How can studies properly be used to support the government’s or the claimant’s interests?

With those questions in mind, we evaluate the Masterpiece Study. We ultimately conclude that the Masterpiece Study does not prove a Masterpiece Effect. Thus, the study does not justify denying exemptions to laws that infringe on the First Amendment.

Our evaluation proceeds as follows. Part I describes the study and its conclusions. Part II critiques the empirical claims made by the study. We demonstrate that the study’s own data does not show a Masterpiece Effect. We also highlight several methodical assumptions that undermine the reliability of the Masterpiece Study. Part III analyzes the study’s legal conclusions and policy recommendations. We show that from a legal perspective, the study does not justify laws that infringe on free exercise because it doesn’t provide sufficient evidence to support a government’s interest in a law that burdens religious freedom. We also challenge Professor Barak-Corren’s reliance on a purely “consequentialist” view of the law in conjunction with the study. And we discuss how the freedom of expression (as distinct from the freedom of religion) affects the Masterpiece Study’s legal conclusions and relevance. Part IV concludes.

I. The Masterpiece Study

This overview summarizes the Masterpiece Study’s methodology and conclusions. For reasons of space, simplicity, and clarity, we focus on the most relevant methods and conclusions, not all of them.

A. Background, Structure, and Methodology

The Masterpiece Study sought to examine “the consequences of religious exemptions to antidiscrimination laws” and their “normative implications” based on the assumption that the Masterpiece decision would result in a religious exemption for Mr. Phillips.[13] Professor Barak-Corren did so by employing an “auditing” methodology, where researchers posing as customers contact a research subject with a question, elicit a response from him or her, and then record the response.[14]

Professor Barak-Corren surveyed four states: Indiana, Iowa, North Carolina, and Texas. She selected these states because they had similar levels of religiosity and political leanings but varied as to religious freedom restoration acts (RFRA) and statewide or local antidiscrimination laws (AD) that prohibited discrimination on the basis of sexual orientation.[15] All told, there were four different legal regimes: (1) no statewide RFRA or AD (North Carolina), (2) a statewide RFRA and local AD (some jurisdictions in Indiana and Texas), (3) a statewide RFRA and no local AD (the remaining jurisdictions in Indiana and Texas), and (4) no RFRA but a statewide AD (Iowa).[16]

Professor Barak-Corren believed that the Masterpiece decision would “draw extensive coverage and discussion in the public media” and could therefore potentially have “an impact on public attitudes and conduct.”[17] This assumption led her to conclude that Masterpiece “created a favorable setting for the empirical test of the effects (or lack thereof) of religious exemptions and sexual orientation discrimination.”[18] To measure these potential effects, the study surveyors sent fictitious email inquiries to the creative professionals typically involved in recent court proceedings—florists, bakers, and photographers.[19]

The study sent four waves of emails. Waves 1 and 2 occurred in May 2018, before the Masterpiece ruling. The study surveyors sent email messages from fictitious same-sex couples (Wave 1) followed by messages from fictitious opposite-sex couples (Wave 2).[20] Names suggested the couple’s sexual orientation.[21] Same-sex couples received a much higher positive response rate (70.8% positive response rate) in Wave 1 than opposite-sex couples did in Wave 2 (58.7% positive response rate).[22] The Masterpiece Study attributed the decline in positive response rates between Wave 1 and Wave 2 to respondent “attrition.”[23]

Professor Barak-Corren sent Waves 3 and 4 several weeks after the June 2018 Masterpiece ruling.[24] In order to avoid spurious correlation between sexual orientation and unmeasured characteristics in each respective wave, Professor Barak-Corren randomly blended the sexual orientation for Waves 3 and 4.[25] Professor Barak-Corren also did this because she recognized the attrition problem in Waves 1 and 2 and wanted to mitigate that problem in Waves 3 and 4. So approximately half of the creative professionals received a same-sex inquiry in Wave 3 while the other half received an opposite-sex inquiry. The inquiries also blended same-sex and opposite-sex couples in Wave 4.[26]

A non-response was considered a rejection. Professor Barak-Corren further assumed that rejections were based on discriminatory intent. Therefore, in Waves 3 and 4, Professor Barak-Corren considered higher instances of non-responses for one group as evidence of discrimination.[27] Stated simply: a non-response to a request for a good or service for a same-sex wedding was considered sexual orientation discrimination.[28]

Overall response rates by Wave were as follows:[29]

 

 

B. Findings and Conclusions

The study concludes that “post-Masterpiece inquiries from a same-sex couple had a 66.3% chance of receiving a positive response [and] [e]quivalent inquiries from an opposite-sex couple have a 75.5% chance of being answered positively.”[30] The study then attributed this 9.2% difference solely to the identity of the couple.[31] Professor Barak-Corren concluded from this data that there is a Masterpiece Effect—i.e., that Masterpiece caused creative professionals to decline to provide services for same-sex weddings more frequently after the decision.

Professor Barak-Corren explained that broad coverage from “mainstream,” “progressive,” and “conservative” news outlets “had an expressive effect on” creative professionals which caused a change in their “perceptions of the social norm regarding service refusal” for same-sex weddings and emboldened them to more often decline such inquiries.[32] In her view, this coverage created a new perceived social norm which caused professionals to be more willing to decline to provide certain goods and services based on their religious beliefs.[33]

To attempt to confirm the results, the study compared four different categories of results for (a) all businesses; (b) businesses in what is called the control group (businesses that were first contacted after Masterpiece); (c) “Pre-Masterpiece Gay Friendly Businesses” (businesses that positively responded to same-sex inquiries before Masterpiece); and (d) “Pre-Masterpiece Generally Keen Businesses” (businesses that positively responded to both same-sex and opposite-sex inquiries before Masterpiece).[34] The study tracked each category separately. These different categories measured the Masterpiece Effect across creative professionals’ profiles, comparing those willing to serve same-sex couples before Masterpiece (“gay friendly”) to businesses that took all customers (“generally keen”) without regard to sexual orientation.[35]

Professor Barak-Corren concludes that the results of her study “provide the missing piece to the puzzle of applying a strict scrutiny analysis.”[36] She argues her study is especially relevant to the “least restrictive means” component of strict scrutiny because it supports universal enforcement of antidiscrimination laws.[37] In her view, the Masterpiece Study illustrates that any exemptions from antidiscrimination laws “substantially detract[] [from the government’s goal of ending discrimination] in most regimes, by substantially expanding discrimination against same-sex couples.”[38]

II. The Masterpiece Study’s Faulty Methodology and Conclusions

 

This Part explores the six main problems with the study’s methodology. (1) The study’s data shows discrimination against opposite-sex couples before Masterpiece, (2) the study fails to adequately consider the regression to the mean to account for reduced responsiveness in Waves 3 and 4, (3) the study uses non-responses to determine discrimination, (4) the study has a “gay friendly” fallacy, (5) the study deploys a pseudo-control group, and (6) the study fails to measure the audience of the Masterpiece decision among the audited population, i.e. creative professionals.[39]

A. The Study’s Data Shows Discrimination Against Heterosexual Couples Before Masterpiece

The study’s pre-Masterpiece data shows same-sex couples were more likely to receive an explicitly positive response (71% positive response rate) to their inquiry than opposite-sex couples (59% positive response rate).[40] Conversely, and as a necessary corollary, opposite-sex couples were more likely than same-sex couples to receive an explicit decline or a non-response to their inquiries for wedding services. Because the Masterpiece Study counts non-responses as discrimination, the prevalence of explicit denials and non-responses to opposite-sex couples compared to same-sex couples before Masterpiece would suggest there was pre-Masterpiece discrimination against opposite-sex couples.

This counterintuitive finding makes it easier to detect a supposed change after Masterpiece that creates an illusory Masterpiece Effect. The study uses the pre-Masterpiece comparisons between responsiveness to opposite-sex and same-sex inquiries to support its conclusion of post-Masterpiece same–sex discrimination. It was easier to show that responsiveness to opposite-sex couples increased after Masterpiece compared to responsiveness to same-sex couples because of the low pre-Masterpiece responsiveness to opposite-sex couples.

Even if perfect equality in responsiveness for same-sex and opposite-sex inquiries were found post-Masterpiece, under the study’s logic, one could conclude that Masterpiece caused an increase in discrimination against same-sex couples. Post-Masterpiece, creative professionals responded positively or cooperatively to opposite-sex inquiries about 58% of the time.[41] But pre-Masterpiece, creative professionals responded positively or cooperatively to same-sex inquiries about 64% of the time.[42] So even if same-sex and opposite-sex positive responsiveness were the same after Masterpiece, Professor Barak-Corren’s logic would still have found discrimination against same-sex couples.

This raises other problems too. The study attributes the differences in non-responses between Waves 1 and 2 to “attrition,” but it attributes the differences in non-responses in Waves 3 and 4 to discrimination.[43] The study admits its finding of pre-Masterpiece discrimination against opposite-sex couples is “tenuous,”[44] but it later concludes that requests for services for same-sex weddings were more likely to be declined post-Masterpiece.[45] For the sake of consistency, the Masterpiece Study needed to characterize non-responses the same across waves—either they should be classified as discriminatory non-responses (which would be inaccurate in our view, as explained below) or as non-responses due to attrition. But not both. Professor Barak-Corren does not offer a defensible explanation for this inconsistency.[46]

Compounding the problem, the study uses the term “attrition” incorrectly. Professor Barak-Corren claims that attrition is “common to studies.”[47] “Attrition” in social science refers to the phenomenon in survey studies when people know they are being studied at multiple points across time and drop out of the study before it concludes.[48] The creative professionals in the Masterpiece Study did not know they were in a study and therefore were not in a position to “drop out” in the conventional social science sense.

Professor Barak-Corren also does not account for how other variables associated with Waves 1 and 2 might account for the differences in responsiveness between Waves 1 and 2 and Waves 3 and 4. Like most auditing studies, the Masterpiece Study was intended to detect the “signal in the noise.” This refers to the idea that social scientists must separate the variable of interest—the “signal,” which, in this case, is Masterpiece’s effect on discrimination—from the randomness of numbers arising in the course of the measurement—the “noise.”[49] But the inquiries were sent at different times, and the scripts contained different wording.[50] The higher non-response rate of Wave 2 (i.e., inquiries from opposite-sex couples) likely arose from the “noise” associated with the timing of contact and/or the wording of the inquiries and not discrimination against opposite sex couples.[51] We reach this conclusion because the “signal” in Waves 1 and 2 amounts to an unusual result: widespread pre-Masterpiece discrimination against opposite-sex couples. This result is unusual, and Professor Barak-Corren does not accept it.[52] But the study never adjusts for or addresses these possibilities.[53]

These statistical differences in non-responses between Waves 1 and 2 cast serious doubt on any ability to draw inferences from changes across waves pre- and post-Masterpiece using this data.[54] That is especially true given how the study attributes different causes to non-responses in Waves 1 and 2 compared to Waves 3 and 4.

B. The Study Fails to Account for Regression to the Mean

As we have explained, the Masterpiece Study experienced significant “attrition” between Waves 1 and 2. This irregular pattern of responses prevented the study from detecting potential discrimination pre-Masterpiece.[55] Professor Barak-Corren attempts to get over the attrition hurdle by measuring the change in responsiveness to inquiries for same-sex wedding services by pre-Masterpiece “gay friendly” businesses.[56] “Gay friendly” businesses, as Professor Barak-Corren uses the term, are businesses that positively responded to requests for same-sex wedding services in Wave 1.[57]

Professor Barak-Corren claims that previously gay friendly businesses “randomly contacted by opposite-sex or same-sex couples after the decision was rendered respond[ed] less favorably to same-sex couples” after Masterpiece.[58] But the oddly high level of positive responses to inquiries for same-sex wedding services pre-Masterpiece makes it much easier to find a significant decrease in responses to same-sex wedding inquiries after the ruling. Claiming to detect an effect in such a situation is a textbook example of the “regression fallacy.” Regression in this sense “describes a tendency of extreme measurements to move closer to the mean when they are repeated.”[59] This is a well-established, well-investigated phenomenon across a wide range of activities.[60]

Consider an example. In a classic article, Amos Tversky and Daniel Kahnman described how regression to the mean might work with students.[61] If one selects the ten top scoring children on an aptitude test, he will usually observe a performance decrease in a second test. Conversely, if one selects the ten worst scoring students, he will typically find their performance to improve on a subsequent test. In each case, the students’ performances moderate to their average performance level.[62] Tversky and Kahnman explain that failure to recognize regression to the mean can lead to “spurious” causal explanations and counter-productive policies.[63]

When regression to the mean is possible, the analyst must ascertain whether high performance in the first testing arose from a statistical aberration.[64] If the first measurement derives from a performance above the norm, a later measurement, which may well be average, will appear to be a slump.[65] The Masterpiece Study does not delineate these possibilities. The supposed effects of Masterpiece are measured against an unusually favorable pre-Masterpiece responsiveness to same-sex inquiries in the group that is selected precisely because they positively responded to same-sex inquiries—i.e., gay friendly businesses. Under these circumstances, a subsequent decline in the professionals’ responsiveness is unsurprising. In fact, such a slump is likely an artifact of regression to the mean—rather than a result of attitudinal changes post-Masterpiece—based on a pre-Masterpiece sampling consisting of a high-scoring gay-friendly group.[66] The Masterpiece Study could have tried to adjust its methods and conclusions to account for regression to the mean with tools like an analysis of covariance.[67] But the study did not employ this tool or any others and therefore failed to account for the possibility of regression to the mean.

C. The Study Measures Discrimination via Nonresponses

Another concern is that the study counts failure to respond to an email as discrimination. Professor Barak-Corren noted that, for a variety of reasons, “no response” was the “most common form of declining service.”[68] Carefully controlled and disseminated auditing studies that show differential non-responses can comprise evidence of general discrimination.[69] But increased discriminatory sentiment is typically manifested in both a higher non-response rate for same-sex couples and a higher explicit rejection rate, because the discriminatory sentiment operates in both ways.[70] So if there was a post-Masterpiece increase in discrimination, and we assume for the sake of argument that non-response correlates to and approximates discrimination, we would also expect to measure an increase in the directly measurable form of discrimination: explicit declines.

Explicit rejections are a more definite signal that a creative professional has intentionally declined the request. Measuring these rules out circumstances where creative professionals did not read the request, were too busy to respond, or did not respond to the inquiry for a variety of other reasons.[71] To be fair, Professor Barak-Corren acknowledges that non-responses in Waves 3 and 4 could have had other causes.[72] She defends the study’s reliance on nonresponses by claiming that non-responses did not “randomly and therefore equally” distribute across couple types and therefore showed discrimination.[73] But the study’s failure to account for the regression to the mean in responsiveness in Waves 3 and 4 eliminates the study’s ability to draw any distribution inferences from Waves 3 and 4.

We analyzed the Masterpiece Study’s original, anonymized data, but we only considered explicit rejections to investigate potential discrimination.[74] For opposite-sex couples, the explicit rejection rate before Masterpiece was 5.6% while after Masterpiece that rate was 8.7%. This difference is statistically significant at p=.011.[75] The same-sex explicit rejection rate increased from 7.3% to 9.7%, but with an insignificant p-value of .06.[76] For that reason, we are unable to attribute the increase from 7.3% to 9.7% to an actual increase in discrimination, as opposed to random happenstance from the noise. So while explicit rejections increased after Masterpiece for opposite-sex couples, we cannot conclude that explicit rejections increased for same-sex couples after Masterpiece.

D. The Study Suffers from a “Gay Friendly” Fallacy

According to the study, creative professionals who agreed to serve same-sex couples before Masterpiece—“gay friendly” businesses—showed lower responsiveness to same-sex couples in the post-Masterpiece waves. As discussed above, this shift was statistically expected regardless of whether there was in fact an underlying change in attitudes because of the unusually high Wave 1 responsiveness and regression to the mean. Even so, this apparent change invites a question not addressed in Professor Barak-Corren’s study: How did businesses that declined requests for services for same-sex weddings before Masterpiece react after Masterpiece?

To answer this question, we identified businesses that explicitly declined a same-sex inquiry before Masterpiece. Then we looked at how many in this group also explicitly declined to serve opposite-sex couples before Masterpiece. Finally, we analyzed how many in the same group explicitly declined same-sex and opposite-sex inquiries after Masterpiece. This essentially inverts the Masterpiece Study’s “gay friendly” analysis whereby it looked at businesses that positively responded to same-sex inquiries before Masterpiece and then compared that group’s responsiveness to same-sex inquiries after Masterpiece.

To begin, we sampled businesses that explicitly rejected requests to provide services for a same-sex wedding in Wave 1. We call this group “gay antagonists” because of the express rejection of same-sex inquiries (although we don’t know their reasons for declining service). Of that group, we found that 59% also explicitly refused to serve opposite-sex couples. In other words, of all of the creative professionals that explicitly declined same-sex inquiries pre-Masterpiece, 59% also explicitly declined opposite-sex inquiries. The pre-Masterpiece response disparities were highly statistically significant based on our analysis ([p<.001]).[77] Again, this means there is a less than one in a thousand chance that this difference arose from chance, so we can be fairly certain that there was a real pre-Masterpiece gap between rejecting same-sex and opposite-sex couples for the “gay antagonist” group, especially because the social sciences only demand a less than 1 in 20 chance that the relationship is due to random noise.[78]

However, after the Masterpiece decision, the same gay antagonist group was no more likely to expressly deny a same-sex inquiry (59% rejection rate) than an opposite-sex inquiry (52%) (the difference between these two rejection rates is a statistical tie at p=.39; again a statistical tie because p exceeds .05).[79] For this group, there was a statistically significant decline in same-sex explicit rejections post-Masterpiece. Because of the gay antagonist group’s composition—i.e., only creative professionals who explicitly declined a same-sex inquiry before Masterpiece—the group explicitly declined same-sex inquiries 100% of the time before Masterpiece. After Masterpiece, though, this same group explicitly declined 59% of the time. On the other hand, opposite-sex rejections by this group did not significantly decline—those rejections went from 59% before Masterpiece to 52% after.

Given this data, and using the study’s logic that an explicit decline is an act of discrimination no matter the stated reason for the decline, Masterpiece appeared to cause a change of heart among formerly “gay antagonistic” creative professionals. These professionals became more accepting of same-sex couples as demonstrated by the decline in the explicit rejection rate post-Masterpiece. In short, they discriminated less after Masterpiece. Of course, we are not arguing that Masterpiece caused more people to support same-sex marriage. Rather, we are simply demonstrating that if one selects only those respondents that score high on a certain variable (here, explicit declines to same-sex inquiries), they will naturally shift toward the mean in later measurements, making it (falsely) appear they changed their minds.

If Masterpiece caused a real and statistically detectable increase in decisions to refuse inquiries for same-sex weddings, the conclusions would not differ so significantly based on how we cut the data (e.g. express rejections versus non-responses) and defined our variables. Indeed, changing focus from the “gay friendly” group to the “gay antagonist” group shows creative professionals expressly declined same-sex inquiries less after Masterpiece, the exact opposite of a purported Masterpiece Effect. In reality, the stories told by the data change depending on the operationalization of the variables. Reliable conclusions robust to a variety of alternative specifications should not be easily called into question merely because the test is set up differently.[80]

E. The Study Has No Reliable Control Group

Typically, scientific studies require at least two groups: the group receiving the treatment and the control group.[81] The treatment group is exposed to the treatment expected to lead to a particular outcome, while the control group is not.[82] Studies which purport to measure the effect of a treatment or to evaluate the cause of a change in behavior should test a treatment group and a control group.[83] This ensures that the effect of the treatment is attributable to the treatment rather than to the particular composition of the experimental group, overall trends that might affect the experimental and control group, or an otherwise unique feature of the experimental group.

Professor Barak-Corren claims her control group is the businesses she contacted for the first time after Masterpiece “to evaluate the possibility that the repeated measurement of the experimental procedure had an independent effect on business behavior.”[84] Professor Barak-Corren claims that it was important to use a control group because it supposedly allowed the study to measure post-Masterpiece discrimination against same-sex couples even though the study couldn’t make this determination pre-Masterpiece because of the attrition issue.[85]

In the Masterpiece Study, the “treatment” was being exposed to or becoming aware of the Masterpiece decision. That is because the Masterpiece decision is the variable hypothesized to cause the change in the dependent variable, i.e., whether creative professionals respond differently to inquiries for same-sex wedding services. In these circumstances, a true control group would be exposed to more or less the same conditions, but would lack exposure to the Masterpiece ruling.[86]

But the Masterpiece Study has no methodologically valid control group. As we explain in more detail in the next section, there is no valid control group because the Masterpiece Study never measures whether the creative professionals were or were not exposed to the particular treatment—the Masterpiece decision. Without knowing that information, it is impossible for the “control group” to independently verify any causal link between Masterpiece and creative professionals’ responses or non-responses to inquiries post-Masterpiece.[87]

To truly evaluate the effects, if any, of a judicial decision like Masterpiece, a correct study would need to evaluate the decision in a localized market and then compare the results from that jurisdiction to persons in other jurisdictions who were not aware of the decision.[88] By doing so, the control group would allow observation of a population unaffected by the decision. Only then could the study potentially isolate the Masterpiece Effect by taking into account trends and other factors that were affecting results at the same time. Conversely, if the control group exhibited behavior similar to the treated group, any pre-and-post Masterpiece change would be attributed to factors other than Masterpiece. While circumstances may have precluded formation of a true control group in the Masterpiece Study, this omission undermines the study’s conclusions because there was no opportunity to test the hypothesis against a population not exposed to Masterpiece. So any Masterpiece Effect cannot be separated from the myriad of explanatory factors we have presented.

F. The Study Did Not Measure Audience Awareness of Masterpiece

As noted, the Masterpiece Study is an audit-style study which sought to measure how the Masterpiece decision affected the behaviors of creative professionals in the wedding services industry by sending fictitious inquiries to those professionals.[89] Typically, to test how a Supreme Court decision changes individuals’ attitudes or behaviors, there must be information about whether the public or the individual knew about the decision and the individual’s attitude toward the decision.[90] That makes common sense. If we want to measure whether a particular book about low carbohydrate diets caused someone to change diets, we first need to know whether he or she read the book.

Professor Barak-Corren argues that that Masterpiece emboldened creative professionals to exercise their rights to religious exemptions from public-accommodations laws, which caused them to more frequently decline same-sex inquiries after Masterpiece.[91] To feel so emboldened, the creative professionals must have known about the decision. A creative professional could potentially learn of the case by reading the opinion, hearing about the opinion from media reports, or by hearing about the opinion second-hand by speaking with others who were aware of the case. The creative professional could even be exposed to the decision without explicitly knowing about it if others conveyed its concepts to the professional.[92]

But the study did not measure whether the creative professionals were aware of the decision or, if they were aware of the decision, how they became aware of it.[93] These omissions are important. Of course, if creative professionals did not know about the decision at all and did not notice their peers changing their behavior, it is impossible to conclude that the Masterpiece decision caused any behavioral changes.[94] Unaware creative professionals could not have changed their responsiveness to same-sex wedding inquiries as a result of Masterpiece. And even if the Masterpiece Study measured whether the studied professionals knew about the decision—either directly or as told by others—we would still need to know what they understood from the decision.[95] That is necessary for two reasons.

First, the ruling did not establish a religious exemption for creative professionals to decline to create custom works for same-sex weddings that conflict with their religious beliefs. Instead, the Court held that Colorado’s “hostility” towards Masterpiece Cakeshop and Mr. Phillips violated his religious freedom under the First Amendment.[96] At the same time, as Professor Barak-Corren acknowledges, “the decision also affirmed the need for AD laws to protect against sexual orientation discrimination in the marketplace.”[97] For example, the Court explained that it is the “general rule” that “religious and philosophical objections . . . do not allow business owners and other actors in the economy and in society to deny protected persons equal access to goods and services under a neutral and generally applicable public-accommodations law.”[98]

So Professor Barak-Corren’s behavioral claims require several inferences. One must first infer that the creative professionals misunderstood the case as granting a religious exemption instead of protecting against religious hostility. Next, one must infer that the public broadly misunderstood the decision in this way because there could only have been a change in creative professionals’ “perceptions of the social norm regarding service refusal” if the public widely misunderstood the decision.[99] The Masterpiece Study does not opine on how a proper understanding of Masterpiece as a religious-hostility case would have influenced professionals’ behaviors or attitudes or the study’s conclusions.

Second, different media outlets reported on the decision differently. Professor Barak-Corren sampled media outlets she deemed “mainstream,” “progressive,” and “conservative.”[100] In Professor Barak-Corren’s estimation, mainstream outlets characterized the decision as a “narrow” one that “did not resolve the big constitutional questions at issue.”[101] The progressive outlets criticized the decision and specifically “voiced concerns that Masterpiece Cakeshop will grant objectors a license to discriminate.”[102] And the conservative outlets explained the decision was a “victory” and “express[ed] significantly less reservations about its scope.”[103]

These characterizations expose a hidden assumption in the study’s media analysis: creative professionals rely primarily on “conservative” outlets for their news. This assumption is necessary to support the study’s conclusion that Masterpiece caused a change in social norms which encouraged creative professionals to more frequently not respond to inquiries for services for same-sex weddings.[104] That is so because mainstream and progressive media outlets would presumably not have changed social norms on the topic—mainstream outlets claimed the opinion was narrow and progressive outlets criticized the decision.[105] By contrast, in Professor Barak-Corren’s view, conservative outlets described the decision in broad terms and did not “mention its recognition of the important role of AD laws in protecting against sexual orientation discrimination.”[106]

But the Masterpiece Study does not specify which type of media the creative professionals observed. And at least some commentators from media not surveyed by the Masterpiece Study, claimed, at the time of the decision, that “‘[n]arrow’ has emerged as one of the most common descriptions of the Supreme Court’s decision” in Masterpiece.[107] If the public commonly believed the decision was narrow, there would be no change in social perceptions and no basis for professionals to feel emboldened to decline inquiries for services related to same-sex weddings.

Creative professionals also could have different perceptions of the Masterpiece decision if they saw mainstream or progressive coverage combined with conservative coverage. For example, Professor Barak-Corren cites one study that examined public embrace of Supreme Court rulings based on the degree and type of media coverage.[108] That study found that the media can influence the public’s opinion of Supreme Court rulings, but the degree of influence depends on whether coverage is either one-sided supportive coverage or two sided (both supportive and critical).[109] The study concluded that the type of media coverage dictates whether widely-reported cases do or do not change public opinion.[110] Likewise, Professors Johnston and Bartels found that individuals’ attitudes towards the Supreme Court depend on the type of media they consume.[111] Consumers of “sensationalist media”—political talk radio and cable television—are more likely to have negative attitudes about the Court compared to consumers of “sober media”—newspapers and network news.[112] To that end, Professors Johnston and Bartels concluded “that not all information concerning the courts is identical and, thus, where one gets their knowledge is determinative of their subsequent attitudes.”[113] In contrast, the Masterpiece Study does not examine subjects’ media exposure in any robust or methodologically systematic way. Nor does Professor Barak-Corren systemically evaluate how she classifies mainstream, progressive, or conservative media.[114] She instead makes an anecdotal sampling without measuring the key data point, namely, the impact of the media reporting of the ruling on the attitudes and behaviors of creative professionals.

Even creative professionals who viewed only “conservative” outlets may have had different impressions of Masterpiece depending on the articles they read. Professor Barak-Corren cites seven conservative articles.[115] Four articles explained that the case protected religious freedom while also describing the decision as narrow.[116] It is impossible to say what impression of the case a reader might take away from this nuanced coverage. Meanwhile, three articles could fairly be described as promoting the decision as granting a religious exemption to public-accommodations laws. But it would be foolhardy to claim these three articles caused a seismic shift in public perceptions. One was a news release by the Family Research Council,[117] an organization whose “mission is to advance faith, family, and freedom in public policy and the culture from a biblical worldview.”[118] The study provides no information about how widely this release circulates, and, in any event, subscribers to this release likely would have already had religion-based objections to providing services to celebrate same-sex weddings. Another article quoted Jack Phillips as describing the case as “‘a big win’” without elaboration.[119] Only one article from an actual media source (The Daily Signal) said Masterpiece offered broader protections for religious liberty despite its seemingly narrow opinion.[120]

III. The Masterpiece Study Gets the Law Wrong

Having shown that the data does not support the Masterpiece Effect, we will now turn to the study’s legal underpinnings and conclusions. We conclude that general auditing studies like the Masterpiece Study are ill-equipped to shed light on how to reconcile public-accommodation laws with the First Amendment. We evaluate both religious freedom and free speech claims and defenses here because businesses often defend themselves against public-accommodations laws by invoking the expressive character of the activity being regulated and their free exercise rights.[121]

Courts apply strict scrutiny to laws that infringe on religious liberty and free speech.[122] The federal RFRA and state RFRAs (generally) codify strict scrutiny.[123] Strict scrutiny is “the most demanding test known to constitutional law.”[124] It requires the government to demonstrate that the law furthers a compelling government interest in the most narrowly tailored way to achieve that interest.[125] In evaluating compelling interests, courts look beyond “broadly formulated interests justifying the general applicability of government mandates.”[126] This means the government must have a compelling interest in declining an exception for a particular claimant.[127] As for narrow tailoring, if the government can achieve its purposes in a manner that does not burden speech or religion, it must do so.[128]

Generally, governments claim that applying public-accommodations laws further compelling interests by ensuring the public has equal access to goods and services and by preventing dignitary harms associated with being denied a good or service. Indeed, those are the interests advanced by Colorado in 303 Creative LLC v. Elenis in applying its law to a website designer.[129]

The Masterpiece Study generally analyzed the first interest—how religious exemptions could affect the government’s interest in ensuring access to wedding-related services.[130] So we evaluate that interest first. We also analyze whether the Masterpiece Study can support an interest in preventing dignitary harms. After showing that the Masterpiece Study cannot generally be used to support either interest, we evaluate the consequentialist approach Professor Barak-Corren uses to evaluate public-accommodations laws and religious liberty. Finally, we discuss a material omission in the Masterpiece Study in the context of 303 Creative LLC v. Elenis: an inquiry into the likely effect of a free speech-based exemption from public-accommodations laws for objections to providing goods or services that express ideas and values that conflict with the creative professionals’ views or beliefs.

A. Broadly Formulated Access Interests Don’t Satisfy Strict Scrutiny

We start with the often-asserted claim that public-accommodations laws serve a compelling governmental interest by ensuring equal access to goods and services. As a matter of fact and as a matter of law, the Masterpiece Study does not demonstrate that granting a religious exemption limits access to wedding-related goods or services for same-sex couples.

We have shown that the Masterpiece Study did not demonstrate a Masterpiece Effect which limited the availability of wedding-related services to same-sex couples.[131] But even assuming the Masterpiece Effect, Professor Barak-Corren concedes that her study does not show a lack-of-access problem.[132] So factually, the Masterpiece Study does not support the argument that the government has a compelling interest in ensuring equal access to goods and services for same-sex weddings by eliminating religious exemptions.[133] When access is not denied, there are serious questions about whether the state’s interest lies in preventing discrimination throughout the economy or whether the government is regulating religious events or observances.

As a matter of law, the study inappropriately seeks to use generalized data to resolve case-specific disputes. But courts must scrutinize the asserted harms caused by granting specific exemptions in specific cases. For example, when a law allegedly violates the religious freedom of a business owner, the question is not whether the government has a compelling interest in enforcing antidiscrimination laws generally, but whether it has a compelling interest in denying a religious exemption to that particular business.[134]

The Masterpiece Study is not equipped to address these nuances. The study did not examine whether exempting any particular public accommodation would eliminate the market alternatives Professor Barak-Corren identified. And the study acknowledges that “independent vendors in one area could be different than independent vendors in another area.”[135] Even so, the study makes a blanket statement about the Masterpiece Effect: that any religious exemption for any public accommodation would increase the willingness of creative professionals to object to providing certain services that violate their religious beliefs.[136] But as Justice Samuel Alito observed in his concurring opinion in Fulton, the availability of alternative services to same-sex couples undercuts the government’s interest when the service providers do not enjoy market domination.[137] The Supreme Court made a similar point in Hurley v. Irish–American Gay, Lesbian & Bisexual Group of Boston where the government had no compelling interest in forcing parade organizers to include a banner when alternative parades were “presumably” available.[138]

For that reason, the Masterpiece Study’s lack of evidence about specific objectors in specific jurisdictions or market alternatives in those jurisdictions casts doubt on whether the study can apply in any particular case. Professor Barak-Corren’s study cannot be used for particular cases because she generally makes no specific findings within specific jurisdictions.[139] For example, a photographer in Austin, Texas (which is subject to a state RFRA and a local AD), might respond differently than one in rural Texas with different cultural leanings and subject only to the state RFRA. While the study examines general religiosity as a factor, it does not measure these important intrastate differences or the nuances arising from a diversity of attitudes about same-sex marriage. Because it fails to measure these distinctions, the study is not useful as specific evidence in any one jurisdiction because it imparts no finding relevant to the jurisdiction under review.

As a matter of real-life experience, the access issue has little force because of the business risk to creative professionals of declining to create expressive goods for same-sex weddings. Businesses have faced significant public backlash for declining to provide goods for same-sex weddings or to support same-sex marriage more broadly. This backlash has often resulted in lost profits,[140] closed businesses,[141] and limited access to markets.[142] Some public-accommodations laws impose criminal penalties, which further disincentivizes same-sex wedding inquiry declinations.[143] And the same-sex wedding industry is growing and profitable.[144] All of these potential penalties and losses associated with only providing goods and services for opposite-sex weddings naturally deter most creative professionals from declining same-sex wedding inquiries. Religious objectors may already experience a penalty in the marketplace in the form of potential penalties or profits foregone on serving same-sex weddings. A law and economics approach would suggest creative professionals suffer by refusing same-sex business and that a decision to decline this business is not economically rational.[145]

B. Broadly Formulated Dignitary Harm Interests Don’t Satisfy Strict Scrutiny

Next, we address the claim that public-accommodations laws serve a compelling government interest by reducing the dignitary harm associated with being declined a service. The claim is that denial of a product or service by reason of suspect class status may be an affront to personal dignity. But if this is the interest in view, it is important to pinpoint the reason for the decline in service. The Supreme Court has described this interest in the context of outright refusals to serve a particular class of persons because of their status in the provision of basic goods and services.[146] These cases have not involved legitimate religiously based objections to providing custom, expressive goods.

But the Supreme Court has consistently rejected possible dignitary harms as a justification for compelling or eliminating religiously or philosophically motivated speech.[147] Relatedly, the Supreme Court has also made clear that religious based objections to same-sex marriage cannot serve as the basis for personal affront. In Masterpiece, the court said that “gay persons could recognize and accept without serious diminishment to their own dignity and worth” legitimate declines in service based on sincerely held religious beliefs.[148] Likewise, in Obergefell, the Supreme Court described religiously-based objections to same-sex marriage as “decent and honorable” and made sure to emphasize that those “beliefs are not disparaged here.”[149]

In any event, the most frequent form of declining service in the Masterpiece Study was a non-response. Those types of declines are especially weak support for any supposed violation of a dignity interest. A potential customer who receives a non-response would have no way of knowing the reason for the non-response.[150] And service providers decline to reply to requests for services all the time for a myriad of reasons that have nothing to do with the status of the person making the request.

The creative professional’s dignity is also worth considering. Prosecuting a creative professional and stripping him or her of a livelihood imposes a choice between martyrdom and a broken conscience.[151] In Masterpiece, Colorado’s public-accommodations law forced Mr. Phillips to cease making wedding cakes, which caused layoffs and a significant loss of business.[152] In these cases, the creative professional must repeatedly violate his conscience or face financial ruin. That’s generally an unconstitutional choice: “In our constitutional tradition, freedom means that all persons have the right to believe or strive to believe in a divine creator and a divine law. For those who choose this course, free exercise is essential in preserving their own dignity and in striving for a self-definition shaped by their religious precepts.”[153]

C. The Study Emphasizes a Consequentialist Approach to Law

The Masterpiece Study seeks to contribute “to the consequentialist debate on religious exemptions, by studying . . . the effects of religious exemptions on sexual orientation discrimination.”[154] In law, “rule consequentialism[] evaluates legal rules solely based on their consequences.”[155] On this view, legal rules “may (or must) go into effect if and only if justified by their consequences.”[156] This approach can be contrasted with nonconsequentialism which “does not ignore consequences entirely, but instead denies that the rightness or wrongness of our conduct is determined solely by the goodness or badness of the consequences.”[157]

Professor Barak-Corren pursues a consequentialist theory of law because she believes the Supreme Court “has consistently cited consequentialist concerns (or lack thereof) in rejecting (or granting) requested religious exemptions.”[158] In her view, consequentialism was at least in the Justices’ minds as they considered Masterpiece.[159] Professor Barak-Corren explains her position that “[i]n constitutional law, as elsewhere, arguments about outcomes should rest on actual data.”[160]

Based on the Masterpiece Effect, Professor Barak-Corren concludes that religious exemptions should generally be avoided to prevent increased non-responses to same-sex wedding inquiries.[161] She argues that the Masterpiece Study shows that Masterpiece “substantially detracted” from public-accommodations laws’ goal of ending discrimination “in most regimes, by substantially expanding discrimination against same-sex couples.” [162] She concludes by suggesting “these results vindicate states that currently insist on enforcing AD laws without providing exemptions.”[163]

But the problem with the Masterpiece Study’s consequentialist theory is that it asks courts to predict outcomes based on public reaction to media reports about court decisions. As we have explained, the Masterpiece Study depends on a link between the Masterpiece decision and the public reaction. The link is the news media.[164] According to Professor Barak-Corren’s recommendations, courts must consider potential public reaction when deciding cases involving a potential religious exemption. Then, courts should fashion their opinions in a way to avoid potential misreporting by the media. Professor Barak-Corren states that “even an intentionally narrow and case-specific exemption can have a substantial impact on an industry and its customers.”[165]

How the public would react to any given decision is a matter of speculation. The Masterpiece Study did not measure how audiences absorbed media reports or whether the public understood Masterpiece as preventing government hostility towards religion or granting a religious exemption. If the media broadly mischaracterized the decision, then the media, and not the courts, would be the cause of any Masterpiece Effect. But, of course, neither the courts nor government more broadly can predict or control how the media might report on particular cases.[166]

Consequentialism itself is not a sound vessel irrespective of its conflicts with constitutional jurisprudence. Among the problems with consequentialism is that “the effects of any legal rule can be described in an infinite number of ways.”[167] And even consequentialists acknowledge consequentialism is out of place in matters of free expression due to state incapacity to assess actual harm in matters of speech.[168] That’s for good reason. A consequentialist approach would lead to a balancing between core First Amendment rights and a speculative prediction of how the consequences of a decision exempting those rights might affect other members of the public. But the very point of the First Amendment—as well as the Bill or Rights generally—is to place these rights “beyond the reach of majorities and officials and to establish them as legal principles to be applied by the courts.”[169] Put differently, a consequentialist approach turns on what courts think to be good policy, but “good policy” changes over time. Professor Barak-Corren essentially advocates for a “value judgment” by supposing that governments’ general interest in preventing discrimination outweighs individual religious exemptions.[170] By contrast, the First Amendment assumes that protecting certain freedoms is good policy regardless of particular outcomes in particular cases.

In fact, looking at particular cases or potential outcomes of Professor Barak-Corren’s consequentialist theory of judicial review highlights one of its main flaws: it implicitly encourages governments to treat religious business owners with hostility, coerce or stifle religious speech, and to otherwise discriminate against religion. This result cannot be reconciled with the First Amendment. As one legal scholar noted in the context of speech, “bad consequences that come about because the speech persuades people to do certain things cannot justify suppression.”[171] Professor Barak-Corren implicitly accepts that courts should allow the government to show hostility to religion so that the public does not misunderstand religious hostility cases as granting religious exemptions and therefore feel emboldened to deny requests for services related to same-sex weddings.

Take Masterpiece. Under Professor Barak-Corren’s theory, the Supreme Court should have allowed Colorado to treat Mr. Phillips and his bakery with religious hostility to avoid “a negative effect on vendor receptiveness to same-sex ceremonies[.]”[172] This conclusion implies courts should allow governments not just to disregard but to disparage religious beliefs. That conflicts with bedrock free exercise protections.[173]

And it has serious practical consequences. For example, Klein v. Oregon Bureau of Labor and Industries set aside a $135,000 fine imposed by the Oregon State Bureau of Labor and Industries (“BOLI”) on bakers who refused to prepare a custom wedding cake for a same-sex marriage based on their religious belief about marriage.[174] Guided by Masterpiece, the court ruled that BOLI’s “handling of the damages portion of the case does not reflect the neutrality toward religion required by the Free Exercise Clause.”[175] But under Professor Barak-Corren’s consequentialist approach, this ruling—tinged with bias and hostility and resulting in a significant fine—would withstand review for fear it would open the door to dangerous, unknown consequences. Such an approach may also present the greatest risk to minority religions who most frequently request religious exemptions, at least under RFRA.[176]

Professor Barak-Corren says the study calls for “a clear and bright line decision that provides specific and unambiguous behavioral instructions.”[177] But she acknowledges the evanescence of any effects, noting that she could not accurately measure a Masterpiece Effect after time passage and occurrence of ongoing societal effects.[178] In stark contrast, the First Amendment stands the test of time.

D. The Study Does Not Account for Exemptions for Freedom of Expression

In contrast to diffuse experimentation brought about by consequentialism, the Constitution categorically protects sincerely held religious beliefs. Often this freedom dovetails with the First Amendment’s free speech guarantee.[179] Of course, the Free Speech Clause applies beyond religiously motivated speech—it applies to speech regardless of the motivation.[180] And because many types of public accommodations create expression as their good or communicate ideas through their service, these laws and the First Amendment have often collided.[181] Free speech—as well as free exercise—often plays a role in the services public accommodations do and do not provide.

As previously noted, Professor Barak-Corren chose to audit photographers, bakers, and florists, and the choice of these professionals “was influenced by recent cases in which businesses refused service to same-sex couples.”[182] She intends for her study to inform debates and litigation over conflicts between same-sex couples and wedding vendors who object to their unions. Many of these creative professionals claim they are engaged in protected expression as well as religious adherence.[183] How would Professor Barak-Corren classify an artist’s objection to creating a requested cake, floral arrangement, or photograph for a same-sex couple when the objection is based on the artist’s artistic judgment? Is that discrimination or artistic license? Professor Barak-Corren does not answer these questions or address how the Masterpiece Study or consequentialist jurisprudence would handle objections by creative professionals who contend that public-accommodations laws involve forced or restricted speech. But these artistic decisions should be considered valid artistic or aesthetic judgments rather than illegal discrimination.

Courts have held or opined in dicta that wedding photography,[184] wedding cake design,[185] and wedding floral arranging[186] are or can be expressive and thus merit First Amendment protection. Some have said the same about wedding-related activities not addressed by Professor Barak-Corren, such as web site design and calligraphy.[187] Beyond weddings, many other organizations and businesses have successfully asserted a First Amendment defense to anti-discrimination laws when application of those laws interfered with their desired expression. These include television casting,[188] Amazon’s charitable-giving program,[189] search algorithms,[190] a softball league designed to advance “the idea of athletic competition and good physical health in support of the gay lifestyle,”[191] a beauty pageant,[192] parades,[193] Boy Scouts,[194] newspapers,[195] public speakers,[196] and custom t-shirt printers.[197] Free speech doctrine recognizes that creative professionals have a right to express their own views and not be forced by the government to express views they disagree with.

These precedents explain that photographers, bakers, florists, and other businesses and organizations engaged in expression have the constitutional freedom to reject an engagement because it does not fit their personal values or artistic and stylistic approach. These artistic choices differ in kind from the invidious discrimination that public-accommodations laws are meant to prevent. But the Masterpiece Study did not delineate between the reasons for non-responses of photographers, bakers, and florists Specifically, the Masterpiece Study never evaluates whether any of the post-Masterpiece non-responses resulted from artistic judgments as opposed to sexual orientation discrimination. These omissions undermine the Masterpiece Study’s applicability to claims involving speech-based objections to creating an expressive product for a same-sex wedding.

IV. Conclusion

The Masterpiece Study suffers both in its methodology and conclusions. The study detected an anomalous pre-Masterpiece discrimination against opposite-sex couples. This caused Professor Barak-Corren to inconsistently label non-responses between Waves 1 and 2 and Waves 3 and 4, created a regression-to-the-mean issue that the study never addresses, and contributed to significant inconsistencies in inquiries between pre-and-post Masterpiece waves. This then led to the spurious causal explanation that underpins the Masterpiece Effect. The study also counted non-responses as discrimination without giving reasons to rule out other explanations. And when we evaluated explicit denials, we attained different conclusions. In our analysis, we examined previously “gay antagonist” creative professionals. This population explicitly declined same-sex wedding inquiries less often after Masterpiece. Statistically sound studies do not vary in their results based on how the variables contained in the data are arranged. Entirely different findings based on explicit rejections make clear that the study’s conclusions are not sound. The study maintained no true control group. And the study never measured creative professionals’ exposure to Masterpiece to establish a link between knowledge of the opinion and a change in behavior.

These shortcomings undermine one of the potential uses of the study: evaluating the government interest prong of strict scrutiny analysis. But the Masterpiece Study establishes no factual basis to conclude that granting a religious exemption limits access to goods and services or causes widespread dignitary harm. Without proper methodology or reliable conclusions, the study cannot provide an evidentiary basis to deny religious exemptions. The study’s purported legal value also rests on a questionable doctrine of consequentialism. “The very purpose of a Bill of Rights was to withdraw certain subjects from the vicissitudes of political controversy, to place them beyond the reach of majorities and officials.”[198] A consequentialist approach discards these important protections in order to guard against diffuse and unproven discrimination. It also opens the door to forced artistic expression and suppression of speech to combat discrimination that has not been proven to exist.

Finally, Professor Barak-Corren fails to measure or even consider whether her recommendations would increase anti-religious animus.[199] Under her approach, creative professionals would face the choice between being forced out of business or a broken conscience. When judicial decisions are built on the quicksand of inconclusive social science, unintended and unanticipated effects are likely to follow. Better that courts perform their hard tasks with the sound tools of constitutional interpretation and legal analysis already at their disposal.

---

Note from the Editor: The Federalist Society takes no positions on particular legal and public policy matters. Any expressions of opinion are those of the author. We welcome responses to the views presented here. To join the debate, please email us at [email protected].