Same-sex marriage is now the law of the land—and that’s a good thing. While Justice Kennedy’s majority opinion made a hash of the reasoning, Obergefell v. Hodges was a rather simple case: states need not be in the business of licensing marriage, but if they are, they can’t draw a line the way they did. Nobody’s hurt by a big gay wedding.

Unfortunately, some people are now using that very basic lesson in equality under the law to hurt people who disagree with the Supreme Court. The examples are well-known and will alas grow as public opinion shifts further in favor of gay marriage: the New Mexico photographer, the Washington florist, the New York farm owners. Each of these businesses was fined for declining to provide their services for a same-sex ceremony. Adding insult to injury, the Oregon bakers were both assessed $135,000 for causing “emotional damages” and ordered not to speak about their case.

But why? There are more than 100 wedding photographers in the Albuquerque area, and plenty of bakers willing to make whatever confections Adam and Steve want for their nuptials in Oregon. This isn’t the Jim Crow South, where the government enforced segregation laws, black travelers couldn’t find a place to stay for hundreds of miles, and violence was always in the offing.

Indeed, businesses now attract customers—even and especially straight customers—by advertising their gay-friendliness.

Moreover, there’s a difference between denying service to certain kinds of people and declining to participate in certain kinds of events. Should gay photographers be forced to work fundamentalist celebrations? Should blacks be forced to work KKK rallies? Should environmentalist bakers—plenty of those in Oregon!—be forced to make bear claws for job fairs in logging communities?

I don’t even know why you’d want to have as a wedding vendor someone who can’t in good faith (literally) support your union. If progressives respect diversity, shouldn’t they refrain from bending the will of fellow Americans toward prevailing pieties?

The response to my plea for tolerance is that this isn’t the case where “nobody is hurt”: those couples who are refused service certainly feel slighted, made to feel like second-class citizens. Perhaps the punishment just ought to fit the “crime” better, and this should be treated more like a traffic violation: write a ticket for $100 and be on your way.

Let’s think about that for a second. We know of societies where the cost of exercising one’s religious beliefs was a small tax—in ancient Islamic states where non-Muslims were excluded from political rights. Is that really what a nation founded on religious toleration has come to?

And this goes beyond gay weddings: through an ever-growing list of mandates and regulations, government compulsion squeezes out civil society and foments social clashes. Look at last week’s ruling against the Little Sisters of the Poor, where the appellate court decided that it knew better than the nuns what constituted a burden on religion.

The most basic principle of a free society is that the government can’t willy-nilly force people to do things that violate their beliefs. Some may argue that in these wedding cases there’s a conflict between religious freedom and gay rights—and marriage equality is more important—but that’s a “false choice,” as President Obama would say.

There’s no clash of individual rights in any circumstance other than when the government itself declines to consistently recognize and protect everyone’s rights. County clerks act on the state’s behalf and so must issue marriage licenses regardless of their personal beliefs, but bakers aren’t government agents and so should maintain freedom of conscience.

With certain things, such as national defense, basic infrastructure, and other “public goods,” we largely agree, at least inside reasonable margins. But we have vast disagreements about social programs, economic regulation, and so much else that government now dominates—forcing dissenters to plead for carve-outs of individual liberty.

Those who opposed same-sex marriage before the Supreme Court were thus rightly concerned that people would be forced to do what their deepest beliefs prohibit.

Justice Kennedy could’ve forestalled some of this mischief by making clear that his ruling protects not just the right to “advocate” and “teach” religion also but to “exercise” it. But he didn’t, so it’s left to the better angels of our pluralistic nature to respect views and lifestyles we may not like.