Deep Dive Episode 151 – Public Input in Agency Rulemaking

[Music and narration]

 

Introduction:  Welcome to the Regulatory Transparency Project’s Fourth Branch podcast series. All expressions of opinion are those of the speaker. 

Colton Graub:  Good afternoon and welcome to The Federalist Society’s Fourth Branch Podcast for the Regulatory Transparency Project. My name is Colton Graub. I am the Deputy Director of RTP. As always, please note that all expressions of opinion are those of the guest speakers on today’s call.

 

If you would like to learn more about each of our speakers and their work, you can visit www.regproject.org where we have their full bios. After opening remarks and discussion between our panelists, we will go to audience Q&A, so please be thinking of the questions you’d like to ask our speakers. 

 

This afternoon, we’re pleased to host a conversation exploring the role of public comments in agency rulemaking and the implications of mass commenting campaigns. Our moderator for today’s discussion is Susan Dudley, Director of the GW Regulatory Study Center and Distinguished Professor of Practice at the George Washington University’s Trachtenberg School of Public Policy & Public Administration. Susan, the floor is yours.

 

Prof. Susan Dudley:  Thank you, Colton. I’m looking forward to this conversation. Let me first introduce our speakers today. Steve Balla is an Associate Professor of Political Science, Public Policy and Public Administration, and International Affairs at the George Washington University and also a Co-director of the Regulatory Studies Center. He is a leading expert in government consultation and public participation and policymaking in the U.S. and also in China. 

 

We’ll also hear from Reeve Bull, who is the Research Director of the Administrative Conference of the United States, although he’s speaking here in his personal capacity today. Reeve is widely published on issues in administrative law and regulation, including citizen participation in regulation. 

 

And also from Michael Livermore, who is the Edward F. Howrey Professor of Law at the University of Virginia where he teaches environmental law, administrative law, regulatory law and policy. Especially relevant for today’s discussion is his research on computationally assisted regulatory participation. 

 

As Colton mentioned, the event page has links to more detailed bios of the panelists and also relevant articles if you’re interested in digging deeper after this conversation. And who wouldn’t be? This is an interesting topic. 

 

The opportunity for public participation in regulation has deep roots in the United States, and yet, it isn’t well understood, even by people who engage in rulemaking. The Administrative Procedure Act of 1946 required agencies to provide public notice before issuing regulations and to take public comment into account in developing final rules. It’s a practice that distinguishes the U.S. from many other developed countries, which focus more on engaging particular stakeholders in regulatory development but don’t usually solicit the same broad feedback on complete drafts of regulatory proposals. 

 

Reeve, can you start us off by providing more information on the history and purpose of public comment?

 

Reeve Bull:  Absolutely. I would love to do so. Thanks, Susan. I’ll start with the obligatory disclaimer that, as you noted, Susan, I’m speaking in my personal capacity and not in my capacity as an ACUS employee, an employee of the Administrative Conference, and my remarks are exclusively my own. 

 

You asked first about histories. As you had indicated, notice and comment was part of original Administrative Procedure Act enacted in 1946. It basically divided agency procedures up into adjudication and rulemaking. And rulemaking itself was actually something of an innovation. 

 

Up until the adoption of the Administrative Procedure Act, most agencies would tend to make policy through an adjudicative process. They would decide individual cases, and then those cases would serve as precedent in subsequent cases, whereas rulemaking was intended to be more like legislation, essentially. You would adopt a rule that would apply across the board prospectively to govern conduct of regulated parties. And because rulemaking was innovative, and particularly because public comment was innovative, it’s something that agencies had not previously done. 

 

Now, potentially two major purposes that the framers of the APA intended the process to serve. One was very clearly informational. And it’s actually very innovative in that respect. So now, you hear about crowdsourcing and the wisdom of crowds gathering dispersed knowledge, that was really one of the major purposes of notice and comment. The idea was that regulated parties, beneficiaries of regulation, people throughout society have information, valuable information, that the agency might not otherwise have access to. The idea was by opening up rules to public comment, the agency could get that information, and then that would help it to devise a more sophisticated rule, as opposed to what the agency was just developing the information on its own. 

 

So that was very clearly one major purpose of notice and comment and absolutely remains probably the major purpose of notice and comment. But another possible purpose, and one that I think has proven a little bit more controversial, or at least sort of opens it to question, is a democratic purpose. 

 

So obviously, one possible purpose of notice and comment is that it’s a chance to make your views known to the agency. It parallels what you see in the right to petition the government in the First Amendment or your ability to write your Congressperson, for instance, to weigh in on legislation. And the idea was that there’s some value, especially where unlike with legislation in Congress where the policymakers are not elected, they’re appointed officials, there’s potentially some democratic value in getting the views of members of the public as part of the rulemaking process. 

 

What’s interesting is to the extent that that was a purpose, there’s really a major divergence, or at least there’s become a divergence between what agencies probably perceive is the purpose, which really puts the emphasis on the informational aspect that this is valuable to get information that the agency can use and how the public has come to perceive the process. And the public -- and I think in some way this underscores how deeply embedded democratic norms are in our society, which I think is generally a good thing. 

 

But it becomes pretty clear, especially in light of more recent rules that have garnered just a huge, huge public response that people tend to think of this very much in democratic terms. Indeed, they almost tend to think of it as a vote, essentially, that by submitting a comment, they’re telling the agency how they want a particular rule to come out, either I support it, or I oppose it. 

 

The example par excellence, I think, is the recent net neutrality repeal, and then prior to that, the original net neutrality rule. The original rule, I think, garnered several million comments, and then the more recent repeal of net neutrality garnered, I believe it was somewhere around 24 to 26 million comments, by far the largest that any rule has ever received. And it received a lot of popular press. Actually, [John] Oliver did a segment on his show where he encouraged people to weigh in on the rule. And the idea is, obviously, that people tend to think that their views are -- will affect, will influence the agency’s process. 

 

So to some right, I think basically there’s clearly notice and comment as intended for informational purposes. It may also have some democratic elements. But there seems to be some sort of divergence between how the public may view it, and I think they tend to see it more as democratic, and how agencies may perceive the value of the comments received. And there seems to be more of an emphasis on the informational aspects. And I think part of our discussion today will be to what extent in particular does the democratic element -- should it influence the process? To what extent should policy views that people express have an effect on the process?

 

So let me stop there. Hopefully, that gave some helpful background. 

 

Prof. Susan Dudley:  It was very helpful. Thank you, Reeve. Michael, clearly a lot has changed since 1946, not only in the perceptions that people have but also how modes of public comment have changed. I’ll probably show my age here, but when I was first involved in regulation — and no, it’s not as far back as 1946 — but I would have to go to the bowels of the government buildings with a pocket full of nickels so that I could Xerox relevant pages from the regulatory docket. Could you talk to us about what opportunities new challenges bring for engaging the public?

 

Prof. Michael Livermore:  Sure. Thank you very much. Yeah, it wasn’t that long ago that we had analog, essentially, process for facilitating interaction between agencies and the public. It would have taken quite a bit of foresight on the part of the folks who drafted the Administrative Procedure Act to anticipate developments like the internet or the massive computational power that we all carry around in our pockets, and our ability to access information so easily, and to engage in a multidirectional conversation on social media and other channels. So a lot is different from when the APA was originally adopted. 

 

With respect to rulemakings, one simple way that we could think about these technological developments is in terms of reducing the costs of participation. As you said, it used to be the case that if you wanted to engage in any kind of way in the public comment process as an outsider, you went to some library and you needed to make copies and get ahold of the Federal Register. And obviously typing up your comments isn’t that big of a deal, but it still costs something to mail them in. 

 

I think the other really big part of the issue is getting information about a regulatory proposal. Agencies put out their own documents, of course, of the proposal itself, and it had some explanation in there. But finding out the views of the relevant interest groups, finding out the views of academics or other folks at think tanks and the like who study these issues and have something to say, that would have been very, very difficult prior to the advent of the internet. You’d literally have to start making phone calls or rely on the relatively short summaries that you’re going to find in popular newspapers and the like. 

 

So in any case, now things have changed, and it’s very, very easy to get access certainly to the documents that the agencies put out, not only the rulemakings themselves but the dockets, technical reports, and the like that the agencies have compiled in support of a rulemaking. The views of interested parties and observers are accessible at the click of a keyboard stoke. 

 

And there’s lateral conversations as well. Reeve talked a little bit about mass comment campaigns, but the idea of commenters interacting with each other and coordinating is not something that was particularly cheap or easy 10 or 20 years ago. But now, groups can cultivate email lists, there’s obviously social media, there’s these other channels that allow for this spread of  information that a rulemaking is even happening and enough information to keep people’s interest. So this has changed, and this is a big technological change that has reduced the cost of participating. 

 

And what we’ve seen is a big explosion in the number of comments that are received by the agency and a big change in the types of folks who are engaged in the public comment process. So whereas in the earlier days, the pre-internet days, most rulemakings, or almost all rulemakings, really, perhaps all rulemakings, the interaction was mostly in terms of beltway insiders, interest groups, folks that were paying attorneys to follow the process and could have established procedures and so on to get access to the relevant materials. So it was a really insider process. What has happened through this technology is that the process is now not just for insiders. It’s now expanded out to become a form of popular political participation. 

 

And as Reeve mentioned, we don’t know what to do about that. The technology, not surprisingly, has gotten out in front of both the government agencies’ ability to figure out what to do, but also just normatively. What do we want? What do we want the administrative process to look like now that we have these technologies that reduce the cost of participation? That’s now, due to technology, no longer a constraint. Now the constraint is really our own vision about what we want the administrative process to look like. 

 

So that’s a little 75 years of technological history rulemaking in 5 minutes or less. And I’ll turn it back to you.

 

Prof. Susan Dudley:  So next up, I’m going to ask Steve a question because, Steve, you’ve studied changes that we’ve seen in who participates and how. What have you found about who participates and whether agencies pay attention to comments and what types of comments they make?

 

Prof. Steven Balla:  Sure. Thank you, Susan. And I want to address both of those questions briefly, who participates and does it matter, and maybe say a few things about how the answers to those questions have changed over time. 

 

So first off, who participates? Historically, most proposed rules have received a small number of comments, a few dozen, no more than a couple hundred. What’s interesting is that these comments, though, have historically tended to be quite lengthy, substantive in content. They tend to come from known stakeholders who interact with agencies on a regular basis, telecommunications companies, environmental advocacy groups. 

 

Every now and then, though, the comment process has blown up — and I’m talking historically here — and agencies received thousands and, more recently, hundreds of thousands and millions of comments. And of course, this latter set of regulations which is much smaller than the overall set of regulations that we’re talking about here, but these are the ones that many of us have heard about. We’ve already talked about net neutrality. We can think about the Clean Power Plan, Waters of the United States. These are rules that address the leading issues of our time, climate change, the revolution in information and communication technology and so forth. 

 

And in these instances, the regular players that Michael was just talking about, the inside the beltway players, are joined by large numbers of groups and individuals who most of the time ignore the regulatory process. And in fact, these new players are often drafted into the comment process by the regulars themselves. Michael just talked about mass comment campaigns. So these are coordinated by we might think of the regulars in that they not only submit their own substantive comments, but they coordinate the submissions of large numbers of comments that are duplicate or near identical in content. They tend to be rather short in length. They state not much more than a position in favor of or in opposition to a proposed rule. 

 

And all of this activity, I think, raises the second question of, well, do public comments matter, whether they’re of the substantive variety or the mass campaign variety? And on this score, I would argue that there are certainly instances in which comments demonstrably have affected regulatory outcomes. But I think it’s fair to say that on this score, the data analytics, the empirical results are a bit mixed. There’s some research that we can point to that highlights the influence of comments. There’s other research that downplays the role of public input. 

 

And as an aside, I think it’s important to acknowledge in all of this that a comment might matter for reasons other than steering the regulation, the content of the regulation. So what might we mean by that? Well, comments, especially when they’re submitted en masse, they can attract political attention for or against a proposed rule; that is, they can provide cover for political allies in the administration, or they can place pressure on officials in opposing administrations. So in addition to talking about the substance of the rule, they can attract political attention. 

 

And comments can also serve organizational maintenance purposes; that is, they can be a vehicle for groups to demonstrate to their members that they’re actively involved in pursuing their shared mission, in pursuing policy objectives of the group, and therefore that they deserve ongoing financial and grassroots support. 

 

And then, finally, let me briefly address has any of this changed over time? Has the movement to e-rulemaking changed what we know about participation patterns and the influence, or lack thereof, of comments? I think with one exception — and I want to maybe fine tune what Michael said a bit, offer a slightly difference perspective — I think the evidence suggests that participation patterns have remained remarkably stable through the changes of the past several decades. That is, most proposed rules still generate small numbers of comments from repeat play stakeholders. 

 

Furthermore, mass commenting occurred before the internet. There were postcard campaigns in the old days that are not that long ago. So they’re nothing new in that regard. I think the one thing that has changed is that the scope of mass comment campaigns today are orders of magnitude larger than their postcard predecessor. In the old days, I think tens of thousands of comments would have been a lot for an agency to receive. But these days, the high water mark tends to be measured in the millions. 

 

But having said that, do these massive submissions that we’ve seen recently, do they leave any mark on rulemaking other than causing the occasional server crash, providing fodder for John Oliver? Well, the evidence suggests that mass comment campaigns are less connected that substantive comments to the changes that occur between proposed and final rules.

 

But I think, yes, to get back to Reeve’s initial dichotomy, I think rulemaking is both a legal and a political exercise, a technocratic and democratic exercise. But it appears that by and large, the law, the dictate to consider, quote, “relevant matter” generally trumps politics when it comes to agency rulemaking. 

 

Thanks, Susan. I’ll stop there.

 

Prof. Susan Dudley:  Thanks, Steve. That was a “trumps” with a lower case t, I think. 

 

Prof. Steven Balla:  That’s correct. 

 

Prof. Susan Dudley:  All right. Well, that actually leads me to another question to revisit. This is all really interesting. So how do agencies deal with these mass comments, and not just -- we haven’t talked much about fake comments and bot-generated comments. Could you talk a little bit more about that and what agencies seem to be doing, and whether you think they’re a problem?

 

Reeve Bull:  Absolutely, happy to do so. And I’ll break that up into two separate pieces. So first, just technically, how do agencies deal with these different types of comments, and then, secondly, are they a problem? 

 

On the first point, the technical point, each of them is a little different in terms of how the agency deals with them. So first, with respect to mass comment campaigns that both Steve and Mike had alluded to, most agencies have software that’s usually called de-duping software, de-duplicating software, where they can go in and if an organization has encouraged its members to submit comments to the agencies and they provide a form, then the software can go in and say, okay, these comments are either 100 percent identical, or maybe they’re 95 percent identical. 

 

Sometimes the organizations encourage their members to change something slightly. But the software can tell you this is how close it is, and then the agencies can use that to help sort things. So they don’t necessarily have to read every single one of these. If they get 100,000 comments but they all basically say the same thing, then the agencies can usually sort through those pretty quickly. 

 

You also mentioned, Susan, computer-generated comments or bot comments where it’s not a human being submitting the comments. Somebody writes an algorithm that actually submits a comment. There are a few technical things that agencies can do with this. Like any other organization, there’s the reCAPTCHA process where you have to type in some string of numbers and letters. Agencies I don't think are extensively deploying this, but it’s something that they’re giving some thought to. 

 

But we are also getting the sense -- so the Administrative Conference did a program about two years ago on mass computer-generated fraudulent comments. And most agencies’ perspective at the time was that with respect to the computer-generated comments, they’re easy enough to recognize. Usually, the syntax is very butchered. It’s very clearly something generated by a computer. 

 

But what’s interesting is the algorithms have become more sophisticated in recent years. There was a study where comments were submitted using an algorithm. They were actually indistinguishable from human-generated comments. So it’s traditionally not been as huge of a problem, but it’s becoming more of a problem as the technology gets more sophisticated. 

 

And then, finally, fake comments where somebody submits the comment under a name other than his or her actual name. The sense, generally, has been that that’s not very common. And usually, the way the agency approaches it if somebody writes the agency and says, “This comment is up there, and I didn’t submit it,” then the agency will then take the comment down. So that’s technically, basically how the agencies are currently dealing with these. 

 

But you also asked the question, and I think it’s the key question, are they a problem? And what’s interesting is I think it sort of goes back to what the purpose is. So we talked about democratic versus technocratic, and I think arguably it’s a problem with respect to either of those, but probably more of a problem if you think of the process in democratic terms. 

 

So to elaborate on that, obviously, the process isn’t a vote. I think agencies are very consistent on that. But the agency would probably still like to have some sense of whether the comments reflect actual people’s views. And with respect to computer-generated and fake comments, they don’t. A computer-generated comment, other than the person who wrote the algorithm, is not an actual person. And a fake comment, a person wrote it, or in some cases, a bot wrote it. But again, it doesn’t correspond to the person who is purporting to submit the comment. So those sorts of comments, under a democratic theory, really have no value. They don’t actually tell you what real people think. 

 

With respect to mass comments, arguably, they do, but there’s a problem there as well in the sense that, first, they’re not necessarily representative, or they’re not at all representative. If you have an organization that encourages all of its members to submit a comment, you might get 60,000 comments supporting the rule or opposing the rule, which would overwhelm comments, potentially, in the other direction. But that doesn’t really tell you what a demographically representative sample of the populace thinks. It just tells you what the people submitting -- what the organization encouraging the members to submit the comments thinks. 

 

And then, also, the views can often be somewhat extreme in one direction or the other. People tend to be motivated to submit a comment if they feel strongly one way or the other, whereas people who are more in the middle may be less motivated to submit a comment. So if the goal was to get a sense of what the populace writ large thinks, then each of these types of campaigns, mass comments, computer-generated, fraudulent comments, can potentially undermine that. 

 

With respect to the technocratic view, it’s perhaps less of an issue. If all you care about is the information in the comments, then a bot-generated or a fraudulent comment may be okay. As long as it includes useful information, the agency should still consider it. And with respect to mass comments, the agency can consider them for what they’re worth, but given that they’re identical or nearly identical, you’d really only consider the comment once. You wouldn’t take into account that it’s been submitted in multiple iterations. 

 

The only real risk on the technocratic side is if this becomes increasingly common, it may run the risk of crashing the process. There’s just too much information for the agency to consider. But otherwise, it’s not a problem in terms of the representativeness of the comments. So let me stop there. 

 

Prof. Susan Dudley:  Okay. Thank you, Reeve. So Michael, I’m curious what you think about this. We’ve heard from Steve that agencies generally take -- seem to focus more on -- if we look at Reeve’s dichotomy between the informational content of the comment versus the sentiment or expression of sentiment or democratic value of it, agencies seem to take the -- more seriously focus more on the technocratic or the information content in the comment. 

 

Do you think that’s appropriate? Do you agree with Reeve? And do you think there are ways that agencies could do a better job of also taking into account the sentiment that people are expressing in these mass comment campaigns?

 

Prof. Michael Livermore:  Great, yeah. Thank you. Yes, so I think Steve’s absolutely right. He’s done some empirical work on this, but I think anecdotally and in talking to folks at agencies, agency personnel are very much focused on the information content, and they’re particularly focused on the information content inasmuch as they view the arguments that are made as ones that might appear in subsequent litigation. 

 

And what that highlights is that agencies are responding to their incentives. And courts require agencies to engage in certain types of discourse with certain types of comments, and in particular are the ones that have the greatest substantive content, technical content. And courts really aren’t in the business of requiring agencies to engage in a meaningful way with these kind of mass democratic sentiment style of comments. 

 

To my mind, there’s two normative paths that we might want to go. I think we’re in that wonderful place right now inasmuch as the perception of some broader group of folks in our political community believe that the public comment process is one in which popular participation matters, in which their voice counts in some meaningful way, even when their voice is an expression of sentiment, is in a technical document that is 5,000 words long and has lots of scientific and engineering information in it. 

 

And that perception is facilitated both sometimes in the course of mass comment campaigns or in the popular culture or in the press or even sometimes by agencies when they say things like, “Your voice matters.” They say that kind of thing, “Please participate,” and so it’s not a good situation if we’re misleading the public about its role in this process. 

 

And so one fork in the path to get us back onto something like a more normatively desirable situation would be to figure out how to engage with folks in a popular way that facilitates and makes their participation meaningful. Another fork in the path would just to be much more explicit about the kinds of comments that actually matter and make it very clear that unless you’re prepared to sit down and hire an attorney or you’re an expert in some meaningful way or you’re going to become an expert, it’s extremely unlikely that your comment is going to have any influence. In fact, it’s kind of explicitly not going to have any influence. And we could do that. Agencies have been hesitant to act for some pretty obvious reasons. So that’s one path. The other path is figuring out how to engage to the public more broadly. 

 

There’s two issues that Reeve raised that just wanted to briefly mention as initial issues. So one is the fake comment, bot comment issue. I see that more as a technical problem with potentially a technical solution. Obviously, we figure out how to verify people’s identities in other contexts. So that’s something that we could in theory do. It might change the public comment process, make anonymous comments more difficult. But in any case, I feel like that’s something we could probably think carefully about and address. 

 

The other issue that’s perhaps more tricky is the one of non-representativeness. And again, this probably takes us back to the underlying thing that we’ve been turning over, which is what is the purpose of the public comment process? And so inasmuch as the democratic side, the goal is to solicit an understanding of very, very broadly what people think about a rulemaking, the same way that you might use an opinion poll to understand and randomly select a group of people for polling. The public comment process is never going to look like that. We’re never going to get to the numbers necessary, and obviously people will select into the group of commenters, and so that’s not going to work. So inasmuch as the idea there is to get this sense of everyone’s views, or the median voter, or something like that, we’re going to -- this isn’t going to work. 

 

On the other hand, if the goal is in part to give people who are particularly committed to an issue to have an opportunity to have a voice in how that plays out, then I think the public comment has more potential, and we don’t have to worry as much about the non-representativeness issue because that’s not the goal. And there is an analogy to voting because people select into the community of voters, and that’s not representative at all. But we’ve come to terms with that as a society, and we don’t elect our leaders through a random sample of people that might be more representative. 

 

And I think the reason for that is that we understand that people actually selecting and making a choice about participating says something about their engagement, and we want to create pathways for regular folks to engage and participate in political decision making and not just have policies reflect their views in some very abstract way. So with that, I’ll turn it back to you, Susan.

 

Prof. Susan Dudley:  Thank you. Thanks, Michael. I think that’s very interesting. And I want to come back to that a little bit, the question about how to make it more representative, but let me first ask Steve, given all the research that you’ve done, what do you think could improve the value of information that agencies get from the public on rulemaking?

 

Prof. Steven Balla:  Thanks, Susan. Well, I think that one reason comments may most of the time not make much of a difference in regulatory outcomes, I think part of the problem is it has to do with the point in the process during which public comment periods happen. So if we think about by the time agencies publish proposed rules in the Federal Register, they’ve often been working for years on the issues at hand. And so it’s therefore quite understandable that agencies might be reluctant to make wholesale changes to their proposed rules at that point. 

 

And so I guess the question then becomes how might more meaningful participation be folded into the regulatory process, assuming as I do, that information of potential value to agency officials is indeed held by outside parties. And I think one marginal change, and I want to emphasize that it’s just that, and it’s not a new one, it’s to consider expanding the use of advanced notices of proposed rulemaking. And so advanced notices are documents published in the Federal Register by agencies at a much earlier stage in the life cycle of a rule than an ordinary NPRM before major decisions have been made, before the contours of proposed rules have hardened. And I think that this kind of earlier participation opens up the possibility of agencies being more open to the perspectives of participants. 

 

To address some of the larger, normative issues that are in the conversation, I don’t have any illusions that advanced notices would democratize rulemaking or anything like that. In fact, I’m not sure that that’s the main deficit in the regulatory process. If we think about the notice and comment process, it’s here to stay. We’ve tried to kill it in a number of ways over time. There was negotiated rulemaking. That’s obviously still with us, but there was a moment there in the 1990s, and of course the turn to e-rulemaking. But I think the notice and comment process is here to stay. And what that means then is the legal mandate to consider, quote, “relevant matter” is also here to stay. 

 

And so with that view, what’s really needed then from an agency point of view is better ways of ensuring that the best legal information, economic, scientific, technical information, and I would add to that anecdotal evidence anecdotal experiences, those stories from stakeholders who are going to be directly affected by proposed rules. We need ways of enhancing the ability for all of these various types of information to reach agency decision makers at a point in the process in which it can feasibly make a difference. So I’ll stop there. Thanks.

 

Prof. Susan Dudley:  Okay, Steve. Thanks. That’s great. Colton, why don’t we open it up for questions? So I’ll let you do that, and then I might pose a few more while we let people queue up to ask questions. 

 

Colton Graub:  Sounds great, Susan. Let’s now go to audience questions. We will answer all questions in the order in which they are received. Susan, back to you.

 

Prof. Susan Dudley:  Okay. Well, since I’ll pose mine first, I’ll be first in line to ask questions here. Let me ask all of you, starting with Reeve, what do you think about ways to engage? How can we engage better with more than stakeholders, get at Michael’s concern about the non-representativeness of those who comment, but also Steve’s observation that agencies need relevant information? Do you have some thoughts on that? So I’ll start with Reeve, but I’d like to hear from all three of you on that.

 

Reeve Bull:  Yeah, I do. I think it’s an excellent question, Susan. And I think it’s the key imbedded issue with each of our presentations is you have these two divergent purposes, potentially, and I think, potentially, there are ways to improve on both fronts. So with respect to getting more relevant information, I think there are a handful of things that are worth considering. 

 

I absolutely agree with Steve that ANPRMs are something that are worth taking a very careful look at. Particularly, I’ve done a little bit of comparative work with the European Union, and what the European Union doesn’t do so well is they don’t really have an equivalent of notice and comment, which I think is problematic that they’re moving more in that direction. But I think what they do do well is they get very early input in the process. They reach out to specific stakeholders. 

 

There may be some problems with how they structure it and might kind of promote insiderism, but I think the virtue of an ANPRM is you’re getting broad input, you’re opening up to the world, and you’re allowing people to weigh in before the agency has actually decided on a particular course of action because at the point that it issues an ANPRM, it’s usually relatively settled in terms of what it wants to do. And it’s seeing the comments more in the face of potential litigation as opposed to using the input in a way to try and structure its policy to take into account dispersed information. So I think that’s definitely something worth looking at. 

 

I think it’s probably also worth looking at ways to engage groups that don’t traditionally participate. And I think, as Mike alluded to, technology is actually an ally here. You can use social media. You can use other mechanisms for trying to engage and try to get broader participation. Then with respect to engaging -- if we do care about the public’s views in terms of the underlying policies, I think that the social media outreach is very valuable on that front as well. I think it diversifies and increases the participation so that you’re not just hearing from the same groups over and over again. 

 

I think it’s also worth looking at -- and this is much more avant-garde. I’m not sure that agencies would necessarily find this useful in all -- certainly not all cases and probably not most cases either. But there are things that they could do like listening sessions or targeted stakeholder groups or even put together like a demographically representative group. I’ve written on a citizen jury idea, if you will, where you get input from groups or entities that don’t necessarily participate or even individual people who don’t necessarily always participate in the process. And that would give you a truer sense than what notice and comment as to how people feel about a particular issue. 

 

Now, of course, that one is probably not going to be applicable in the vast majority of cases, but it may be something worth considering to the extent that an agency really does want to get a sense of how the public writ large would react to a proposal rather than just hearing from the people who actually bothered to submit a comment. 

 

So let me stop there. But I think there are definitely ways to improve on both fronts, getting better information, and then also making the process more open and more participatory.

 

Prof. Susan Dudley:  Thanks, Reeve. Michael and Steve, do you have more thoughts on that?

 

Prof. Michael Livermore:  Sure. So I think the point that Reeve makes and that Steve has made that is really important to attend to here is that there are really very -- there’s a lot of diversity in the types of rules that are adopted in the history of agencies. The net neutrality and Clean Power Plan and equivalently high profile rules are very, very much the exception rather than the norm. So for many, many, many rules, it’s not clear that the process isn’t working well enough or that it makes sense to try to engage the public in any broad way when the issues are narrow bore and hard to understand and very technical. 

 

The opportunity really comes up in these higher profile rulemakings. And again, I think the question then becomes that we have to just confront head on is what’s the purpose here? So something like a citizen jury has a lot of advantages in the sense of being more representative. It could facilitate deliberation amongst the group or within the group. But it is a very small number of people who will be participating in a forum like that. 

 

And again, if the goal is to generate a broader social conversation about things like what we want the internet to look like or how do we want to address greenhouse gas emissions or what have you, then the citizen jury could be part of the equation, but it’s not going to be the whole story. 

 

And again, maybe there’s an opportunity for technology here. I don't know that I personally have the solutions. It may be the kind of thing that’s going to require some experimentation either on the part of agencies in the U.S. or looking globally or looking to states or localities to start to experiment with alternative participatory tools that are enabled by technology, but we haven’t yet figured out how to make them work.

 

Prof. Susan Dudley:  Yeah. Steve, any thoughts? I also want to agree with you on the value of advanced notices. I think that could be just hugely important for the regulatory process. But other ideas?

 

Prof. Steven Balla:  Yeah, I think it’s interesting because for a couple of decades now in this e-rulemaking period, we’ve had this notion that we want to engage new participants. And I should say, sidebar, yes, what Michael just said, we’re talking about a very, very small fraction of almost the highest salience -- the most salient rulemakings here. I think the vast majority of cases are really off the table here in terms of any kind of reforms that we might be considering. 

 

But in those rare cases, there’s this notion. How do we engage new participants? Well, then we successfully engage new participants, but all of these new participants came across the transom in the form of mass comment campaigns. And then we say, “Oh, wait a minute. We don’t like the results so much.” 

 

The agency doesn’t like the results so much because it might be fixed on its mission of identifying solutions to the legal administrative problems that it’s been tasked with. The public might not be satisfied with the outcome here with mass comment campaigns because they can point to instances in which, quote, “the majority lost.” And so I think it’s just an interesting situation we find ourselves in in that we wanted something to happen, and it’s happened, and we might not now in retrospect be that enthused with the result. 

 

One last thing to add to that is I do think that in our research on mass comment campaigns, we have found that not all are created alike. There really are the one sentence campaigns that say, “I like this rule,” “I hate this rule.” But there are other mass comment campaigns where individual respondents in campaigns are encouraged to provide their own personal stories as small farmers or as dry cleaners or what have you. 

 

And I’m not under any illusion that having an influx of those kinds of anecdotal accounts in dockets is necessarily going to change the course of many rulemakings, but I do think that potentially fits the definition of relevant matter and something that agencies will take more than a passing look at that will survive the parsing out of their de-duplication software. 

 

But it’s interesting when talking to the kinds of organizations who generate mass comment campaigns. They’re stuck in a hard place between the notion of, yes, we would like this kind of unique stakeholder information to be part of our campaigns, but on the other hand, they know that the number of clicks they’re going to generate is going to plummet if they ask their members to not only click here and, quote, “vote,” but to also say something specific and unique that’s relevant to their own circumstance and how they would be affected by a rule. 

 

And so organizations are really stuck between trying to push their numbers up, which really means lots of duplicate comments, and on the other hand trying to encourage their members to say something original, which is a hard sell. So I’ll stop there. Thank you.

 

Prof. Susan Dudley:  Yeah, thank you. Everything you guys say gives me more questions. But let me ask Colton first. Do we have people who’d like to ask questions other than me, Colton?

 

Colton Graub:  We do. We have two questions. We will go to the first question now. 

 

Devon Judge-Lord:  Hi. My name is Devon Judge-Lord, and I want to thank you for this conversation. It’s really fascinating. As Steve mentioned, mass comment campaigns are actually relatively rare. Most rules still receive relatively few comments, and their likely impact on the process is even more rare. Oftentimes, it probably doesn’t matter. 

 

My question is under what conditions, if any, we might expect to a democratic flavor of information rather than the technocratic information to actually matter? Steve used the words political cover. I thought that was interesting because that’s exactly the words that an agency official I interviewed used to describe the impact of mass comments on the FTC’s Do Not Call rule. So perhaps attracting the attention of political actors is one mechanism, but I’d be very interested to hear about other plausible ways in which this phenomenon might matter. Thank you.

 

Prof. Susan Dudley:  Steve, that might be -- well, I’m interested in hearing everybody’s response. Steve, do you want to start?

 

Prof. Steven Balla:  Sure. One of the classic examples in this arena goes back actually to the very, very beginning of e-rulemaking, the end of the Clinton administration, in fact. And the USDA, when it issued its proposed rule on organic labeling and that kind of thing, they received this outpouring of comments, high volume of comments from organic farmers to consumers, and so mass comment campaign variety participation. 

 

And then Dan Glickman, who was the administrator of the USDA at the time said, “Hey, we hear you loud and clear. If we can’t make farmers and consumers happy, we need to rethink our approach,” which is what they did. And so what’s interesting about that case is it’s an instance where just, I think, the sheer volume of opposition from a variety of stakeholders really had an impact on the agency. 

 

What’s also interesting about it is that here we are, 20 plus years later, and I’m still relying on that anecdote when thinking about the direct influence of mass participation, democratic participation on regulation. So that certainly is a possibility. But I think more often than not, these mass comment campaigns are really just oriented for audiences outside of the rulemaking process per se, so they’re oriented toward political supervisors, members of Congress. They’re oriented toward the mass media. They’re oriented toward, like I said earlier, the maintenance of the organization, so they’re non-rulemaking purposes altogether. 

 

And so if we expand the definition of mass comment campaigns mattering to mean attracting political attention, helping organizations do what they do, well, they definitely matter. And we’ll continue to see them occur for that reason. But I’m pretty skeptical about the conditions under which they’re going to have a direct impact on the content of regulations. I think it’s a knife’s edge situation where it’s likely to happen very occasionally at best.

 

Prof. Michael Livermore:  So I’ll just add something to this. I think it’s actually very hard to draw conclusions, like a kind of a causal conclusion there because the system evolves over time. And as the public comment has expanded, agency decision making has been in light of that reality. And so we actually don’t observe the counterfactual situation where everything else is the same in our society but there’s no mass commenting campaigns. And so it’s possible that the reality that mass comment campaigns exist and that there’s this, at least for certain high profile rulemakings, there’s a possibility of massive social opposition or something like that that would become manifest through the public commenting process that agencies’ decisions are conditioned in that way. 

 

So in any case, there’s a bit of a difficult causal problem to untangle. So we actually don’t know that it doesn’t matter. One way that we might think about this is to go back in time to think about rules that were adopted before there was this opportunity for large scale participation. And I’m thinking of some of the seatbelt warning systems rules that were adopted decades ago, and tons of people hated them, and Congress got involved, and it was a real mess. And it may be that if agencies would -- if there had been public comments at the time, mass public comments at the time or the potential for mass public comments, that the agency would have been less prone to move in that direction, less prone to move in a direction that would generate that kind of very substantial public backlash. 

 

So in any case, I do think it’s a little hard to fully suss out, but that would be my instinct is that the fact of the public commenting process being so generally open may make agencies more hesitant to move in directions that they feel are going to elicit a lot of widespread opposition. 

 

Reeve Bull:  I’ll just briefly weigh in as well, and I don’t really have -- I agree with everything that both Steve and Mike have said. And I think from my perspective, I think from a -- if the goal is to get a true sense of what the public actually believes or what their policy views are, then from my perspective, the notice and comment process is not an effective way of achieving that for the representativeness reason and the others we’ve discussed. 

 

However, I absolutely agree with Steve and Mike that if the goal is not necessarily to determine what the true underlying policy views are but rather to get a sense of how people might react to something, then I think it can be very valuable from that perspective. I think it gives you useful political information. And I think the example that Mike gave is an excellent one, that if you want to see how will the public react to this, will there be widespread opposition, if you get a huge number of comments indicating that, then I think that that’s something that you should take into account. 

 

So I think there are two different ways it might matter. I think if the goal is democratic, then I think there are more effective ways of determining public views. But if the goal is simply to see what the political reality is or what the likely reaction would be, then I think it can actually be very valuable. And I think the mass comment campaigns in particular may give you a real sense of how people feel on a particular issue, which can be of use to the agency. 

 

Prof. Susan Dudley:  Thanks, everyone. Colton, since we’re getting rather late, do we have other calls? Should we turn to other callers and get more questions?

 

Colton Graub:  We do. We have two callers who have questions, and we have one that was entered in the chat. Let’s go to the next caller.

 

Roberto Borgert:  Hi. My name is Roberto Borgert. And I wanted to pick up on a tension I noticed between the agencies’ desire to gather information and, as one commenter mentioned, the fact that an agency, by the time the notice and comment period has opened up, has already decided a policy to pursue. 

 

And so I’m just wondering, from the commenter’s perspective, does it make more sense to invest time and resources into developing factual arguments, either in favor or in opposition to a proposed policy, or I guess, what marginal utility comes from setting forth legal arguments, which presumably the agency lawyers have already considered? Thank you. 

 

Prof. Susan Dudley:  I would say the answer to that is yes, both. Steve, you looked at what agencies pay attention to. Do you have a different response?

 

Prof. Steven Balla:  Okay, sure. I’ll tell you what I tell my students. I teach a class on how to understand the rulemaking process, and so I think there are two distinctive goals that you might have. I assume a kind of sophisticated commentary that you have some resources and you’re going to put in some really substantive comments. 

 

So there’s the path of trying to convince the agency, and then there’s the path as teeing up litigation. Obviously, legal arguments are arguments that are at least framed in legal terms. They’re going to matter more from a litigation perspective. Those might be quite factual, so you might be teeing up an arbitrary and capricious challenge, in which case you’re going to be making very factually grounded arguments. 

 

But when you frame things that way, it is likely to go into the agency general counsel’s office, whereas if you think you have a shot at actually convincing the agency using -- just focusing on the facts and not necessarily accusing the agency of engaging in arbitrary and capricious decision making all the time and rather just presenting your argument, that may ultimately be more persuasive, and it may get your comments directed to other parts of the agency, the technical, the economic side, the scientific side, where it might have more actual influence. 

 

Prof. Susan Dudley:  I think that makes sense to me. Any other thoughts on that? Colton, would you like me to read the question in the Q&A?

 

Colton Graub:  Sure, absolutely.

 

Prof. Susan Dudley:  The question -- and I think it’s an interesting question. Did the rulemaking process change under Trump, and will the Biden-Harris administration change this? And I think she’s specifically asking will the influence of mass comment messaging be more important or less important, do you think, in the next administration?

 

Prof. Steven Balla:  One thing on that, Susan, is that when we looked at mass comment campaigning in the Obama administration, we found two reasons why they occurred. One, if we just take, say, the Waters of the United States rulemaking, environmental advocacy groups, say, like the Sierra Club, they argued that they wanted to submit mass comment campaigns in large numbers to provide Obama EPA administrators with the support that they might need when they go before a skeptical Congress. There was a Republican Senate, for example, at the time. And so we were hearing from environmental advocacy groups that, “Oh, we just wanted to submit as many as possible for those reasons.” 

 

And then when we talked to farmers and other industries that might be affected by the Waters of the United States rule, their argument was, “We don’t generally do mass comment campaigning, but we found ourselves getting killed in the press. And we needed to counteract that narrative with our own by mobilizing our own members.” And so we found it interesting in that there were two different motivations. 

 

And so I might suspect that in a Biden administration that you might interestingly see more industry oriented mass comment campaigns because of the change in the kinds of proposed rules and the political support for those proposed rules in the administration.

 

Prof. Susan Dudley:  So that’s interesting, Steve. So it gets back to your view of the mass comments as really targeting a different audience altogether, the political audience as opposed to the agency, or giving the agency cover with the political audience. 

 

Any other thoughts on that? I know we’re running out of time. Colton, tell us what we should do because I think we have one more question in the chat.

 

Colton Graub:  We do have one more question, and actually, the person who chatted in also asked to ask his question out loud, so I’m going to allow him to talk. And he can ask his question, and then we’ll briefly wrap up at the end.

 

Caller 3:  Hello, all. Good day. I’m a tribal court judge, but there have been a lot of agency practice, and I’ve actually changed, as we say, one of the agency rules out there. The hard part is you’ve got 566 federally recognized tribes, and while all occasionally do communicate to each other with the different bar programs such as the Native American Bar Association and the State Bar that’s analogous like with Oklahoma and the Oklahoma Indian Bar Association, by the time an agency puts a rule out, as has been said earlier, and people get notice of it, or if you’re lightly involved with the practice, you kind of miss the public comment. 

 

I just wanted to make sure because back in 2008, we ended up changing one of the rules with Department of Labor, which has had, as we say, the outcome that we wanted for all members of Navajo Bar, Muskogee Creek, and others. But the idea was it seems like the agency makes the rule and they kind of forget about the other, shall we say, regulatory, shall we say, capabilities of Native American communities and tribal governments where most of the time, we’ve been left out. 

 

I know they’re trying to circle back around now with FCC and trying to get areas that are literally without cellular service to have some FCC capability for internet and Wi-Fi, but we still have communities that you drive through that area and it says, “Welcome to the Reservation,” you look at your cell phone, you’ve got no service. So you’ve got no 911 capability. I think if the agencies are going to make rules and do stuff, I think they also need to take into perspective and reach out to the tribes and see what else can be done to make this more seamless and better inclusion, or if there’s a way that that can even be done.

 

Prof. Susan Dudley:  Yes. Thank you. I know that that’s something that there are executive orders that ask agencies to reach out to stakeholders, especially state, local, and tribal stakeholders earlier in the process. Do other people have thoughts on that? I think it is something that agencies could do a better job of.

 

Prof. Michael Livermore:  So maybe very, very quickly, I think that we could just say that this is -- there’s a lot of special issues with respect to tribal governance, but there’s also just the broader digital divide issues. And so access to the internet, access to telecommunications technology is not evenly distributed throughout our society. And so as we’ve mentioned several times, there’s a kind of a representativeness issue with respect to mass comment or broad public participation, and so that certainly is something that we want to be mindful of when thinking normatively about how we want to take these kind of things into consideration and what agencies can and should do to ensure that there are shared opportunities for participation.

 

Prof. Steven Balla:  And I just want to quickly point out that in a study of over 1,000 mass comment campaigns over a number of years, we didn’t uncover one that came from a tribal government or that represented Native American interests. So I just want to point that out there. 

 

Prof. Susan Dudley:  Thank you. And thanks to the participants for staying on a little long. Do we have time, Colton, for just quick final thoughts from each of our panelists before we close?

 

Colton Graub:  Absolutely.

 

Prof. Susan Dudley:  All right. Why don’t we just go in the order we started, so Reeve?

 

Reeve Bull:  Wonderful. Yeah, thanks Susan. So let me say first I’ve really enjoyed this discussion. I think it’s been very enlightening. And I guess I think one of the key themes is really that the notice and comment process goes way back to the very beginning with the APA when it was enacted back in the ‘40s. And it’s really proven remarkably durable since that time. Basically, the original notice and comment process that the APA created is still what we have, and I think for the most part, it still works very, very well. 

 

But as we’ve seen over the course of the discussion, technology has created new challenges, but it’s also created new opportunities. And I think in some ways, there are new problems that the agencies are going to have to confront, but I think the technology also gives them new resources to try to confront those problems. And I think from that perspective, agencies are really going to have to fundamentally look at what are the purposes we’re trying to serve with notice and comment, and how can we enhance both the participation but also the value of the information that we’re receiving? 

 

And I think there are a handful of possible approaches that we’ve discussed today that can really be valuable on that front. So I think that it’s an exciting time, and it’s a time where technology can really allow the agencies to utilize public input in a more effective way than they ever have.

 

Prof. Susan Dudley:  Thanks, Reeve. Michael?

 

Prof. Michael Livermore:  Yes, so I agree very much with what Reeve has to say in terms of opportunities. I think one question that I’ll be paying attention to in the next few years is just where are the sources of innovation here? Is it going to be agencies themselves? Is it going to be Congress? Is it the White House? Is it states? Is it looking internationally? I don’t think it’s going to be courts, absent some change in legislation. 

 

And so I think that the status quo is not perfect for various reasons that we’ve described. And so my big question is if there’s going to be change, where is the engine of change going to come from?

 

Prof. Susan Dudley:  Yeah, good question. Steve, close us out.

 

Prof. Steven Balla:  Sure. I think rulemaking, most of the time, works. I think the uncertainty we have here and the sources of innovation that Michael is asking us to think about, I think that all of that comes from uncertainty about what the purpose of administrative rulemaking is when the public pays attention. And I think the purpose ultimately has to be the same. The emphasis has to be on generating and sorting through relevant matter, but at the same time, allow open participation. 

 

And so when I think about from just a mass participation point of view, what kind of innovations might we expect from the organizations that generate mass comment campaigns? Is there a way to begin generating more sophisticated public participation on a mass scale? And by sophistication, I don’t mean legal and economic and scientific and technical and all of that, but a more sophisticated personal anecdotal participation, which I think is a thing. 

 

And so the driver of that, it strikes me, has to be the organizations that are generating the participation in the first place. And so is there a way to more effectively communicate or think about the motivations or incentives for why these campaigns occur in the first place? Are there ways to alter the incentive structure? I’m a little bit skeptical about that, given the non-regulatory purposes that these campaigns serve, but I do think that that’s one place to think about going to the organizations that generate mass comment campaigns when thinking about the future of participation and how it might be changed in a meaningful manner in those occasional cases of high salience rulemakings. Thank you. 

 

Prof. Susan Dudley:  Well, thank you very much. I thought it was a great panel. And thank you to the participants, too, for staying tuned in, and to the Regulatory Transparency Project. This was great. And with that, I think we will close.

 

[Music]

 

Conclusion:  On behalf of The Federalist Society's Regulatory Transparency Project, thanks for tuning in to the Fourth Branch podcast. To catch every new episode when it's released, you can subscribe on Apple Podcasts, Google Play, and Spreaker. For the latest from RTP, please visit our website at www.regproject.org.

 

[Music]

 

This has been a FedSoc audio production.