Use of Artificial Intelligence by State Bureaucracies Requires Strict Oversight

Burning candles on the AI cake

So July 23 this year was notable on two different counts: First, it was my birthday (send no gifts please, though I know you are dying to!), and, second, President Donald Trump signed an executive order on the use of artificial intelligence in the federal government.

OK, maybe the executive order was more notable than the birthday.

Anyway, the president signed an executive order “preventing woke AI in the federal government,” which conservatives, including the Manhattan Institute’s Christopher Rufo, immediately hailed.

Essentially, the executive order directs the administration not to purchase software from vendors that have embedded woke presumptions and biases into their base code. This is easier to do than you think, but, fortunately, it’s also pretty easy to detect. Specifically, the executive order directs agencies to procure “truthful” generative AI models that “prioritize historical accuracy, scientific inquiry, and objectivity, and shall acknowledge uncertainty where reliable information is incomplete or contradictory.”

The models must also be ideologically neutral and must “not manipulate responses in favor of ideological dogmas such as DEI. Developers shall not intentionally encode partisan or ideological judgments into an LLM’s [large language model] outputs unless those judgments are prompted by or otherwise readily accessible to the end user.”

More about the executive order in a moment but it’s worth pointing out that Wisconsin has also taken some steps—albeit baby steps—to set up guardrails and provide oversight of artificial intelligence.

In the Badger state, in 2023 the governor signed two laws concerning AI, one requiring disclosure in certain political advertisements that use generative AI to produce audio or video content, and another criminalizing the possession of virtual child pornography, which includes obscene material that contains a “depiction of a purported child,” defined as a visual representation that appears to depict an actual child, but may or may not depict an actual child.

In February, too, a legislative task force led by state Sen. Julian Bradley (R-New Berlin) made seven different recommendations for the state to pursue, including extending existing laws regarding disinformation and impersonation to AI models, and also a call for legislative oversight of AI in “state level” governance. (The governor set up a task force, too, but it predictably is worthless, more obsolete pandering to “initiatives to advance equity.”)

The legislative task force’s call for legislative oversight of AI in state agencies is of critical importance and needs to be quickly acted upon, given the speed at which AI is advancing.

However, while both the Trump executive order and the legislative task force give a nod toward the need to scrutinize the use of AI by the administrative state, both are woefully inadequate in this most understudied area of AI concern.

They are starting lines, in other words. But it’s way past time to get off the mark and run the race, because the most malevolent of AI uses are already heading for the finish line. If AI can be used to increase efficiency, specialize tasks, and perform services more quickly, it can also be used to shift accountability from bureaucrats to technology, to reinforce and extend bureaucratic biases, and to reduce transparency by hiding bureaucratic action and interpretations “behind the code.”

Make no mistake, right now bureaucrats everywhere are gearing up their AI for the latter, not the former. They are using it to reduce transparency, to extend biases, and hide their actions. Because that’s what bureaucrats do.

Worse still, bureaucracies can and have used AI models as effective decision-makers, despite ongoing problems with flawed outputs, and have also replaced street-level living, breathing regulators with inflexible models that can never connect with the people and communities they regulate.

The use of AI by the state and federal bureaucracies needs to be addressed now.

Trump’s approach

First let’s take a look at the Trump executive order.

As Christopher Rufo, who helped to draft the order after being contacted by Trump’s AI czar David Sacks, has recounted on his Substack, what the administration wanted to do was “to define the problem of artificial intelligence getting skewed ideologically.”

“Several weeks ago, Sacks reached out to me with a question: How can we define ‘woke AI,’ and what principles can we enumerate to prevent the government from purchasing ideologically compromised software?” Rufo wrote. “The answer begins with the fact that artificial intelligence companies deliberately select the values embedded in the code base, which chatbots use to formulate responses to users’ questions.”

The executive order marches from there, recognizing that AI will play a critical role in how Americans of all ages learn new skills, consume information, and navigate their daily lives but also asserting that, in delivering those benefits, AI must issue reliable outputs that are ideologically neutral. That’s not the case today, when AI has already infected much of the federal government with woke values.

“One of the most pervasive and destructive of these ideologies is so-called ‘diversity, equity, and inclusion (DEI),’” the EO states.

As Rufo wrote, the theory behind the executive order is that Washington has an enormous influence over technology companies via contracting, and so the order will work to reduce biased systems not just in the federal government but universally.

“The federal government is often the largest customer for big high-tech firms, and therefore, the government’s contracting requirements will profoundly influence the future of artificial intelligence, which could become a multi-trillion-dollar industry,” he wrote.

Still, there are outstanding issues.

For one, the executive order demands ideological neutrality—a noble goal—but, as Rufo himself points out, there’s no such thing: “The choice of values is inevitable. All artificial intelligence companies have, explicitly or implicitly, baked an ideological formula into their ‘constitutions,’ ‘system cards,’ ‘alignment principles,’ or ‘trust and safety rubrics.’ The question is not whether an AI system will be built upon a set of values; the question is which set of values the programmers will select.”

So, then, what’s to prevent another administration from switching it all up and embracing woke again, or some other obnoxious ideology. The answer is, nothing, at least in this executive order.

Of course it can be argued that codifying the procurement standards would make the directives hard to undo—it would take new statutes—not to mention the rippling impact that changes in federal contracting would have through the whole ecosystem, as Rufo pointed out.

But all that ignores an even more sinister and substantial problem, the real conundrum we face: not merely the procurement of value-neutral codes, or at least value-competitive ones, but the internal manipulation of those codes by bureaucratic partisans inside the deep state.

Remember, the bureaucracy rules, with its rules. And that’s not just the regulatory code, but the internal rules by which bureaucracy lives. Anybody, or anything, that comes within the hallowed halls of the administrative state must conform.

Even AI.

The recent rant by X’s Grok is a perfect if extreme example of such internal manipulation. In July, Grok starting spouting pro-Hitler statements, calling itself MechaHitler, among other things, apparently because of a “deprecated code update,” whatever that is, and in May the chat box started making comments about “white genocide in South Africa,” which issued after an “unauthorized modification” of the code by a rogue employee, according to X.

These were deliberately outrageous comments and grand-standing manipulation, obviously. Don’t expect bureaucrats in the deep state to be so obvious. They live in the deep state for a reason. Clearly federal jurisdiction needs to be claimed by Congress and legislative oversight codified, and it needs to address not merely procurement of AI models but the use of those models by the administrative state’s field soldiers.

And so too on the state level

In Wisconsin, the two bills enacted into law in 2023 are good pieces of legislation, but they really are only peripherally concerned with AI; they merely apply old law to new technology.

The work of the legislative task force was more important, but ultimately it too failed to delve into the bureaucratic use of AI, though its members touched on some points of departure if such an inquiry is undertaken.

Basically, the subcommittee made seven recommendations, which I repeat here, if not quite verbatim:

First, instead of focusing on regulating the emerging technology that is AI, the legislature should focus on ensuring that data, the “raw material that powers AI,” remains private and the consumer protected.

Second, the Legislature should learn from the experiences of other states and avoid the potential overreach of comprehensive AI legislation, and should instead prioritize high-risk areas susceptible to exploitation or abuse.

Third, the Legislature should ensure that existing laws apply to AI models in the same way that those laws apply to individuals, but should avoid creating duplicative statutes that unnecessarily single out AI.

Fourth, the legislature should ensure that programs related to education and workforce development, such as Fast Forward, have a scope that is broad enough to include AI up-skilling, training, and education; are funded accordingly; and work to address any disparity in access between rural and urban communities.

Fifth, the Legislature should consider establishing a permanent study committee, new legislative standing committee, or inter-branch commission to review emerging technologies, including AI, and make legislative recommendations regarding the same.

Sixth, the Legislature should, as AI technology advances, examine and invest in technology powered by AI that will assist with public safety, such as gun detection software.

And seventh, the legislature should direct the executive branch to promulgate administrative rules to establish clear, consistent guiding principles for state-level AI governance and to provide the legislature with oversight regarding the state’s procurement, development, and use of AI.

Now most of these are just performative, the kind of vague empty shells that lawmakers spout out when they don’t have a clue but want to be seen as actually doing something.

Then there’s one that’s downright scary: Using AI for gun detection is a recipe for disaster, not to mention a egregious violation of individual liberties. As the Electronic Frontier Foundation has pointed out, the software “rarely produces evidence of a gun-related crime” and often misses weapons that people are carrying, but then fingers people who aren’t carrying. In a New York City pilot program in the subways last year, there were 118 false positives out of 2,749 scans, or 4.3 percent. There were no guns found.

Of course, when officers approach someone they think is armed, that automatically makes for a dangerous situation, and particularly so for the person who turns out not to have a gun. And, say, what about illegal searches?

Earlier this year in Florida, for those very reasons two Republican state lawmakers introduced legislation to ban AI gun detection, except for certain exemptions such as in police stations, prisons, and courthouses. State Senator Blaise Ingoglia called the use of AI for gun detection “nothing but a technological infringement upon both our 2nd and 4th Amendment rights,” while Rep. Monique Miller said the state should not allow local governments to infringe upon either the right to carry a firearm or the Fourth Amendment right to not be illegally searched just because artificial intelligence makes it possible.

I agree with the Florida lawmakers. Not investing would be the better way to go. Otherwise we’re just slip-sliding our way to totalitarianism, one technological advance at a time.

But the real swing-and-a-miss for the Wisconsin legislative AI task force was calling for the governor to direct the bureaucracy to promulgate rules to establish guiding principles for the state’s use of AI. Allowing the bureaucracy to set rules for itself would be a blunder of massive proportion, and, given the recent Supreme Court decision on legislative oversight, there would be no oversight of those rules.

Vague direction on rule promulgation is what causes most of our troubles. The legislature needs, through committee or task force, to do the job itself and by enumerated legislation establish the specific concerns needing to be addressed, as well the standards for doing so. That’s the first action, and it needs to be undertaken soon.

And the standards are …

To address the issue comprehensively, the legislature needs to understand just how the bureaucracy can use AI to augment the undermining of democratic function. One way is by policymaking (or call it what it really is, lawmaking), which is what we usually hear about and what the Trump executive order attempts to take aim at, though it stops at the front doors of the bureaucrats.

Still, when political bias and ideological currency are built into the foundational codes and algorithms of any system, it bathes every policy decision in its waters, and Trump’s executive order homes right in on that kind of bias and abuse.

“For example, one major AI model changed the race or sex of historical figures—including the Pope, the Founding Fathers, and Vikings—when prompted for images because it was trained to prioritize DEI requirements at the cost of accuracy,” the order stated.

Second, when it comes to delivering public services, AI poses a dire threat because it tends to reinforce the incompetencies or ideologies of those tasked with delivering them. That’s because bureaucracy does not tolerate dissent. It demands conformity. Bureaucracy leaves and breathes on its own terms and for its own sake. In that universe, any technology, and especially AI, becomes not a tool for efficiency or service to the greater good but a weapon for fortifying bureaucratic power and compounding inefficiencies. The tormenting bureaucracy becomes even more persecuting, and small errors become large catastrophes with real-life consequences.

To cite just one example—and there are tons—one need only look as far as Thomson Reuters’ Pondera Solutions, a company Reuters acquired in 2020. Here’s how Reuters touted its acquisition:

“The acquisition of Pondera Solutions will enhance the offerings in the risk, fraud and compliance space and will allow Thomson Reuters to expand on its strategic approach to deliver insight through advanced analytics, artificial intelligence and human expertise.”

“Enhance” was not quite the words used in a Federal Trade Commission complaint filed against Reuters and Pondera by the Electronic Privacy Information Center in 2024. That complaint pointed to the use of Pondera’s fraud AI by 42 states. Typically, the complaint states, the AI software raided personal data from just about everywhere to make “predictions” about fraud.

“To make these predictions, Thomson Reuters compiles sensitive data about public benefits recipients and retailers from both government and third-party, commercial data sources,” the complaint states. “The data points that Thomson Reuters compiles and uses for fraud predictions include, inter alia, recipients’ home addresses, how far recipients travel to buy groceries, affiliated persons, and social media profiles.”

That kind of surveillance is egregious by itself, but how did those predictions work out for the states?

Well, according to a report by the California Legislative Analyst’s Office, in December 2020, the state hired Pondera to review nearly 10 million unemployment claims issued during the pandemic for potentially fraudulent characteristics.

“EDD [Employment Development Department] and the contractor identified 1.1 million claims as potentially fraudulent,” the report stated. “EDD stopped payments for these claims. Workers were not notified ahead of time. To reopen their accounts, workers had to verify their identity using ID.me or their accounts would be closed permanently. Ultimately, more than half of the claims (600,000) flagged as fraudulent were confirmed as legitimate.”

The Federal Trade Commission (FTC) complaint outlines example after example of alleged errors in the software. Here’s the thing: Despite all the allegations, not to mention documented errors with human consequences, such as in California, state bureaucracies are still jumping on board, without seeming to have the slightest concerns.

Note, too, that the complaint to the FTC comes from a private group, not the states themselves.

Third, AI is posing an ever-increasing risk to civil liberties by expanding government’s surveillance capabilities. Here in Wisconsin, the city of Tomah announced in March that it would consider using an AI program to monitor code violations by attaching a camera to garbage trucks. Such use has become widespread, with cameras using computer vision “predictive AI” to detect code violations. In some places public outcry has forced many municipalities to abandon the program, with citizens calling the scheme a “Peeping Tom on a truck.”

Finally, at the boots-on-the-ground level, AI has the potential to further disconnect the public from those who regulate them most closely, the so-called “street level bureaucrats.” There was a time, years ago, when street-level bureaucrats actually assimilated and became part of their communities, or were actually locals themselves. In the past, before ideological conformity and bias became their mission, local DNR wardens would work with citizens to find satisfactory solutions for parties contending with regulatory obstacles. They realized that most people were just trying to run a small business, or a farm, and were law-abiding hard-working citizens, not outlaws.

Those days have pretty much passed. Regulators have become dissociated government enforcers out to hammer down the law as they and the bureaucracy see it, and now, writes Kate Vredenburgh in Inquiry in 2023, AI can make the disconnect—and regulatory rigidity—that much worse.

Vredenburgh calls the use of AI at the street level comparable to that of a real but indifferent bureaucrat.

“The indifferent bureaucrat aims to process cases quickly and efficiently,” Vredenburgh writes.

AI takes the indifferent bureaucrat to new levels, Vredenburgh asserts.

“AI systems encode a commitment to what the administratively relevant facts are,” she wrote.

In other words, while modern bureaucrats have no use for case-specific contexts, AI elevates that to the extreme and takes the administrative state to its logical conclusion: Bureaucratic Pathology. Bureaucracy, with its faceless pencil-pushing enforcers, was always designed to be anti-individual; now AI is putting a smothering pillow over every human face, and rapidly.

There are no easy solutions, but clearly elected officials in Wisconsin and elsewhere need to get ahead of the curve rather than languishing behind the eight-ball. And a permanent legislative body to really dig into these issues and to provide real oversight, as the task force recommended earlier this year, would be a good idea.

There have been some solutions proposed as starting points. Crucial in the bureaucratic use of AI is transparency, and in particular the use of open-source technology. In addition, the government must require adopted AI models to be externally and internally competitive. One or two companies cannot be allowed to dominate the landscape.

All internal codes must be value-competitive and reflect a commitment to diversity of opinion—claim and counterclaim, bias and counter-bias built into model inputs, with public access to outputs as well as to the evidence for those outputs.

The Trump executive order calls for that, too. It directs all agencies to ground their AI use in two foundational principles: truth-seeking and ideological neutrality, which means value-competitive. To that I would add strict standards for the use of predictive AI so that it does not profile law-abiding citizens as criminals, or deprive eligible applicants of needed benefits.

Those are just for starters, but transparency and oversight are key. Today’s elected officials must realize that the race is not just starting; the indifferent, stalking, ideologically partisan and rigid AI bureaucrat is already halfway round the track.