Weak AI regulations may leave artificial intelligence less safe - Earth.com

New research warns that weak AI safety regulations can actually make artificial intelligence less safe.

Instead of encouraging companies to improve their products, poorly designed rules may shift responsibility in ways that reduce overall safety.

The risk appears when regulations target only the companies that adapt AI for specific jobs, while the firms building the underlying models remain largely untouched.

As states and countries roll out new AI laws, the study suggests that who gets regulated may be just as important as how AI is regulated.

Benjamin Laufer, a doctoral researcher at Cornell University, built the analysis with his adviser Jon Kleinberg, in collaboration with Professor Hoda Heidari of Carnegie Mellon University (CMU).

Together they modeled how safety rules ripple through the chain of companies that build and sell AI. That chain usually has two links.

One company trains a large, general-purpose AI model, the kind that powers popular chatbots.

Another company takes that model and adapts it for a narrow job, like reading medical scans or handling customer complaints.

The team’s central result is blunt. A weak rule placed only on the second company can lower the safety of the finished product.

It sinks below the level those same companies would have reached with no rule at all. That result surprised them. It also held up across a wide range of settings in their model, not just one lucky example.

The researchers treated the two companies as players in a game. Each one chooses how much to invest in AI safety and raw performance.

Safety costs money. So does performance. Whatever they build, they split the revenue it earns.

The model maker moves first and sets the starting point. Then the domain specialists who adapt the model decide how much further to push it.

Before any of that, the two sides strike a deal on how to divide the eventual payout.

A regulator sits above all this. It can set a minimum safety level for the first company, the second, both, or neither.

The researchers then solved for how rational, profit-seeking firms would respond.

This builds on earlier work from the same group on how general and specialist firms bargain over fine-tuning. The new twist is the safety floor and the question of who should have to clear it.

The backfiring comes from a simple piece of self-interest. Consider a rule that forces the downstream company to meet a set safety bar.

The model maker now knows the final product will clear that bar no matter what because the law requires it.

So the maker can quietly spend less on safety. The downstream safety floor does the work instead. In the unregulated version, the maker had reason to invest more because no one else was guaranteed to.

“There’s a free-riding behavior that occurs,” said Laufer. “The regulation acts as a tool for the general provider to offload the safety burden onto the downstream specialist.”

The net effect runs backward. The rule was meant to raise safety, but it hands the upstream company an excuse to cut corners. Total safety ends up lower than before.

They found this holds whenever both companies share revenue and both put in real effort. That covers a broad slice of how AI is actually built today.

The second finding runs the other way. Rules aimed at both companies, set at the right level, can make products safer and leave both firms better off.

The reason is trust. Two companies building a product may both want higher safety, yet neither can trust the other to follow through.

Each has a private incentive to skimp at the last minute. So the safer, more profitable path never gets taken.

A rule that binds both sides removes the guesswork. Neither has to take the other at its word. That lets them reach a combination of safety and profit they both prefer but could not lock in alone.

“The goal of regulation should be the mutual benefit of everybody in society, and this can include those developing the technology, but also end users and the public,” said Laufer.

This is why some companies openly ask to be regulated. A well-placed rule can act like a contract with teeth, and both parties may be willing to pay for it.

A separate study similarly found that safety rules can ease a firm’s entry into a market rather than block it.

Right now, much of the debate is guesswork. Recent proposals differ on which companies should bear responsibility for AI safety, and there is little hard evidence showing how those choices play out in practice.

“There isn’t much AI safety regulation, and so a lot of possible regulations are just proposals at this stage,” said Laufer.

“To some extent, regulation is poking in the dark, so it’s worth reasoning through what effects these regulations might have on incentives.”

The model offers one clear lesson. Aiming safety rules only at the companies adapting AI for niche uses can quietly make things worse.

Spreading the requirements across both the model maker and the adapter tends to work better.

That question is already shaping real policy. The European Union’s AI Act and past state bills have wrestled with exactly where to draw the line.

The team’s findings may also extend beyond AI safety.

A related analysis from the same researchers found that rules encouraging AI models to become more open can backfire in much the same way.

The study is still a simplified picture, and the authors want to test their predictions against real-world regulations as they emerge.

Even so, the work highlights a trap policymakers may want to avoid. Weak rules aimed at the wrong companies could do less than nothing.

The study is published in Proceedings of the National Academy of Sciences.

—–

Like what you read? Subscribe to our newsletter for engaging articles, exclusive content, and the latest updates.

Check us out on EarthSnap, a free app brought to you by Eric Ralls and Earth.com.

—–