Legislature considers bills to regulate AI decision-making

Not that long ago, artificial intelligence was a staple of science fiction, a technological fantasy akin to traveling to the stars. But with recent, headline-grabbing developments like ChatGPT’s stunningly human-like word processing, AI is not only becoming more and more of a reality every day, but it’s seeping into our everyday lives as well.

Even now, automated systems employing AI are being used to make hiring and college-acceptance decisions, controversial arrangements that could unfairly filter out worthy candidates, leading to discrimination, or infringe on the rights of individuals or marginalized groups.

Despite the stakes readily apparent to virtually everyone working in AI, the technology is largely unregulated in the United States. President Trump issued a couple of executive orders on the topic, but they’re seen as largely useless. President Biden in October released a Blueprint for an AI Bill of Rights, a document listing ethical principles for the use of AI, but it’s entirely nonbinding, with no legal authority.

The developers of AI are even floundering when it comes to governance issues. A recent study by BABL AI, an Iowa City, Iowa-based AI consultant, found that “significantly less than half of all organizations that use or develop AI [in the U.S. or Europe] have any formal or substantial governance structures for AI.”

Despite the stakes readily apparent to virtually everyone working in AI, the technology is largely unregulated in the United States.

That’s left the issue of AI governance – in the United States, at least – to be overseen by the states themselves. And, no surprise, California legislators have been quick to jump into the void.

Democrat Sens. Bill Dodd of Napa and Josh Becker of San Mateo and Assembly members Rebecca Bauer-Kahan of Orinda and Christopher Ward of San Diego have all introduced bills this session to regulate AI and the powerful potential it holds to impact Californians’ lives.

Ward’s Assembly Bill 302 requires the state Department of Technology to create a list of “all high-risk automated decision systems” that are in use by or have been proposed for state agencies by September 2024 and then deliver a report about its findings to the Legislature a few months later.

Dodd’s Senate Bill 313 would create within the Department of Technology an Office of Artificial Intelligence, which would have the authority to “guide the design, use, or deployment of automated systems by a state agency.” Becker’s SB 721 would establish a “California Interagency AI Working Group” to study AI and deliver a report to the Legislature.

Bauer-Kahan’s proposal, AB 331, is far more comprehensive, requiring the users of AI products, beginning in 2025, to perform annual impact assessments on the automated tools they build. These assessments would apply to any automated processes involving decisions on employment, education, housing, utilities, family planning, health care or health insurance, financial services including loans, the criminal justice system, legal services, voting or access to benefits or services.

Users and developers of these AI tools would be expected in their annual assessments to describe not only the purpose of their tools but to document their designs in reports they would make available to the state’s Civil Rights Department. AB 331 also requires users of AI tools in decision making to notify “any natural person that is the subject” of automated decision that the decision impacting them was made by AI.

“There just needs to be guardrails around this,” said Dodd, who described his bill as a work in progress.

He said the bill shows his interest in making an impact on the issue, but he intends to seek feedback from stakeholders and watch developments at the federal level and amend the bill accordingly. Ultimately, he said, Gov. Gavin Newsom’s administration might not want to add an Office of Artificial Intelligence, but he feels confident the administration will want to do something, and he’s prepared to help California be a leader in this sector.

Dodd said his interest in AI legislation should not be construed as any sort of opposition to the technology. He said of course everyone has had the experience of a business directing them to an automated bot, which might not always be the most pleasant experience. But he said he believes AI has the potential to be a tremendous asset to California’s economy and its people.

At the same time, he said, “it’s important that we have regulation.”

Ward agrees. He described his AB 302 as an attempt to review the automated decision-making systems the state currently employs, to make sure that those systems are treating Californians equitably.

“We don’t even know if that’s true” right now, Ward said, because no one has reviewed the state’s AI decision-making capabilities, which the assemblyman noted includes unemployment fraud detection and an identification verification algorithm employed by the Department of Motor Vehicles.

“We want to use technologies smartly,” said Ward, whose bill has already passed out of the Assembly Committee on Privacy and Consumer Protection and is now in the Assembly Committee on Appropriations.  But he also wants to ensure that automated decisions by the state are being made fairly.

Bauer-Kahan, who has been struggling with technology and privacy issues as a member of the privacy committee since she entered the Legislature in 2018, said she’s interested in AI governance because she too is concerned about the impact automated decision making can have on equity.

But she also noted to Capitol Weekly that while AI has the potential to be biased against certain groups or to increase inequality in our society, it also has the potential, if governed properly, to improve equity by eliminating human biases and producer fairer, more objective decisions.

The critical element, Bauer-Kahan said, has been trying to craft policy that provides proper oversight of biases in AI without stifling technological innovation. She said finding that balance, along with defining AI properly, was the most challenging part of drafting AB 331, challenges that she’s seen with AI on the privacy committee for years now.

“It’s been a hard nut to crack,” she said.

Hayley Tsukayama, senior legislative activist at Electronic Frontier Foundation, a nonprofit dedicated to defending digital privacy and backing AB 302, said the AI bills before the Legislature are a solid first step in regulating AI. But more, eventually, will be needed.

“AI implicates so many areas,” she said. “I really see all of this as a first swing at this.”

Brian Joseph is a former Sacramento bureau chief for the Orange County Register.