State AI projects like ‘building a plane in flight,’ CIOs say

State technology officials at a technology conference on Monday compared the process of incorporating generative artificial intelligence into state operations to “building a plane in flight,” as their agencies struggle to keep pace with the rapidly advancing technology.

Beyond other common problems state officials express, such as the lack of digital literacy among their workforces, and data privacy, they said that using generative AI tools opens state operations to concerns about cost, including sustainable energy resources to power the technology, managing the increase of public records it will create, realistic procurement expenses and unexpected legal challenges.

North Carolina CIO Jim Weaver expressed frustration at not receiving a realistic picture of the true cost of generative AI.

“The cost of ownership is what really scares me,” Weaver said during a panel discussion at the National Association of State Chief Information Officers annual conference in New Orleans. “North Carolina has a tight end-budget process, and as we’re looking at some of the use cases, is it going to cost $50 a month, $15,000 a month, $150,000 a month? We don’t know. The scalability of some of these applications could have a dramatic impact on our budgets, and we’re not quite sure what happens if we run out of money what’s going to be put at risk.”

“We’re not just buying products, we’re buying outcomes,” Vermont CIO Denise Reilly-Hughes said during a separate NASCIO panel.

Similar to budgetary concerns about long-term investments, state CIOs said they’re also uneasy about the environmental costs that may come with adopting generative AI, as the technology requires significant computing power, which can put additional strain on power grids.

The international think tank World Economic Forum in July published a report showing that generative AI systems use roughly 33 times more energy than “task-specific software.” The group estimated that AI’s current energy use represents only a fraction of the technology sector’s power consumption, about 2-3% of total global emissions, but that level will jump as more companies, government organizations and individuals adopt it.

“We’re starting to look at power-generation requirements,” Weaver said, adding that he doesn’t yet feel comfortable issuing demands of the state’s power grid for generative AI. “We’re seeing population growth across North Carolina. The power grid can’t sustain this type of growth.”

Generative AI systems will also require state data centers to hold the substantial amount of data they produce.

“[Generative AI] is going to create a lot more public records that historically have not been accessible or available to media, discovery, legal action, and engagement. And so this is an area that we’re going to have to take a look at,” Weaver said. “We already struggle today to do public records, so I’m not quite sure we’re there yet.”

Paula Peters, Missouri’s deputy chief information officer, said during the panel that a lack of transparency is creating anxiety for state attorneys, because vendors don’t always disclose how — or even if — AI is being used in their products.

“We’re scaring the heck out of our lawyers,” Peters said. “How is your application using AI? Where’s the data coming from? Are you disclosing to the users that you’re using AI in the system? If you are, are you asking for permission? Are you giving them that information that contains AI or this using AI? A lot of those things are things that we have to start talking about.”

Aside from the new challenges, Weaver said he’s also excited about the new solutions the technology brings, such as legislative code analysis and bill summary.

“Our legislature in North Carolina is infamous for hiding stuff in bills that I never would have read,” Weaver said. “So we really need the ability to do that code analysis and start doing summation and making sure that we’re saying that fully being what’s being asked of us.”

However, he also acknowledged that public trust in how state governments deliver services is fragile, and he encouraged states to tread lightly as they adopt and roll out new AI technologies.

“You get one opportunity to get this right, and then after that, people don’t trust what you’re delivering,” Weaver said.