Indiana’s new AI projects advertise importance of data quality

Nearly a year after Indiana launched its first artificial intelligence project, state Chief Information Officer Tracy Barnes said that work and other AI projects that followed it are sparking fresh interest in maintaining higher data quality standards across the state.

After more than three years of development, Indiana last November launched Pivot, a personalized recommendation engine for the state’s workforce. By combining the data of six state agencies involved in education, training and employment, Pivot helps job seekers sort through state workforce programs and find new work or training to advance their careers.

An Indiana Department of Workforce Development document describing the project’s progress explains: “Thanks to the use of information available only to the State of Indiana, Pivot can compare and contrast similar individuals to suggest pathways to the user that will provide a higher wage and meet the individual’s career needs and goals.”

Barnes told StateScoop that his Office of Technology followed up that project last July with a public beta test of an AI chatbot, which he said feeds on the information hosted on public state websites to help direct users to the answers they’re seeking. He said that, so far, it’s proving a more convenient option than the alternative of requiring website visitors to know the exact functions and acronyms of each state agency.

But all of the state’s AI work is igniting an interest among Indiana’s leaders in a topic that technologists like Barnes have been trumpeting for many years: the importance of maintaining clean datasets.

“It has truly forced us and a number of our agencies to start respecting and recognizing that data quality and data management has to be a part of any AI or [machine learning] journey and initiative,” Barnes said. “… It’s really kind of forcing us to evaluate our data strategy and pushing our agencies how we’re capturing, managing, tracking and organizing that data in a much more practical fashion that allows it to be ingested and provide right, expected outcomes for something like an AI engine.”

Data quality programs are gaining interest among state governments, but only 22% of states operate one, according to survey data published last week by the professional services firm Ernst & Young and the National Association of State Chief Information Officers. 

Barnes said that he and other state CIOs have been pushing for support of data quality efforts for many years, but that those efforts were often pushed aside by projects of more obvious value. But as the fuel of AI, clean data is now getting some helpful attention, he said.

“What we say in tech is never waste a good crisis, and this the other side of it: Never waste a good innovation. AI can help us from that perspective,” Barnes said.

According to Indiana’s workforce development agency, the Pivot service will offer a new AI-powered feature this fall as it begins offering users recommendations on training providers.

Barnes said early feedback of Pivot has been positive and that agencies are pleased with how many people are using it. Of 68,000 unemployment claimants between last November and last March, nearly 42,000 used Pivot to search for work.