One quarter of recent web pages have disappeared, Pew researchers found

One quarter of web pages that existed sometime between 2013 and 2023 were no longer accessible as of last October, according to a study published by Friday by the Pew Research Center.

In most cases, researchers found, much of the internet’s missing content is due to pages being removed from otherwise functional websites. The study, which included examinations of news and government websites, as well as social media posts, found that more than half of Wikipedia pages contain at least one broken “references” link.

Pew researchers, who called that phenomenon of missing content “digital decay,” found that 21% of government websites contain at least one broken link. Local government websites were the most likely to have broken links. Researchers said they evaluated 500,000 government websites containing 42 million links, most of which were internal links.

Across all types of websites, the trend was found to be more prevalent for older content — 38% of 2013 web pages were missing, compared to 8% of 2023 web pages.

Ensuring users of government websites are able to find the content they need and avoid frustration is a growing priority among government technology officials. New York state last year hired Tonya Webster, a former executive with Blue Cross Blue Shield of Massachusetts, Comcast and Citigroup, as chief customer experience officer to ensure agencies are improving the digital experiences residents have when interacting with their state government.

Webster told StateScoop New York has various initiatives underway to ensure seamless online experiences, including conducting surveys to identify broken links.

“We are launching voice of the customer surveys on our websites to capture user feedback,” Webster wrote in an emailed statement. “The survey feedback will help us better understand pain points on our web pages such as broken links and will provide us with a greater ability to address areas for improvement. Our goal is to ensure that our websites are consistent, up-to-date, and provide a frictionless experience for users.”

Webster said her office is also working with other state agencies to update content and links across their websites, as well as redesigning several old websites, including the Higher Education Services Corporation website.

“Improved web experiences for government pages is a core piece of our strategy,” she said. “We are addressing digital decay by staffing up and leveraging best-in-class tools. We are deploying a common set of standards and guidelines across the state for our websites, and implementing design systems, as Governor [Kathy] Hochul highlighted in her 2024 State of the State.”

Pew researchers also spent three months looking at digital decay on Twitter, where nearly one-in-five posts had been deleted or been made private.

Communities of digital archivists have emerged to preserve online content, while websites like the Internet Archive provide a resource for researchers, journalists and academics looking for websites deleted from their original locations. Others concerned with even longer term concerns of preserving media, such as those behind the Eternal Access Project, have taken to storing digital copies of “banned, threatened and at-risk books” in the Norwegian arctic.