Official state records are disappearing

Jan. 13, 2010

By ANTHONY PIGNATARO

If there’s one thing our state government does exceptionally well, it’s the production of records. Every month, dozens of state agencies and departments — to say nothing of the Legislature itself — churns out thousands of pages of reports, studies, analyses, papers and fact sheets, as well laws, orders, directives and, of course, regulations. Seriously, it’s a lot; anyone suffering from insomnia should check out the California State Library’s monthly index of new government publications.

Given the way computers permeate government, the vast majority of these records exist in digital form. In fact, State Library officials estimate that “80 percent to 90 percent of all California state publications are now issued on the Web. For many of these there is no print counterpart.”

Posting documents online was meant to increase the public’s access to the workings of government, but it’s actually having the opposite effect. For years now, official government records created electronically have been vanishing. And while a June 21, 2009 University of California press release makes mention of “the wholesale disappearance of information,” no one seems to be able to quantify the extent of losses.

“The problem is, I don’t think anybody has done a scientific evaluation of exactly how many electronic-only documents of California state government are disappearing,” David Cismowski, the State Library’s bureau chief for library services, e-mailed on Jan. 7.

The state Office of the Chief Information Officer (OCIO) also couldn’t say for certain what has been lost. “While we are not aware of any critical documents that are lost or being destroyed, given the exponential growth of digital media the OCIO has been focused on the issue of records and document management,” OCIO spokesman Bill Maile e-mailed on Jan. 11. “Whether it is a digital photograph, video file, spreadsheet or word processing document, we are modernizing our approach to organizing, storing and managing electronic records.”

Terry Francke, general counsel of Californians Aware — a Carmichael-based non-profit organization that advocates for open government — wasn’t surprised by the news that state e-records have been disappearing for years.

“I don’t believe that there’s any records retention law that applies generally to documents of the executive branch,” he said. “Without some kind of legal requirement, agencies are left to their own devices. Without knowing what’s being lost or at what rate, it’s easy to over- or under-estimate the importance of this. But I can imagine it’s like a warehouse full of records burning down every six months.”

None of this is new. In fact, the deletion — accidental or purposeful — of state e-records has been going on a long time. This is made clear by examining two state reports, both released in August 2004. Ironically, despite the fact that they’re nearly six years old, the reports represent the most recent studies of the loss of government e-records.

The first is Managing and Sustaining A State Government Publications Program in California, written by Judith Cobb and Gayle Palmer of OCLC, a library services consulting firm. It’s a thorough analysis of the State Library’s function as a depository for government records that’s uncompromising in its conclusions.

“Now that most state government publications are available only through the World Wide Web, the [California State Depository Library] program fails to fulfill its mission because there are few mechanisms in place to preserve those digital publications and provide access to them over time, or even to notify librarians and the public about their existence,” Cobb and Palmer wrote, who estimate that the state government spends about $2 billion every year on its “current technological infrastructure.” “The resultant loss of state government information is untold… The disappearance of state government information has present and future implications for ever Californian. Preservation of, and permanent access to, this information is imperative; the state’s historical, cultural, and intellectual record is at stake.”

According to Cobb and Palmer’s report, “digitally published documents are dynamic, volatile and uncontrolled.” Software and hardware changes may account for some losses. Sometimes agencies will dump some records to make room for others. Still other records may disappear through “link rot” — “information that has become inaccessible because of an invalid link to a Web page, deletion or removal of Web sites, and/or loss of access to information previously published on Web pages.”

Then, especially in the case of e-mails, there’s this view from First Amendment Coalition’s Peter Scheer: “People feel compelled to delete things.”

Made public around the same time as Cobb and Palmer’s report, the California Performance Review — Governor Arnold Schwarzenegger’s big “blow up the boxes” look at state government that never really went anywhere — also identified the disappearance of state records as a big problem. Recommendation GG 45, “State Digital Records Vanishing,” stated that because “Many digital documents are deleted or otherwise lost each year… the governor should issue an executive order that requires all state agencies to alert the State Library of publication of digital documents, Web sites or other products that may be candidates for permanent public access through the State Library.”

To the best of Cismowski’s knowledge, GG 45 “was not fully implemented.” But Cismowski did say that libraries statewide are working on preserving government e-records.

“[T]he State Library has worked with the California Digital Library (CDL) and State Archives to study ways of preserving these vital electronic publications,” Cismowski e-mailed on Jan. 6.

For instance, there’s the CDL’s Web Archiving Service, which preserves access to a pretty random collection of old websites of historical import from all over the world. There’s the 2003 California Recall Election pages, a group of sites from the Guantanamo Bay Detention Camp, a wide variety of sites dealing with labor unions, anarchism and even Trostkyism, as well as various local government Web sites from around the state. Then there’s the far more massive Online Archive of California that gives the public online access to libraries and research centers across the state.

How easy it is for researchers — to say nothing of members of the public — to access these multiple archives is another story.

“Searchability is critical to access,” Scheer said. “If there’s no way to find something, then it might as well not exist.”
PHOTO COURTESY NASA.GOV