History of the Electronic Records and ERA
The National Archives and Records Administration (NARA) has been involved with taking in, preserving, and providing access to early types of computer records since1965. The first transfer of electronic records from US Federal agencies came to NARA in 1969.
In the 40 years since the 1960s, managing electronic records at NARA can be described by three main periods – Formation, Rejuvenation, and Transition. In addition, a major event occurred around 1993 regarding electronic records of the Executive Office of the President in which NARA was a co-defendant (Armstrong v. EOP).
This section provides background on ERA's history, including:
- 1969-1993: Formation, stagnation, then rejuvenation
- 1993: Armstrong v. The Executive Office of the President
- 1998: Transition to e-Government supports an archives of the future
- 2000: Establishment of the ERA Program Management Office
- 2004: Seeking a development contractor
- 2008: Initial Operating Capability
- 2009-2011: Last phases of development
From 2009 thru 2011, new capabilities and enhancements to existing functionality were under development to support the processes for scheduling records, transferring records to NARA, ingesting and processing electronic records, and providing access to publicly available electronic records through the Online Public Access prototype. The focus of this last year of development is on capabilities that will ensure successful adoption of the ERA System by all agencies of the Federal government by the end of 2012. At that time, all Federal agencies will be required to use ERA to schedule and transfer to the National Archives all permanent electronic and non-electronic records.
In January 2009, a second module of ERA was deployed that is designed for presidential electronic records. Increment 2 capabilities included:
- Rapid ingest, search, and retrieval of Executive Office of the President electronic records from the George W. Bush Administration;
- Transformation of proprietary data;
- Simple search and access retrieval;
- Spillage management;
- Access control; and
- Basic case management for special access requests.
In June 2008, the ERA System’s Initial Operating Capability (IOC) was successfully deployed, providing basic infrastructure and support of NARA’s business processes for managing Federal records.
In 2004, NARA conducted a one-year competition between the Harris Corporation and Lockheed Martin to design a prototype for the ERA System. In 2005, Lockheed Martin was awarded a design contract to build the foundation of the ERA system that would be developed in five increments.
From the start, it took NARA ten years to develop the Electronic Records Archives (ERA) System. During that time, several key events and activities were accomplished:
- Exploratory research began in 1998, to determine whether it would be possible to build an ever expanding system that could handle the increasingly complex electronic records that would have to be preserved permanently. NARA’s first step was to establish important critical partnerships with world-class experts in computer science, information management, information technology, library/academic and private/public sector communities.
Many of these key partners have sustained their interest and understanding of the complex information technology and archival challenges related to the preservation of electronic records, and continue to work alongside NARA today.
- To build a system like ERA, NARA needed to understand and carefully specify its business requirements. NARA worked with its staff and a variety of, stakeholders from across the Federal government and from the public to specify what they would like ERA to be able to do.
These discussions led to analyses of NARA business processes and operations, and to the development of the ERA System Requirements Document – the document that spells out the requirements that defined what a system like ERA would be able to do to help NARA conduct business.
With 853 high-level requirements mapped out, the ERA system was on its way to begin development.
- 2000: the ERA Program Management Office becomes official
NARA had never developed a large computer system, and needed to be able to manage the development of a complex system such as the ERA system.
In 2000, NARA established the ERA Program Management Office (PMO) – patterned after PMOs used by the Department of Defense to manage complex IT projects. The ERA PMO was approved and funded under a separate line item of NARA’s annual budget request, to ensure annual funding for this mission-critical program.
- In 2004, NARA conducted a one-year competition between the Harris Corporation and Lockheed Martin to design a prototype for the ERA System. In 2005, Lockheed Martin was awarded a design contract to build the foundation of the ERA system that would be developed in five increments.
Also in 2005, NARA established an Advisory Committee for the Electronic Records Archives (ACERA), a deliberative body of outside experts who advise the Archivist of the United States on technical, mission, and service issues related to ERA.
- In 2007, four Federal agencies became early users to work closely with NARA to test the early releases of ERA. Those agencies were the U.S. Patent and Trademark Office, the National Nuclear Security Administration (Kansas City plant), the Bureau of Labor Statistics, and the Naval Oceanographic Office.
Under the leadership of Dr. Kenneth Thibodeau, the ERA Program Director and former head of NARA’s Center for Electronic Records, the ERA Program has been highly successful. NARA has received the steadfast and support of the ERA effort from three different NARA heads, three U.S. Presidents, and several Congresses through increased funding.
Around 1998, consequences from these and other events led to the exploration, research, planning, and funding for the establishment of NARA’s Electronic Records Archives (ERA) Program in the year 2000.
As discussed in this ERA history section, the volume of electronic records that first came to NARA in 1970 was approximately 6,000 data files; twenty five years later, the number was well over 200,000 data files. Around August 1998, the Archivist of the United States, John Carlin was given statistical charts and analytical reports developed as a result of the 1993 lawsuit against the Executive Office of the President (Armstrong v. EOP) that illustrated NARA’s turmoil: the continuing, exponential volume of electronic records.
The Federal government's increasing reliance on computers was leading to an overall transition to eGovernment - that is, the use of information and communication technology to provide and improve government services, transactions and interactions with citizens, businesses, and other arms of government. The records generated from these services and transactions would be varied and complex (e.g. satellite data, image files, relational databases, email), and would also include future formats of electronic records that would yet to be generated by emerging technologies.
During this time, NARA began to anticipate the tsunami of even newer electronic records that would overwhelm an already backlogged processing unit of archivists at the end of the Clinton administration in the year 2000.
From the statistics and analyses, Carlin understood that NARA’s current systems and operations could not scale to handling to the projected volume of records coming in. As a result, he authorized exploratory research into the technological possibilities for handing the ever growing volumes and variety of electronic records – leading to the start of research and collaborations that would become the Electronic Records Archives (ERA) Program.
A major upheaval came to NARA in 1993 when a lawsuit against the Executive Office of the President of the U.S., involved NARA as a co-defendant [Armstrong v. Executive Office of the President, 1 F.3d 1274 [DC Cir 1993]. This event focused both Government and public attention on a specific form of electronic record -- electronic mail – which was increasingly being used by Federal government agencies to communicate with each other, including communications with the White House.
The lawsuit targeted email created under the administrations of Presidents Reagan and George H. W. Bush, that included, for example, email correspondence pertaining to the Iran-Contra affair. These email records contained information of historical value that demonstrated what officials knew, and when they knew it. Court decisions in the case ultimately led to the transfer of email to NARA in January 1993 of the first major presidential electronic records.
NARA’s turmoil begins
As discussed in the early phases of NARA’s history of electronic records, NARA’s holdings of electronic records was approximately 15,000+data files. The volume of digital files transferred in the lawsuit included thousands of times more digital files than all those NARA had received in more than 20 years – increasing to about 200,000 data files of electronic records. Moreover, this transfer included a great variety of digital media and file formats – many of which NARA had never seen before – and much of the media that contained the records had suffered physical deterioration.
Predicting a tsunami
The responsibility for preserving the early electronic records, as well as taking in and preserving these new records coming in to NARA as a result of the lawsuit, was a grave concern for NARA because a new, even larger volume of complex records was expected to come in at the end of the Clinton administration.
Approximately 6,000 data files began arriving to NARA between 1970 -1988. These were primarily simple “flat” database and ASCII records created by early computers commonly used by some Federal government agencies.
Around 1989, improved methods for taking electronic records were introduced to NARA’s Center for Electronic Records – including two new systems, the Archival Preservation System (APS), and the Archival Electronic Records Inspection and Control (AERIC) system – more than doubling the volume of electronic records to a little more than 14,000 data files for which NARA was now responsible for maintaining.
While these were major improvements, the two systems provided very rudimentary capabilities for processing and maintaining the early electronic records. For example, a simple computer program was written to copy each transferred file to a new magnetic tape for physical preservation, and to print the start of each file, so that archivists could visually inspect the printout to see if the contents of the files corresponded to technical information we had about them. This was a tedious process for the archivists. Public access to these electronic records was limited; records were available only by purchasing copies of the files on magnetic tapes or, in a few cases, ordering printouts of the files. And these processes were applied only to very simple types of electronic records.
Around 1993, a lawsuit occurred, in which NARA became a co-defendant – significantly increasing the new volume and new formats of electronic records created under the Reagan and George H.W. Bush administrations.
Return to the ERA Main Page