Four Months to the Millennium: Are We Ready?
The View from the National Archives
JOHN W. CARLIN
Archivist of the United States
Address to the Society of American Archivists
August 26, 1999
[Luciana Duranti, president of the Society of American Archivists, introduced John Carlin.]
Thank you very much for your kind introduction. I took great pleasure in seeing recently that you've been honored by the academic community as well as the archival community. Let me join your colleagues in congratulating you on your Distinguished Academic Award from the Confederation of University Faculty Associations of British Columbia. They didn't ask me, but I certainly would have testified that you are a stimulating thinkerand a forceful educatorwhich I know from personal experience.
I have learned much from you and your colleagues. And I've enjoyed what I think is the success of our mutual efforts to keep SAA and NARA together on issues and maintain close communication. When I met earlier this summer with the SAA Council in Washington, we talked about our progress in staying in touch, and somebody remarked, "The archival profession is stronger with NARA now more in it." I'd just like to say that NARA, too, is stronger. And I look forward to continuing to work closely with you, your colleagues on the SAA Council, your staff Director Susan Fox, and your successor as president, Tom Hickerson, to whom I offer warm congratulations.
I welcomed your invitation to speak this morning as an opportunity to keep our communications going. And there are things I'd really like to say about your conference theme, "Meeting the Challenges of Contemporary Records." You're dealing with that theme in three parts, as I understand it: "creation, preservation, and access." I'd like to start with three stories that illustrate different facets of what we're up against.
Last year I published an article in the Washington Post about the challenges of new technologies to recordkeeping. And lo and behold, substantial reports on the subject have subsequently appeared in a half-dozen significant publications. One of these was the Wall Street Journal. It declared that "the National Archives has a monumental challenge: making sure government records are preservedand accessible" in the electronic age. I had to grin after reading that, however, because right on the same page was some company's advertisement showing several business people studying a problem, with this caption: "With enough crumpled paper, you can solve anything."
Alas, for archivists, crumpled paper is part of the problem.
My second story comes from another of the press reports I mentioned on electronic records issues. Many of you know James Henderson, Maine's state archivist. The Associated Press asked him how electronic records differ from traditional kinds. I hope Jim actually said what they printed because it's wonderful. The AP quoted him as responding,
"You're dependent on the machine . . . The machines are constantly changing, and the software is constantly changing, and the media on which the information is stored is constantly changing. In the good old days, when man was deciphering hieroglyphics on rocks, at least the rocks just stayed right there."
Making electronic media hold still is, indeed, a problem.
Finally, my third story comes from a rather surprising e-mail message we recently received at NARA. The message was signed by, quote, "DearMYRTLE, Daily Genealogy Columnist." The message said:
"Dear sirs, Your excellent presentation of the National Archives Digital Classroom has won DearMYRTLE's Best of the Internet for Genealogists Award this week."
Well, that award may not be quite as prestigious as the one Luciana received. But it certainly encourages us to recognize, in our work to provide electronic access, that the public is watching.
Very recently, NARA received news in another area the public has been watching. After months of litigation in the GRS 20 lawsuit, an appeals court has upheld NARA's authority in promulgating GRS 20. SAA was not a party to this case, but you have taken a strong interest in it. In fact, your statement regarding the case when it was first filed was very useful to me as I considered how to approach the issue. And I'd like to talk about what this decision really means.
In 1995, soon after I became Archivist of the United States, I issued a revision of a general records schedule, GRS 20, for the disposition of some Federal electronic records. GRS 20 authorized Federal agencies to dispose of certain electronic copies of e-mail, word-processing documents, and other computer-generated material if first saved by copying to a paper, microform, or electronic recordkeeping system.
In 1997, in response to a lawsuit opposing this action, a Federal district court ruled that portions of GRS 20 reached beyond my authority under the Federal Records Act, and declared GRS 20 null and void. In response to the Government's appeal, a three-judge panel of the U.S. Court of Appeals has now overturned the lower court decision.
However, the decision of the Court of Appeals does not take effect until seven days after the 45-day time period during which plaintiffs have the right to seek a rehearing before the entire Appeals Court. That means that the district court's orders remain in effect until at least September 27. If the plaintiffs do seek a rehearing, then the decision would not take effect unless and until the Court of Appeals denied the plaintiffs' motion. In the meantime, the orders of the District Court declaring GRS 20 null and void, but allowing agencies to continue their current disposition practices, remain in effect.
I am pleased that the Appeals Court has upheld my legal authority as Archivist and the propriety of my action. But the fact that in issuing GRS 20 I behaved legally does not mean that all of the policies in GRS 20 were sound. The work done by the Electronic Records Working Group last year, in which SAA members participated and provided advice, only reinforced that NARA must look for new ways to manage electronic records. And I do welcome the opportunity this decision provides to continue to work in an orderly way to develop practical, workable strategies and methods for managing and preserving records in the electronic age and ensuring ready access to them.
Regardless of how the GRS 20 decision works out, NARA is planning to undertake this year a far-reaching effort to examine comprehensively how records in all formats are scheduled and appraised in the Federal Government. The responsibility for approving the disposition of records is perhaps the most critical statutory responsibility held by the Archivist of the United States, because it determines how long records must be kept to protect individual rights, provide accountability in government, and document the national experience. Therefore, the scheduling and appraisal policies and process that we use to carry out this responsibility are central to NARA's mission. The process is an important tool for managing records throughout their life cycle, and it's a process that our strategic plan identifies as flawed and in need of a major overhaul.
The scheduling policies and processes developed during the 20th century and currently used by the Federal Government are based primarily on paper-based recordkeeping systems as they were used at mid-century. The reality at the end of the 20th century is that most records are created electronically and may be maintained in a variety of media. Each of the media typespaper, microform, film, electronichas advantages and disadvantages as a recording medium. And agencies need to know how to manage the disposition of all documentation they create, regardless of media.
Therefore, the goal of this project is to define what should be the Federal Government's policies on determining the disposition of Federal records, the processes that will best implement those policies, and the tools that are needed to support the revised policies and processes. We must answer a number of basic policy questions about Federal documentation, the goals and purposes of scheduling, the appraisal criteria to be used in determining appropriate retentions, and the respective roles of NARA, Federal agencies, and the public in achieving the goals and making the process work effectively.
This is the first in a series of efforts we will make to review our policies to overhaul all the processes used to manage records throughout their life cycle. And as we move forward in this project, we hope to have wide participation from agency personnel, NARA staff, professional organizations, and others.
Moreoverand this is what I most want to say this morning I think we're really beginning to get somewhere in addressing the challenges of contemporary records. With new funds in our current budget, we have filled thirteen professional positions, all requiring experience in all media, including electronic, in our records management staff. This will allow us to expand assistance to Federal agency headquarters and field offices through our new Targeted Assistance Program. And if we get the funding we requested for next fiscal year, we will fill another eighteen professional positions, bringing our total new positions for this effort to thirty-one.
But adding staff alone won't solve our electronic records management problems. We must work in partnership with others to find solutions. And we are doing just that.
For example, concerning electronic recordkeeping systems: With input from NARA staff and quite a bit of help, I understand, from Dr. Duranti and her colleagues, the Department of Defense developed a set of baseline requirements for the management of its electronic records, and issued criteria for the design of computer software for use in electronic-records management. After independent evaluation, we at NARA endorsed this DoD standard as consistent with the Federal Records Act and of potential usefulness to other Federal agencies.
The standard does not answer all pertinent questions nor preclude other approaches, but it certainly does provide at least a starting point for agencies that want to begin implementing electronic recordkeeping now. We are at this time reviewing DoD's certification process for software meeting baseline requirements. And we are working with DoD and other Federal agencies on several other technical projects of potential value to government agencies at all levels in dealing with electronic records.
Additionally, we've now launched a "Fast Track Guidance Development Project" to identify "best practices" currently available and to provide guidance quickly on electronic-records issues that urgently confront Federal record keepers nowguidance they can use while work goes forward on developing more complete and longer-term solutions. "Fast Track" is a collaborative project involving persons with relevant expertise from NARA's staff, individuals from other Federal agencies with expertise in records management, systems development, and information technology, and expert consultants from outside the Federal Government.
I trust by now it's clear to you that, whatever the final outcome of the GRS 20 case, NARA is not backing away from efforts to meet the challenges of electronic recordkeeping. It also should be clear to you that partnerships are the key to progresspartnerships, like I've described above, of NARA archivists with others of you and with other agencies, all working together. And finally, it should be clear to you by now that in such projects as the reinventing of scheduling that I described, traditional archival policies and practices will be under review, and results could be meaningful for the archival field as a whole.
Now let me add one more major element to the picture I've been drawing, a picture of what's beginning to look like even more progress. We mayand I emphasize "may" because we're still in the R & D stagebut we may have in sight a significant technological breakthrough: the creation of a workable way to provide both preservation and access for all data types without dependence on particular software or hardware.
For some time now, NARA has been able to accession small numbers of large electronic databases. But the government is increasingly generating large numbers of small files, such as e-mail messages. Facing the fact that available technologies are not adequate for the task of preserving and providing access to them, we joined the National Partnership for Advanced Computational Infrastructure to see if advanced computer science could provide a solution. Just recently we were briefed on a prototype system developed by our partners at the San Diego Supercomputer Center, a national laboratory for computational science and engineering at the University of California, San Diego. The prototype has been able to "ingest"their worda million e-mail messages in just two days.
That is progress. But additionally, research being done for us at the Supercomputer Center, and also at the Army Research Laboratory, gives us hope that an Electronic Records Archive can be built to preserve any kind of electronic record in a format that frees it from the computer system in which it was created, and will enable us to meet reference requests for it using a variety of tools available today and advanced technologies that will be developed for tomorrow. And access will be expedited by combining this system with the Archival Research Catalog we are developing to describe all bodies of records in our nationwide holdings. Furthermore, we asked our partners to develop a scalable solution. That is, NARA needs a system that will scale upward, so we can deal with accessions of millions of records, but the approach we are taking also promises to scale downward. So it will be useful for smaller archives as well, not just ours.
Whether all this works or doesn't is yet to be seen. But at least there's a chance that archives will be able to enter the 21st century with tools more suitable for a truly new era in recordkeeping.
However, there are also other things that archivists will need if we are to operate effectively in the new millennium.
For example, storage standards. As we have communicated to you in various ways and times, we at NARA are making our records center program reimbursable. That means that starting on October 1, assuming FY 2000 funding is approved by Congress, all agencies of the Federal Government will reimburse us for the costs of storing records still in their legal custody and the costs of providing other records center services.
Clearly, the reimbursable program is a significant step resulting from our strategic plan initiative to find alternative sources of funding for storage of government records. But also, one significant bi-product of that effort has been our creation, for the first time, of comprehensive standards for the storage of records still in the legal custody of the agencies. These standards will soon be finalized and made public. And then we will be working on standards for the storage of archival records. I appreciate the useful feedback you provided individually and collectively, and as we move forward I will again seek your help on what the standards for archival storage should be.
As you know, we also continue to need more and better storage space for records in formats other than electronic into the next millennium. We recognized in NARA's Strategic Plan that adequate and appropriate space for records was a need we had to address. And during much of the past year, we engaged in a space planning effort in which we sought input from customers and leaders of genealogical, historical, archival, and other organizations. SAA provided a statement on our space efforts that was useful. And based on what we learned, I've decided to focus now on our most immediate facility needs rather than a comprehensive space plan.
In addition to the renovation of the National Archives Building in Washington and a new regional facility in Alaska, our immediate space needs are to secure appropriate space for our military personnel records in St. Louis and for the Southeast Region's records housed in our East Point facility near Atlanta. Going beyond these priorities at this time did not make practical senseperiod.
Something else of major importance for the archival field as we approach the 21st century is professional training. What skills do archivists most need in the new era of electronic information? What changes are advisable in the curricula of formal education programs, in professional development opportunities offered by such organizations as SAA, and in the in-house training programs of archives such as NARA? I know that such issues are of major concern to SAA, and members of my staff have been in contact with archival educators and others in SAA about them.
Our Strategic Plan calls for a total new look at NARA's training needs and how we meet them. We already have some advice from archival educators on this subject, and we'll want to consult further with SAA, because all of us must be sure in the future that professional development opportunities provide what archivists really will need. I know that on several projects over the years, SAA has worked with the NHPRC, our grant commission, to develop educational projects. And I understand that the NHPRC is funding a conference on continuing education in Atlanta next spring. In all of thisonce againwe will be working together.
Clearly, there are other important topics of concern to us all. But I hope it is also obvious that on vital matters such as electronic records we've come a very long way.
I think back to the last time I addressed an SAA audience a couple of years ago. Then we were still alerting each other to the dangers of being overwhelmed by electronic records. We were debating what the change from traditional to electronic records might mean to the archival profession and its mission. And we were reviewing the progress of electronic records R&D projects here and there about the country, many of them funded by the National Historical Publications and Records Commission.
Subsequently, NHPRC has continued helping, and we've successfully persuaded the President and the Congress to increase financial help to NARA for meeting these needs. And today I've been able to speak more optimistically about signs of progress, which have become more tangible. It seems clear to me today that, if the funding keeps coming, and if we continue working together as we have been doing, we will be able to manage records of all kinds effectively in the new millennium; we will be able to preserve all the new media; and we will be able to assure our public access to them.
Let me conclude by saying, however, that in my mind, there has never been a choice. Sometimes, to understand the importance of something you're involved in, you have to stand back and look around more widely. Let me do that to explain what I mean.
In the 1990s, the United States has been militarily engaged in both the Middle East and the Balkan Peninsula. The Gulf War originally began, you recall, when Iraq made an attempt to take over the nation of Kuwait. But it was more than the borders of Kuwait that Iraq intended to erase. It was also the separate identity of Kuwait's people. One of the United Nations' resolutions that led to the Gulf War was a condemnation of attempts by Iraq "to destroy the civil records maintained by the legitimate government of Kuwait."
Why? Because such records document the legitimacy of a government. And such records document whatever legal standing, rights, and entitlements the citizens of a country have. Saddam Hussein certainly understood that getting rid of records, or otherwise denying access to them, makes it a lot easier to end a nation's independence and subjugate its citizens. Combatants in Bosnia understood that, too, when they shelled the National Library in Sarajevo along with other cultural institutions. And as you know, because SAA issued a protest about it, Yugoslav forces in the Kosovo War, according to a news report, were "deliberately destroying the archives of the Kosovar people: property deeds, marriage licenses, birth certificates, financial and other records," as part of "identity elimination."
The more I learn about recordkeeping, the more strongly I feel not only that it buttresses our culture and our peoples' rights and identities, but that democracy itself needs it. Open government, accountable to the people, requires open records, accessible to the people, now, and in the years to come. Under authoritarian regimes, government records support not rights, identities, entitlements, public insight, and historical understanding. Instead government files support surveillance, suspicion, political suppression, and persecution. And not the least of the importance of records in a democracy is their indispensability for justice. The government and the public both need records to defend themselves in courts that don't deny access to evidence, but allow people to present evidence so that justice can prevail.
In our country, every time NARA provides ready access to records that are useful and beneficial to people, and every time we come up with records that help people document their identities and verify their entitlements to rights and benefits as citizens, I feel we're contributing to the health of our democracy. Every time our records enable people to analyze the actions of our government and hold our officials responsible; every time we help people figure out what really happened in our history and assess the meaning of itI repeat, every time we do those things, we're contributing to the health of our democracy. And at a time in the United States when voter turnout indicates voter apathy, and polls are showing that Americans' faith in institutions is low, it's more important than ever that people be able to count on the agencies that document their rights, the actions of their government, and what is significant in their history.
Every time I go into the rotunda of our National Archives Building in downtown Washington and see hundreds of people lined up for a look at America's Charters of Freedomour Declaration of Independence, our Constitution, and our Bill of RightsI feel the significance of what we do. Every time my staff helps locate material of use to those who are tracing assets looted by the Nazis from Holocaust survivors, I feel the significance of what we do. And I feel it just as strongly when I get messages such as we received recently from a user of our electronic database. I quote, "Thanks for allowing us to access this most important database. This assures me that we are a government of the people, by the people'."
It was an electronic message in an era of new information technology. And it said a lot about why we have to meet the preservation and access challenges of that technology.
Even more basically, it said a lot about the role of archives in a free society. I said, "the role of archives," not just of the National Archives and Records Administration. NARA alone does not give this country its memory, or document all the activity for which public officials are accountable, or keep records that all organizations need to make sense of themselves, or maintain all the documentation on which citizens depend for rights, entitlements, and identities. Our heritage as a people, our prerogatives as individuals, our cultural foundations, and the conduct of our institutions are documented in records at every levellocal, state, and federaland in archives and manuscript collections, in the private as well as the public sector.
And just as we all help provide a critical social service, we also all face similar challenges. Particularly in the electronic information age, NARA has no monopoly on the talent it's going to take to meet them. You look to us for solutions. We also look to you. I do, though, want to take this opportunity to acknowledge the staff at NARA for aggressively undertaking the work laid out in our Strategic Plan so we can begin to meet the challenges of the next century. But on many of our common problems, solutions are being sought in many institutions, and we will increasingly work collectively as we share what we are learning and partner in seeking progress. And no one is more important in promoting this than professional associations such as SAA.
We all need each other in this business. That's why I'm here today. And that's why I'm grateful that you invited me to come. I look forward to continued work together.