Reaching Out - New Approaches, Best Practices, and Partnerships at NARA
John W. Carlin
Archivist of the United States
Address to The Annual Meeting of the National Association of Government Archivists and Records Managers (NAGARA)
Portland, Oregon, July 19, 2001
Good Morning! It is a pleasure to join you, and it's great to see so many familiar faces.
I always look forward to NAGARA's annual meeting because it gives me a chance to spend time with all of you who deal daily with records issues in Government agencies and archives of all sizes, throughout the country.
You know just as well as I do how important government records and archives are. You know the vital importance of taking good care of the records of Government, whether local, state, or Federal.
You know how critical records are to our citizens in documenting their rights and entitlements. You know how vital records are for government accountability in a democracy and for maintaining public trust in our public institutions. And you know how important government records are in documenting our nation's history.
You are committed, just like we are at NARA, to seeing that government records are created, managed, preserved, and made accessible for as long as needed.
The challenges that we face in the archival and records management communities are common to us all. New technologies have created and will continue to create both possibilities and problems for our profession. Electronic records are being created in large and growing quantities, and we lack adequate methods for managing, preserving, and providing access to them.
Meanwhile, paper records continue to proliferate and the costs of storing them consume much of our budget. Simultaneously, more people want more services, including the broader and quicker access that new technologies now make possible. Finally, we all face the challenge of getting people whose support we need to understand what we do and why we do it.
Five years ago this coming September, I issued a ten-year NARA Strategic Plan. Many of you in this audience helped me in creating that plan with your advice, guidance, and assistance. This plan, which has now been updated twice, is the blueprint I am using during my tenure as Archivist of the United States.
In that plan, we frankly recognized that to meet the challenges facing us as records managers and archivists today, we would need to find new approaches, new technologies, and new partnerships.
In the next few minutes, I want to tell you about some of the new approaches and partnerships we've undertaken at NARA to help us with our mission of ensuring ready access to essential evidence. I also want to talk a little about some of the lessons that we are learning as a result of our new initiatives.
To begin, electronic records represent the most strategic challenge facing all of us today. Electronic records, like records in traditional formats, are critical for the effective functioning of a democracy, and it's up to all of us who deal with government records to make sure that such records are created, preserved, and accessible for as long as needed.
The explosion of electronic records is forever changing the way we, in the records management and archival fields, do our jobs. And we cannot succeed if we do not find partners to help us solve problems, define new approaches, and help finance the technological solutions we need.
Obviously, if we are going to preserve electronic records and ensure they are retrievable and usable years into the future, new technological solutions must be developed.
Although the National Archives and Records Administration has electronic records challenges that are unique in their sheer size, complexity, and need for longevity, we don't have the resources, or frankly the clout to drive technology to satisfy our needs. The Defense Department or NASA may be able to do this, but not NARA.
So we've adopted a strategy to leverage the expertise and resources of other individuals and organizations with similar problems. We looked outside our own profession to other professional disciplines for help. And we recognized that we needed to better explain what we need and why to those who can help us.
As a result, we are now working with partners to create a digital National Archives that will make government records available to anyone, at any time, and in any place, for as long as needed.
Thanks to the partnerships we have built with the San Diego Supercomputer Center, as well as the Georgia Tech Research Institute and other government agencies including the National Science Foundation, the Defense Department, and the Patent and Trademark Office and thanks to the support we have received from the Clinton and Bush Administrations and Congress, we now have hope that we can actually develop an Electronic Records Archives, or ERA.
Overlapping and directly related to our ERA effort, is our involvement in the InterPARES Project, a major international research initiative begun in 1998. As many of you already know, the participants in this project are archival scholars, computer engineering scholars, national archival institutions, and private industry representatives from Europe, Asia, Canada, the United States and Australia.
The partners in this project are working together to develop the theoretical and methodological knowledge required for the permanent preservation of authentic records created in electronic systems.
We have viewed these collaborations as a means not only of furthering NARA's program, but also fostering further collaboration among our partners.
Our partnerships are not just different organizations working together - they also represent collaboration between different professional disciplines.
The challenge to build an information management architecture for a persistent archives is one faced by both public and private sector organizations. And, this challenge requires a new relationship between the scientific and archival/records management communities.
In fact, until very recently, engineers and archivists didn't interact very much, but technology has changed that by giving us common interests and common problems.
However, we did quickly find out that the scientific community and the archival community tend to view the problems differently. This is where communication is so critically important.
For example, Ken Thibodeau told me that one of the engineers he is working with on ERA came to him very excited that he had found mistakes in the information contained in some of our records...and that now we could correct them. Ken had to quickly explain that while engineers are concerned with the absolute accuracy of information, archivists are concerned with the authenticity of a record.
The reality is we have come a long way in just a few short years. In 1998, we told the folks at the San Diego Supercomputer Center that we needed the ability to process millions of records as quickly as possible.
At the time, they succeeded in bringing in a million e-mail messages from the Internet, processing them into a preservable format, and bringing them back in a different format. This all took less than two days on a supercomputer two years ago.
Now they can do 10 million records in a day and are exploring the feasibility of processing 100 million records a day.
And, we expect that in the near future we will have the technology to process a million messages a day from a single workstation. This means that this technology promises to be useful to many kinds of archives, libraries, government agencies, and businesses, regardless of size.
In fact, as many of you know, the NHPRC has funded a project at the Supercomputer Center so that the technology we are developing can be adapted for use in state capitols and city halls, as well as in small archives and libraries, colleges and universities, - wherever citizens need or want records or information.
The ERA project is focused on developing new approaches to preserving and providing access to electronic records.
We are working toward building an archives that can preserve any kind of electronic record indefinitely, free from dependence on any specific hardware and software, and enable individuals to retrieve the electronic records they need on computer systems now in use and those developed in the future.
As we see it now, the infrastructure of ERA will not be dependent on any specific hardware and software.
It will be modular, meaning we intend to leverage obsolescence to work for us by using plug-in components that can be replaced as technology changes. It will be scalable, meaning that the same basic architecture used by NARA can be used to manage the electronic records of a state or local archives or of an historical society.
And, ERA will also be extensible, meaning it will be able to handle additional kinds of electronic records over time. In other words, it will not be limited to specific types of records that exist today.
The Electronic Records Archives will have three integrated components.
First, we need to gain intellectual control through an accessioning process where we wrap the records with the meta-data necessary to preserve their authenticity, and ensure their accessibility.
Second, the collections will be stored in the Archival Repository and described in our Archival Research Catalog or ARC.
ARC is the on-line finding aid to all of our holdings in all formats, and will be available for use later this year, starting with descriptions of only a small percentage of our holdings, but we will be continually adding.
As a consequence of these first two steps, we will have gained intellectual control over the records.
And in the third step, the electronic records will be accessed by researchers through what we are calling the Reference Workbench.
The Workbench is a suite of tools that can be used to discover and query the many variations of records we will have accessioned. In that Workbench, some high-use records may be maintained online or near-line. Other less frequently requested records will be re-assembled from the Archival Repository for presentation when they are requested.
Three years ago, the technology and resources to build ERA simply did not exist. Now they are starting to come together. For example, I was very pleased that the President's 2002 budget included the request of $20 million additional dollars for our Electronic Records Archives. And, there appears to be, I might add, support in Congress as well.
With help from our partners, these resources would allow us, within the next year, to further define a research agenda of archival and technical questions for the development of the Electronic Records Archives and to collaborate in sponsoring research to answer these questions.
It would allow us to begin translating research results into engineering solutions that work. We will also establish a program management structure with responsibility for overseeing the ERA project and ensuring solid return on investment.
We will develop a business model for improvements in basic records management and archives functions in the electronic environment. We will begin the design and development of the information-technology infrastructure I've already described. And we will start creating ERA prototypes and pilots.
Within a year, if resources are available, we expect to launch a pilot program that will allow researchers at our facilities direct access to some of our 10,000 government databases. Ultimately these databases will also be accessible through the Internet.
Next year we will also begin taking in more than a million digital messages a year from the State Department and making those messages available electronically.
We will develop a prototype system for processing the e-mail from the administrations of Ronald Reagan and George Bush. Within five years we expect to be able to accept digital military service files from the Defense Department.
For sure, full deployment of an Electronic Records Archives will be progressive and dependent on the availability of technology and of resources.
The research we are working with now uses technology that is not yet in the marketplace. And the technology and solutions are not going to be cheap.
I am already alerting Members of Congress regarding the cost implications of long-term preservation and access to electronic records. For in terms of complexity and scope for NARA, building and implementing the Electronic Records Archives is equivalent to NASA putting a man on the moon.
But building this new, digital National Archives is not an option. For we have no alternative if citizens are going to continue to have long-term access to the records of their government.
I want to turn now to the most important partners NARA has - the Federal agencies. As you all know, responsibilities for government records begin at the front end of their life cycle, when they are being created by government officials in the course of their daily work.
Our success in ensuring ready access to essential evidence is totally dependent on NARA working in partnership with the agencies of all three branches of government.
We are now in the third year of providing Targeted Assistance to federal agencies. In developing this new approach for NARA, we drew heavily from experiences that had worked successfully at the state and local level.
Through Targeted Assistance, NARA provides expert guidance and assistance, tailored training, and detailed records review to other federal agencies. Quite frankly, in the past, our approach was primarily to evaluate an agency, tell them what they were doing wrong...and leave. And obviously, that wasn't a very effective strategy.
Now we form partnerships with agencies, at their request, to find solutions to their records management problems. Some of the services we provide include:
- Expedited review of critical schedules;
- Tailored records management training;
- Help in records disposition and transfer;
- Analysis of electronic record keeping needs...and much, much more.
With the additional budget support we have received, we have been able to add 39 Targeted Assistance staff to our base records management program. These staff members are operating at our Washington DC area headquarters and at our facilities throughout the country.
And, the agencies we have helped are responding with glowing praise and we are already seeing some good results. For example, one agency was able to expedite the scheduling and review of millions of pages of paper forms, saving them substantial storage costs.
NARA worked with another agency to schedule a large, complex electronic database, which allowed the agency to delete massive numbers of temporary data files, while preserving permanent data. We also trained an agency's regional and headquarters records coordinators in inventorying and scheduling records, which enabled them to comply with a court order.
Later today, Steve Adams from NARA's Southwest Region will speak about the partnership formed between the Region and the Department of Energy, and how they are working together to improve records management at DOE.
During this past year, we had baptism by fire in developing new approaches to working with Presidential administrations in transition. Five years ago, in our strategic plan, we identified the need to work more closely with Presidents and their appointees to smooth transitions and to avoid later difficulties by practicing good record keeping from the start.
The experiences of this past year have re-enforced the importance of this strategy to ensuring ready access to essential evidence. We have had successes and we have had some hard lessons. I'm sharing this with you today because I know many of you also have to deal with leadership transitions.
To begin, we are very happy with the progress we have made in working with the incoming Administration. Early on, I established contacts with the campaigns to make sure we could hit the ground running the day after the election.
Of course, like the rest of the country, we didn't anticipate having time to kill between the election and declaration of a new President. So, in practice, we hit the ground running in mid-December, rather than November 8th, but I can assure you this time lapse wasn't NARA's doing - even though through the Federal Register we did run the Electoral College.
We were aggressive in approaching the incoming Bush Administration. We worked hard to establish communications at all the appropriate levels. I made the initial contacts with the White House senior staff, including Judge Gonzalez, the President's General Counsel.
Deputy Archivist Lew Ballardo has continued contacts with the Counsel's office and the Director of the Office of Administration, who among other things, oversees the White House information systems and records management. The incoming Administration welcomed our help and, as a result, Mike Miller is leading a team that is essentially providing targeted assistance to the White House on both Federal and Presidential records.
This is in sharp contrast to how we handled the Bush to Clinton transition eight years ago. Because we were unprepared for a Presidential transition in the new age of electronic records, we simply stripped the hard drives from the Bush White House computers in the hours before the inauguration, leaving the new administration with inoperative computers.
We've learned a lot in the intervening years, and we were active in both transitioning out the Clinton Administration and transitioning in the Bush Administration. Months before the inauguration we began developing guidance for outgoing and incoming officials and planning for transfers of records and artifacts from the White House to Little Rock, Arkansas.
While the Clinton White House was ahead of the curve in terms of having a records management application for their electronic records and email, there was no long-range planning in developing the system.
Subsequently, there have been several glitches with the system, which have all gotten the attention of the press and various investigators. Consequently, we now have ongoing projects with the Office of Administration to recover and reformat Clinton White House email messages. And in contrast, this time around, Mike and his team are working with the White House as they develop their RMA.
Another major difference in the transition is a result of the proliferation of Web sites within the Government during the Clinton Administration. As we all know, web technology took off during this time.
A true success from our point of view was that the Clinton Library Project had a web site up and running on January 20 that included the Clinton Whitehouse.gov. Never before has a Presidential Library project been able to reach researchers with material the day the Administration leaves office.
Not so smooth, however, was the transition for federal agencies and their Web sites. As you remember, NARA asked all Federal agencies to submit to us a pre- January 20th "snapshot" of their web sites by March 20, 2001.
This meant agencies had to look at all the sub-sites of their organization, and for some agencies this was a huge task. In hindsight, we wish we had been able to start this project sooner. But dealing with the preservation of the ever-changing websites of government agencies is a new challenge both for NARA and the agencies, and continues to be a priority for us to solve as we work on web guidance you can use.
Now I want to turn to an initiative that NARA currently has underway that we believe will result in new records management approaches for NARA and our federal agency partners. As many of you know, Lew is leading a project to document current recordkeeping practices and records use environment in federal agencies.
We hope that we can use the information we learn from this initiative to analyze NARA's current records management policies, and re-design, if necessary, the scheduling and appraisal process.
I know we have been talking about this for several years and some of you in the audience probably believe that we will never get around to taking a hard look at our policies, but we have nearly finished the first phase of this project - the data collection phase. And I am convinced that in the end, our analysis of our current records management policies will benefit from this effort.
The data collection phase of this project was designed to collect data in two parts - a contractor would collect data on agency views and perceptions, and NARA staff would collect data through records systems analyses.
Last September, we awarded a contract to SRA International to collect data on the views and perceptions of Federal agencies in regard to records management. Chuck McClure and Tim Sprehe, whom many of you know, are providing their expertise to this data collection effort and analysis.
With NARA input, SRA developed a methodology for collecting needed information through Web surveys, interviews, and the use of focus groups. They talked to records officers, as well as chief information officers, General Counsels, Inspector Generals, and program staff, seeking information on their views and perceptions of records creation, maintenance, and use.
In addition to collecting this information, NARA staff in Washington DC and the regions visited eleven agencies in twenty-four locations around the country. There, they studied the business practices and the actual records generated by those processes to determine the current state of records management in those agencies.
Although the final reports are not yet in, we do have some preliminary contractor findings. While there was a wide variance in agency response, results showed that in order to understand the recordkeeping views and practices of an agency, we must first understand situational factors such as agency culture.
There is, for example, a big difference in the culture of a Defense organization and that of a Social Services agency, and the records management practices of each organization will reflect their culture. Factors such as work process, business needs, and administrative and financial commitment to records management also affected the view of recordkeeping within an agency.
Now that the data collection phase of this project is essentially completed, our next steps will be to compare and contrast the contractor findings, once they complete their final report, with NARA findings. We will look at agency situational factors and their effect on records management. We will also examine NARA's situational factors and their effect on government records management.
Then we will use this data to begin reviewing NARA's records management policies, guidance, and processes. And we do plan on sharing the lessons we learn from this data collection with you and with others who may be interested once the analysis is complete.
One of the commitments we made in the NARA strategic plan was to test and implement effective records management within NARA itself. If we can't effectively implement our own regulations and guidance internally, how can we expect other federal agencies to have respect for that guidance?
Right now we are in the middle of piloting a records management application at NARA. In addition to helping us with our own records management, we expect this practical experience will help us give better guidance and assistance to other federal agencies trying to improve the management of their electronic records.
In an eight-month test period, we are exploring the capabilities of new auto-filing technology meant to eliminate human intervention in filing records. The software we are testing is TrueArc's Foremost Enterprise 2, an off-the-shelf product. It's currently in use by thirty people in seven different NARA offices.
And, I know that similar work is going on in other places. For example, the State Archives and Office of Management and Budget in Michigan is working in this area with NHPRC support. You can follow their results on their website, and the Michigan State Archives is moderating a roundtable discussion on electronic records tomorrow afternoon.
For us, it's too early to draw conclusions yet, but we have formed some first impressions. One of our early "lessons learned" was that RMA testing requires significant planning - from policy...to file plans...to access... even for a small test like ours.
We also learned that it is critical that RMA systems be able to integrate with existing systems, both network and record creating applications like word processing software and e-mail. Finally, in order for RMA software to be effective, it has to be easy for everyone to use - general users and records managers alike.
At the end of this pilot, we hope to have sufficient data to decide whether and how to proceed with installation of RMA technology in parts or all of NARA. We hope that the lessons we learn from this test will help us provide better guidance and assistance to other agencies. And we'd be very interested, as well, in hearing about your experiences with installing RMA technology.
To this point, I've spent a lot of time talking about new approaches and partnerships in the areas of electronic records and records management. Now, I want to take a few moments to touch on another new approach we are undertaking to implement our Strategic Plan, as well as some general lessons we have learned.
For several years now, we have been re-engineering the way we do business at the National Military Personnel Records Center in St. Louis, the Center that houses and services military personnel records. These documents are critical for veterans to receive medical care, education benefits, home loans, and burial privileges in our national cemeteries, so we get many, many requests for copies of military records.
In undertaking this re-engineering effort, our objectives are to: improve quality of service to the veteran...reduce the cycle time for complete and accurate response...and reduce the cost of responses.
To date, we have focused on bringing the Center, which houses more than two million cubic feet of records, into the 21st century. We are introducing computers into the workplace, preparing our workforce of four hundred to use these new tools, and streamlining the process we use to search for records and prepare responses.
More than 80 percent of our workforce are now trained in our new processes and are beginning to demonstrate the improvements we expect.
With that said, I think a couple of the lessons we've learned might be worth sharing. Yes, business process re-engineering efforts are potentially helpful for all of us in many areas. But such efforts need to be well thought out from the beginning. And you have to have an understanding up front that implementation will not be easy.
Change is difficult, particularly when you are talking about changing the culture of an organization.
Another lesson learned is to be sure to include key stakeholders, that is, your customers, in the re-engineering effort. Such efforts can strengthen your partnership with your customer agencies. And including your customers can yield better results. Not including your customer agencies can weaken your relationships.
The bottom line is, much of your success will depend on how well you communicate your intentions - both internally and externally. And Jean, I know what you're thinking - I hope he remembers this lesson when he gets around to re-engineering records management processes.
In these remarks, I have touched on several of the new approaches we are undertaking with our partners in ensuring the creation, maintenance, preservation and use of government records for as long as needed. We are, of course, implementing new approaches on many other fronts.
For instance, we have implemented a reimbursable records center program, and we are changing the way we describe our records through the implementation of new data standards and the Archival Research Catalog.
We are working to build a new Regional Archives that includes a partnership with the Georgia Division of Archives and History, and Clayton State University. And Jim McSweeney of NARA's Southeast Region will chair a discussion that includes this proposed partnership tomorrow morning at 8:30.
We are also adopting new approaches to our public programs, including the National Archives Experience, traveling exhibits, and a more robust web site. Slated for debut in 2003, the National Archives Experience will combine real records with state-of-the-art technology to create new exhibits, programs and interactive experiences which illustrate for visitors what NARA does, and the records we hold. And for the first time ever, starting this Fall, our traveling exhibit of American Originals will bring some of our greatest treasures to cities around the country.
In order to make the National Archives Experience a reality, we are working closely with those who can help us in the private sector through a re-invigorated National Archives Foundation.
Also, you can find more about the initiatives I've talked about today as well as many others by visiting our Web site at www.archives.gov.
Before I close, I want to mention a subject I know is of great interest to many of you, and that is the business of grant making.
In my capacity as Chairman of the National Historic Publications and Records Commission, or NHPRC, I had the pleasure to welcome Roy Turnbaugh, NAGARA's most recent past President, as NAGARA's new representative to the NHPRC, succeeding Howard Lowell.
Roy joined the Commission at an interesting time! Fiscal Year 2001 has turned out to be a remarkable year for the NHPRC. Applicants requested a record amount of money - nearly $18 million, which is three times our annual appropriation. And, this year the Commission considered proposals from seven states - another record -- for re-grant projects.
Even more importantly, the Commission noticed a surge in the quality of the proposals -- in the amount of ingenuity and thought, that had gone into preparing both the projects, and the proposals requesting funding for them.
To say that the competition was fierce would be an understatement, especially because NHPRC's appropriations remained at roughly the level as the previous year. Because our appropriations fell so far short of the demand, the NHPRC had to turn away some excellent projects. And, the Commission was able to provide re-grant projects with only a dollar-for-dollar match, rather than at the higher levels we've managed to award in previous years.
As far as the FY 2002 budget is concerned, we do have some good news. This year has seen an enormous outpouring of support for the NHPRC as the process of determining the budget has evolved.
And although the process is far from over, it appears that we will have an excellent chance of increasing the funding for the coming fiscal year. Given that the budget process has been compared to sausage making, some of the details may get ugly, but in the end, we hope to be in better shape.
Now to wrap this up, I want to reiterate a few of the general lessons we've learned over the last couple of years as far as implementing new approaches, new technologies, and forming new partnerships at NARA.
First, it's almost always easier to just accept the status quo. But I believe that in order to do what's right, to meet the needs of our citizens, we have to change.
Second, change is hard work. You need to plan, anticipate consequences, be willing to take risks and be willing to adapt when an approach isn't working. Even more difficult, you need to get buy-in for change both internally among employees, and externally among customers and stakeholders. For us that includes the Administration and Congress.
And third, the key to getting any of this done is communication. It is absolutely necessary to tell people who can help you what you want to do, why you want to do it, and what the intended outcome will be. That is why I work hard to communicate with the Administration and Congress about the work we do.
I know from experience that when we can explain to others what we do as archivists and why, people are eager to help. But believe me, communication is hard work. It requires defining your message and staying on message. And we can't just talk to ourselves.
Through communication and the solicitation of input, you will make people a part of the change process and give them a personal stake in the outcome.
In closing, I'd like to share with you an experience I had a couple of weeks ago on July 4th. As you may know, we throw quite a party on the steps of the National Archives Building in Washington D.C. every Fourth of July.
We celebrate with military bands playing patriotic music, historical re-enactors portraying our Founding Fathers, and a dramatic reading of the Declaration of Independence. And of course people come to see the actual Declaration itself.
This year was even more special than usual. Because, not only did we celebrate the 225th birthday of our great nation, and the 225th Anniversary of the Declaration of Independence, but this was also the last day that the Declaration of Independence, the Constitution, and the Bill of Rights were on public display until 2003.
As you know, conservators have found deterioration in the glass encasements that have held these Charters of Freedom since 1952. There doesn't appear to be damage to the actual documents, but we are taking the Charters out of the old encasements, doing any necessary conservation work, and re-encasing them in new, state-of-the-art encasements that will better protect these priceless pieces of history for generations to come.
At the same time as this work is being done on the Charters, we are also doing the first, full-scale renovation ever of the National Archives Building itself.
Now we figured that lots of people would want to see the Charters on July 4th. But, we had no idea that more than 4,000 people from all over the U.S. would come and stand in the heat and the rain to get a glimpse of these documents. People were camped out on our steps when we opened, and the crowd grew as the day went on.
As I talked to some of our visitors in line, the majority of them from out of town, it was clear that they had made a stop at the National Archives a high priority on their trip to Washington, DC. In a city where there are plenty of things to do, especially on July 4th, these people had no reservations about standing in line for four to five hours to see the Charters of Freedom, and they had traveled hundreds and thousands of miles to do so.
Talking to these visitors re-enforced for me once again just how important our job in public archives is. We are not just doing a daily routine job. We are doing something essential for the functioning of our country's democracy.
A society whose records are closed cannot be open. A people who cannot document their rights cannot exercise them. A nation without access to its history cannot analyze itself. And a government whose records are lost cannot accountably govern.
This is what motivates all of us.