Appraisal Policy of the National Archives

This policy sets out the strategic framework, objectives, and guidelines that the National Archives and Records Administration (NARA) uses to determine whether Federal records have archival value. It also provides more specific guidelines for appraising certain categories of records.

Records appraisal is not a rote exercise. It requires informed judgments, knowledge of and sensitivity to researchers' interests, recognition of resource considerations, and a willingness to acknowledge and understand comments and suggestions from diverse perspectives. This document facilitates the appraisal process by providing a consistent framework for appraisal decision making.

The authority and responsibility of the Archivist of the United States to determine the retention and disposition of Federal records stems from Chapters 21, 29, and 33 (3302, 3303a) of the Federal Records Act, 44 United States Code.

Chapters 21 and 29 of the Federal Records Act, 44 United States Code also provide for the transfer of records with archival value to NARA's legal custody when they are no longer needed for the conduct of agency business.

All Federal governmental entities create and maintain records in the conduct of official business. However, this policy applies only to records subject to the Federal Records Act (FRA)--records of all executive branch agencies, the United States District and Circuit Courts, and Legislative branch agencies. Records created by the President and Presidential entities, the Senate, the House of Representatives, the Architect of the Capitol, and the Supreme Court are not subject to the FRA. The appraisal of records created by these entities is not covered by this policy.

Certain agencies such as the Federal Deposit Insurance Corporation are not wholly governmental and are not subject to the FRA. However, these agencies typically follow FRA regulations and are covered by this policy.

List of select entities not covered by the FRA

As employed in this policy, the terms below are defined as follows:

1. Appraisal -- the process of determining the value and thus the final disposition of Federal records, making them either temporary or permanent.

2. Archival value -- the enduring historical or other value, as determined by the Archivist of the United States, that warrants NARA's continued preservation of records beyond the period required to transact the business of their originating agency or its successor in function. Records determined to have archival value are designated on records disposition schedules as "permanent records."

3. Originating agency -- the Federal agency in which records are created, received, and accumulated in the conduct of business.

4. Permanent records -- records appraised by NARA as having archival value.

5. Sampling -- the selection of file units or items from a body of records made in such a way that, taken together, the items selected are representative of the whole.

6. Temporary records --records approved by NARA for destruction, either immediately or after a specified event or period of time.

NARA does not appraise records in isolation. NARA works with other interested parties to ensure that "essential evidence is created, identified, appropriately scheduled, and managed for as long as needed." The Archivist of the United States has the statutory responsibility to decide how long records must be retained and which records have archival value and thus are to be retained permanently. In making appraisal decisions, the Archivist considers the recommendations of NARA staff, and seeks and considers the views expressed by originating agencies and the public.

NARA's mission "...is to provide public access to Federal Government records in our custody and control. Public access to government records strengthens democracy by allowing Americans to claim their rights of citizenship, hold their government accountable, and understand their history so they can participate more effectively in their government." In order for NARA to carry out its mission, it must collect essential evidence from Federal agencies.

Essential evidence is comprised of those records that document the:

Records that document the rights of citizens enable them to establish their identities, protect their rights, and claim their entitlements. Records that document actions of Federal officials that enable them to explain past decisions, form future policy, and be accountable for consequences. Records that document the national experience provide the means for evaluating the effects of Federal actions on the nation and for understanding its history, science, and culture, including the man-made and natural environment.

Not all records that constitute essential evidence possess archival value. Rather, NARA authorizes agencies to destroy most of these records when they are no longer needed to meet agency business needs. Records that are appropriate for permanent retention are discussed below.

NARA uses the essential evidence categories listed above as the beginning point for appraisal. These categories provide an overall high-level framework for the analysis of records to determine whether or not they are permanent. Note that the three categories are not mutually exclusive: some records that warrant permanent retention may fit into multiple categories, while others may relate to a single category.

1. Records documenting the rights of citizens.  Many Federal records provide evidence of the legal status, rights, and obligations of individuals, groups, organizations, and governmental bodies. In most cases, the legal rights implications of records eventually expire. In a few instances, however, the importance of records for protecting legal rights endures despite the passage of time. Records falling into this category are preserved permanently.

2. Records documenting the actions of Federal officials.  Most Federal records document the actions of the Government. NARA seeks to retain that portion containing significant documentation of Government activities and essential to understanding and evaluating Federal actions. For example, NARA retains permanently those records that document the basic organizational structure of Federal agencies and organizational changes over time, policies and procedures that pertain to an agency's core mission, and key agency decisions and actions.

3. Records documenting the national experience.  Some records document the impact of Federal actions on individuals, communities, or the natural and man-made environment. The Government also creates and acquires much information about people, places, material objects, and scientific phenomena, as well as about social conditions, political and economic activities, and events in the United States and other countries. Much of this information does not have archival value. However, some is essential to understanding the role of the Federal Government and the history of our nation, its people, and the environment.

Within the high-level strategic framework of rights, actions of Federal officials, and the national experience outlined above, NARA will identify for permanent retention records that:

1. Retain their importance for documenting legal status, rights and obligations of individuals, groups, organizations, and governmental bodies despite the passage of time;

2. Provide evidence of significant policy formulation and business processes of the Federal Government;

3. Provide evidence of our Government's conduct of foreign relations and national defense;

4. Provide evidence of Federal deliberations, decisions, and actions relating to major social, economic, and environmental issues;

5. Provide evidence of the significant effects of Federal programs and actions on individuals, communities, and the natural and man-made environment;

6. Contribute substantially to knowledge and understanding of the people and communities of our nation.

In determining which records support its appraisal objectives and thus warrant permanent retention, NARA uses the general guidelines outlined in Appendix 1. NARA has also developed specific guidelines for selected kinds of records (see Appendix 2). In the future, NARA may develop specific guidelines for other kinds of records.

NARA will reappraise records when there is compelling evidence that earlier appraisal decisions require review. In such circumstances, NARA will seek Federal agency and public involvement in the reappraisal process.

NARA will review this policy as necessary in consultation with Federal agencies, research communities, and other interested parties.

Personal data records contain information about an individual and may also include information about his or her family members. Included are such records as the personnel folders of Federal employees and members of the armed services; the files that are accumulated in connection with determining an individual's eligibility for Federal Government benefits, such as a pension, medical care, or mortgage guaranty; and the records that document the immigration to the United States of the foreign born or their application for legal residence or citizenship.

In appraising personal data records, it is necessary to take into account the following:

• Size and nature of the population

  Some personal data records cover nearly all of the American population. Records of this sort are more likely to warrant permanent retention than records containing information on only a small percentage of the population. However, even if the total number of people represented in a body of personal data records is relatively low, the records may still warrant permanent retention if they contain information concerning a large percentage of a subgroup of the American population (e.g., an ethnic or racial group or residents of a specific region).

• Nature of the information

  Researchers who use personal data records have traditionally had a high interest in the following types of information concerning individuals. Records that are rich in the kinds of information outlined below are more likely to warrant permanent retention than records that contain only a small number of these elements:

  • previously used names
  • date and place of birth, place(s)of residence;
  • date, place, and cause of death;
  • if foreign-born, date and place of arrival in the US, and if naturalized, date and place of naturalization;
  • names of parents;
  • dates and places of parents' births and deaths;
  • name(s) of spouse(s), date(s) and place(s) of marriage;
  • names of children;
  • date and place of children's birth;
  • education level;
  • educational institutions attended and the dates;
  • occupation;
  • property ownership;
  • names of employers and work location(s);
  • military service, including branch of service, dates of service, and rank;
  • identification photographs of individuals.

  Researchers appear to be increasingly interested in medical information as well. NOTE: It will be necessary to monitor possible effects of the Health Insurance Portability and Accountability Act (HIPAA) on access to medical information.

The physical sciences encompass any of the sciences that analyze the nature and properties of energy and nonliving matter (thereby excluding the biological sciences and social sciences). The physical sciences include chemistry, physics, the space sciences (astronomy, planetary science, and space physics), and a number of earth sciences (including geology, hydrology, meteorology, oceanography, and soil science). These disciplines often are overlapping, as illustrated by space physics, chemical physics, physical chemistry, and geophysics.

While some of the physical sciences (including chemistry and materials sciences) produce experimental data, many of them (including the space sciences and earth sciences) collect data on the observation of natural phenomena. Observational data characteristically are unique and non-repeatable and can be generated in large volumes.

Federal agencies follow a standard work process for developing and using observational data. The steps in this process are collecting, processing, and interpreting data; preparing related products; and documenting, storing, cataloging, and managing data and products to make them available for continuing use.

Raw observational data are collected by means of human perception or measurement (e.g., field notes) or, more commonly, by sensor or other instrument. In situ data collection is carried out in direct contact with the phenomenon under study. Examples of in situ sensors are stream flow gauges, seismic gauges, and sensor packages mounted on buoys. By contrast, remote sensing involves use of instruments, such as cameras or digital sensors, not in close contact with the studied phenomenon.

Data may be collected in any location - the earth, the atmosphere, or in space. In recent decades, data usually have been collected in digital formats, although in the past they commonly were collected in analog formats, such as paper or photographs, a portion of which may not have been converted to digital format.

After collecting raw data, scientists process them at different levels of complexity. Each processing level provides added value through such actions as summarizing and interpreting the raw data and synthesizing new data. Production of processed data may involve incorporating two or more sources, raw or processed, to generate yet another data product, for example, the merging of observational and elevation data to produce three-dimensional representations. Processed data frequently are subjected to scientific peer review. A dataset consists of a body of related data (raw and/or processed), for example, data gathered from a particular satellite sensor for a given time period and geographic region.

Data originators are responsible for documenting their data, including preparation of metadata (i.e., the auxiliary information needed to understand and use data properly). Data originators also are generally responsible for submitting their data and metadata to a data center for long-term storage. Such centers catalog the data and make them available for continuing use, often by posting the data or linkages to the data on appropriate websites.

Primary use of observational data is usually by scientists involved in collecting and initially processing the data. Secondary use of data is by others than those involved in collection and initial processing. Secondary users include scientists and non-scientists. Scientists often exploit the data in new ways and, in so doing, may create additional processed datasets. Non-scientists use data in a variety of ways. For example, farmers use climatological data for decisions on crop selection, and engineers use seismic data in designing critical structures such as nuclear power plants.

Primary or secondary use of data can result in preparation and dissemination of related products - either processed data or non-data products. Examples of non-data products are (1) hurricane warnings and weather forecasts prepared from meteorological data, (2) navigation charts and maps prepared from oceanographic data, and (3) maps of flood-prone areas prepared from hydrologic data. Certain products are produced by running data through a model (software using mathematical formulas) used to simulate natural phenomena. Such models may generate, for example, products used to forecast the weather or the progress of a wildfire.

Agencies holding large quantities of observational data include the National Oceanic and Atmospheric Administration (NOAA), National Aeronautics and Space Administration (NASA), and U.S. Geological Survey (USGS). These agencies both collect data themselves and receive data submitted by outside entities. The agencies maintain in situ and remotely sensed data pertaining primarily to space and the global environment, including the atmosphere, ocean, and land surface. These data are stored, cataloged, and distributed by multiple data centers. The observations not only support real-time monitoring and forecasting but over time provide a historical record.

Appraisal considerations:

• Appraisal of observational data is challenging because appraisers frequently must deal with a high volume of data and become familiar with a diversity of data structures and platforms, metadata requirements, and media types including paper, audio-visual, and electronic.
   
• Observational data are likely to be appraised as either long-term temporary or permanent. Unlike laboratory experimental data, observational data typically document phenomena that can never be repeated. Observational data establish a baseline to help determine future rates of change and frequency of occurrence of unusual events. Moreover, observational data frequently can be processed and used in novel ways, for example, to verify new scientific concepts.
   
• Observational data covering a long time period tend to have more value because they enable long-term patterns to be identified and thereby increase confidence in the reliability of data and the conclusions drawn from them. In addition, data are more likely to be appraised as permanent if they not only can be used for scientific purposes but also for legal, commercial, educational, engineering, resource management, or other purposes.
   
• Appraisal decisions should take into account that the uses of data vary according to the level of processing. Raw or minimally processed data are more difficult for anyone except the primary user(s) to understand and use but are essential for conducting a reanalysis, such as to verify findings or support a new hypothesis. With each higher level of processing, data generally become easier to use but less subject to reanalysis. To facilitate future reanalysis, it is usually appropriate to preserve processed data at the lowest level of processing compatible with effective use. Processed data are more likely to have long-term value if they would be costly to recreate from the raw data. It may be warranted to appraise as permanent both a raw version and one or more processed versions of certain data.
   
• To be appropriate for long-term temporary or permanent retention, observational data should possess authenticity, reliability, integrity, and usability (as defined in ISO 15489-1). Intellectual linkage with the related metadata is essential, and linkage with program or project management records is desirable to provide additional context for the data.
   
• Additional factors favoring long-term or permanent retention are uniqueness, completeness, and quality of observational data; quality and completeness of metadata; and availability of appropriate technology to enable access. In general, data are more likely to have these attributes if the data have successfully undergone the scientific peer review process. Distributed storage of data does not diminish their archival value, provided that a central catalog is maintained to ensure access.
   
• Metadata should include information such as purpose and time period of data collection; location of collection site; methods and instrumentation used in collection; units of measurement, acceptable values, and error tolerance; data aggregation methods; processing history; and quality assessment. The types of metadata required vary with the nature of the data and their likely future uses. It is preferable for metadata, whenever possible, to conform to standards issued by such broad-based organizations as the Federal Geographic Data Committee (FGDC) and the International Organization for Standardization (ISO).
   
• It is appropriate for many observational data of long-term temporary or permanent value to be maintained on a continuing basis by a scientific data center that possesses the necessary expertise to ensure preservation and access. Data centers often issue policies on collecting and data sharing. NARA appraisers will benefit from reviewing these policies and identifying the standards used to determine which data have value for long-term scientific purposes. Similarly, appraisers should seek the advice of scientists when assessing many of the above-mentioned attributes of data and metadata.
   
• Many data series now collected in electronic format were formerly created and maintained in other formats such as paper or photographs. Agencies may still maintain older data in such formats for use in conjunction with the related electronic data. Appraisers should extend their review of electronic systems to include any related data in other formats, as these older data may add to the usefulness of the electronic data if they are still in a usable format. All formats should be considered during the appraisal.
   
• Models are rarely if ever appropriate for appraisal as permanent. Models generally are in a continuous state of development, so that one model will replace another as soon as its value is demonstrated (e.g., improved accuracy in forecasting hydrologic conditions). Data products that result from running data through a model may warrant permanent retention if they relate to a significant phenomenon or occurrence.

This section covers environmental health and safety (EH&S) records relating to Federal actions in regard to hazardous substances and their potential harmful effects on humans and the environment. The types of hazardous substances are diverse and include radioactive substances, asbestos, certain metals such as lead and mercury, synthetic chemicals (such as PCBs), petroleum products, and munitions. Not covered here are records relating to the following types of environmental health and safety hazards: communicable disease agents, fires and natural disasters, and hazards from use of tools and equipment.

Federal government agencies undertake several broad functions that generate a wide variety of EH&S records. The first is production, use, storage, transportation, and disposal of hazardous substances. The second consists of planning and routine monitoring of actions involving potential release of hazardous materials into the environment and exposure of people to them. Planning includes preparation of environmental impact statements (EISs) and similar documents.

A third EH&S function involves emergency response, monitoring, and remediation of spills and other inadvertent or unauthorized releases. These actions may involve either short-term, prompt removal of hazardous substances or long-term remedial response. Monitoring may be included in either the second or third above-mentioned EH&S function and generally consists of data collection, analysis, and documentation of releases and exposure involving hazardous substances. A fourth EH&S function relates to conducting litigation and responding to claims relating to potential harm caused by hazardous substances to individuals and the environment. The third and fourth EH&S functions may relate to substances not known to be hazardous at the time of production, use, or disposal.

Numerous environmental laws establish requirements, including those for recordkeeping, in regard to hazardous substances. Many of these laws require Federal agencies to comply with general environmental standards for production, use, and disposal of hazardous substances, as well as for environmental planning and routine monitoring. An example is the National Environmental Policy Act (NEPA), which mandates that agencies formally analyze the environmental consequences of a proposed major action and consider possible alternatives.

Environmental laws give certain agencies jurisdiction over emergency response and cleanup involving significant releases, spills, and waste sites, including those not caused by the Federal government. These laws set strict requirements for financial oversight and involvement of the public in regard to response and cleanup.

The most important of such laws are the Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA) and its reauthorizing legislation, the Superfund Amendments and Reauthorization Act (SARA). CERCLA and SARA give the Environmental Protection Agency (EPA) overall authority for most major environmental remediation and establish the Superfund to support this remediation. Under the aegis of these and similar laws, national defense agencies perform cleanup of former military sites in conformance with the Defense Base Closure and Realignment Act of 1990 (BRAC), Formerly Used Defense Sites (FUDS) program, and similar programs.

In addition, the Oil Pollution Act (OPA) gives certain agencies, such as the National Oceanic and Atmospheric Administration, authority for response and remediation for major oil spills. Lastly, a number of Federal laws provide for financial compensation to certain workers and other individuals exposed to hazardous substances.

Appraisal considerations:

This section discusses research and development (R&D) records relating to the planning and execution of basic and applied research in engineering and the physical and natural sciences. Basic research seeks to generate new knowledge, and applied research uses the results of basic research and applies them to the design, development, and testing of new products and processes. Agency R&D programs tend to be large in scale, expending hundreds of millions of dollars annually and generating voluminous records. Records pertain to such research fields as biology, chemistry, medicine, physics, materials science, aerospace technology, weapons development, computer science, energy development, and environmental protection.

Appraisal of the records requires an understanding of the entire R&D business process, including the project/product lifecycle and use of outside entities for review or support. Most R&D conducted by or for the Federal government follows a standard workflow based on the scientific method. The basic steps include formulating a hypothesis or statement of need, obtaining approval and/or funding, designing and conducting experiments and analyzing results, and disseminating findings. Records created and accumulated by these steps can be separated into the following categories: program management records covering the processes of formulation, selection, and funding; project records covering design, collection, analysis, and reporting; and dissemination of findings. Types of records found under these categories include planning records, project files, procurement and financial records, laboratory notebooks, research data, and technical reports and similar publications.

The status and availability of records produced by a project often depend upon the funding arrangement. For projects funded by contracts, records specified in the contract as deliverables generally are Federal records and, in conformance with the contract requirements, may be maintained by either the contractor or the funding agency. By contrast, the primary records of grant-funded projects usually are not considered to be Federal records and are maintained by the grantee. Recordkeeping for collaborative projects is affected by the diversity of funding sources and institutions (including non-Federal institutions) involved. Effective appraisal of these records requires a determination of which institutions have responsibility for the records and their disposition.

Appraisal considerations:

• Program management records that document the planning, policies, and priorities of research programs usually are appraised as permanent. Such records may be maintained by offices with agency-wide R&D responsibilities, by individual divisions and laboratories, and by scientific and technical advisory bodies. Using the guidance in Appendix 1 - General Appraisal Guidelines, appraisers should determine for which office(s) within a given agency these records should be scheduled as permanent.
   
• Technical reports, conference proceedings, and similar publications that disseminate the findings, methodology, and conclusions of projects are usually appraised as permanent and are often maintained centrally by an agency component responsible for their collection, management, and distribution. Review of a cross-section of such publications can help determine the subject matter and scope of R&D projects and thereby prove useful in assessing the value of other project-related records and data.
   
• Project files may include such records as statements of work, progress reports, briefing papers and presentations, specifications and drawings, laboratory notebooks, research data, environmental and safety information, and technical reports. Agencies may maintain such files as case files or as separate series. The value of project files varies across R&D programs, based on such factors as the files' organization and content, nature and scope of the research, and extent to which separately maintained series of records such as annual reports, planning records and technical reports provide sufficient documentation of the projects.
   
• Because many R&D projects have a very limited focus and project records often are voluminous, a very strong justification is needed to appraise all of an agency's project files as permanent. If selection criteria are to be applied to identify files for permanent retention, the creating agency or organization must devise a practical arrangement for applying the criteria to the records and agree to implement it, because it is the creating agency which possesses the expertise and resources to evaluate the files individually. For overall guidance on when to apply selection criteria, see  
• Contracting, procurement and other fiscal records generally are appraised as temporary when readily segregable from other project records.
   
• Laboratory notebooks may be informally maintained as part of project files, or as a separate series, issued formally and strictly controlled to protect intellectual property and patent rights. Notebooks issued and managed formally are more likely to be appraised as having long-term temporary value or possibly permanent value.
   
• Research data created by R&D projects may be unprocessed (raw) or processed (compiled or analyzed) at different levels. Raw data are generated by an experiment, whereas processed data consist of raw data manipulated to help identify patterns in the data. It is very difficult to generalize about the value of processed data as opposed to raw data, as both have their own significance for the research process. Research data most often are electronic but also may be in another format such as paper or photographs. To help establish the value of still photographs and moving images, whether conventional or digital, maintained separately from other project records, appraisers need to determine whether the agency maintains intellectual linkage between these and related project records.
   
• Research data commonly have short-term value when they are narrow in scope and can be replicated by a new experiment if necessary. Data may have long-term temporary or even permanent value when they are extremely difficult or impossible to replicate and are potentially useful for such purposes as permitting an important experiment to be reviewed and validated, supporting new scientific research, or providing a legal basis for health-related claims.
   
• For data to be valuable over the long term, they should be unique, complete, valid, and accompanied by appropriate metadata. Moreover, data are usually more valuable when they can be linked to records describing the research protocol and modes of analysis. In considering these qualities of data, appraisers should consult with the relevant scientific experts. Data with long-term research value often are most appropriately maintained by the R&D agencies which created them because the creating agencies usually possess the scientific expertise essential for providing effective access to the data.
   
• R&D agencies, particularly those involved in environmental or health research, may create tissue samples, slides, and specimens which are treated by researchers as project records and preserved by the agency for long periods at substantial expense. Although NARA generally does not consider such materials to meet the definition of Federal records, agencies nonetheless need to manage them properly because of their importance to R&D programs and potential for long-term use.