Assessing Freedom of Information Act Compliance through the 2020 National Archives and Records Administration’s Records Management Self-Assessment
Published September 28, 2021 | Download the assessment
Executive Summary
Since 2016, the Office of Government Information Services (OGIS) has collaborated with the Office of the Chief Records Officer (CRO) for the U.S. Government to gather government-wide information about Freedom of Information Act (FOIA) administration. Data collected through FOIA questions included in the 2020 Records Management Self-Assessment (RMSA) complement the observations we make as the federal FOIA Ombudsman, working to improve the FOIA process for all.
The 2020 RMSA survey—administered to agency records officers from January 19, 2021, to March 19, 2021—included seven questions regarding FOIA administration. Key results highlighted in this report include:
- Nearly half of all respondents (49 percent) reported that the COVID-19 pandemic disrupted their agency's ability to respond to FOIA requests.
A majority of respondents (80 percent) whose FOIA programs were disrupted reported that their agency’s paper records were inaccessible due to office closures while nearly half (46 percent) reported that agency staff were not available to search for records. - A majority of respondents (72 percent) reported that their agencies worked directly with requesters to tailor their requests for most efficient processing during the COVID-19 pandemic. Fifty-six percent said that they included information about anticipated delays in requester communications, including acknowledgement letters; 52 percent assessed their technology to ensure the most efficient administration of FOIA; and 50 percent reported that their agencies posted notices on their FOIA websites informing requesters of the most efficient ways to make requests.
- A majority of respondents (72 percent) reported that their agencies use e-discovery tools to search for records when responding to FOIA and/or legal discovery. (Relates to FOIA Advisory Committee Recommendations Nos. 2020-11 and 2020-13)
- Of the agencies that reported using e-discovery tools, a significant majority (91 percent) reported that their agencies use e-discovery tools for FOIA responses involving requests for email records. A majority reported that their agencies use such tools for lawsuit-related requests (69 percent), legal discovery or third-party subpoena requests (62 percent), managing legal holds (58 percent), and FOIA responses not involving requests for email records (54 percent).
- Of the respondents who reported that their agencies do not use e-discovery tools to search for records, roughly half reported that such tools are not available at their agencies.
- A majority of respondents (51 percent) reported that their Agency Records Officer and Chief FOIA Officer work together on information technology (IT) requirements that benefit both programs, and half (50 percent) reported that their training programs address the importance and relationship between FOIA and records management.
Background
OGIS’s partnership with the CRO to collect government-wide information related to FOIA compliance began with the 2016 RMSA survey, an annual agency self-assessment and reporting tool developed and managed by the CRO to determine whether agencies are complying with statutory and regulatory records management requirements. The CRO has issued the RMSA since 2009. OGIS’s FOIA questions fit naturally with the RMSA data collection because a strong records management program—which allows agencies to find responsive records—is essential to a successful FOIA program.
In working with the CRO, OGIS has leveraged investments by the National Archives and Records Administration (NARA) in survey technology and the CRO’s expertise in collecting self-reported compliance information. The RMSA’s high response rate in the last few years has helped OGIS better understand FOIA administration across the government, and complements the observations we make through our other activities, including providing dispute resolution and ombuds services to requesters and agencies, assessing FOIA compliance, and leading the federal FOIA Advisory Committee and the Chief FOIA Officers Council. In Calendar Year (CY) 2017, CY 2018 and CY 2019, the RMSA response rates were 99 percent, 98 percent, and 96 percent, respectively. In CY 2020, the response rate was 92 percent, a drop likely attributable to the COVID-19 pandemic; in CY 2021, the response rate was 98 percent.
The 2020 RMSA included seven questions relating to FOIA. Three questions related to the COVID-19 pandemic impact on FOIA operations. Three questions addressed the use of electronic discovery (e-discovery) tools by FOIA and agency staff. One question covered FOIA and records management, specifically the relationship between Agency Records Officers and the Chief FOIA Officer.
The reporting period for the 2020 RMSA began January 19, 2021, and ended on March 19, 2021. A total of 266 RMSA respondents from all Cabinet-level departments, departmental components and independent agencies answered the FOIA questions. Not all 266 respondents answered all seven FOIA questions however.
A complete list of the seven 2020 RMSA questions, each of which is discussed in detail below, is available below in the Methodology section.
Discussion of RMSA Results
COVID-19
On March 17, 2020, the Office of Personnel Management (OPM) ordered maximum telework flexibilities across the federal government in response to the COVID-19 pandemic. The move to a full-time work-from-home environment affected FOIA processing governmentwide, particularly at agencies that were not telework-ready, agencies that work with classified records and systems for processing records, and agencies with largely paper-based records.
Of the 263 respondents who answered Question 52 “Has the COVID-19 pandemic disrupted your agency's ability to respond to FOIA requests?”, 49 percent answered “Yes,” 48 percent answered “No,” and 3 percent answered “Do not know.”
In response to Question 53, “Which of the following explains why FOIA has been impacted? (Choose all that apply),” the majority of respondents said that paper records were inaccessible due to office closures (80 percent) and agency staff were not available to conduct searches (46 percent). Thirty-six percent answered, “Other, please be specific;” 12 percent said that their agency’s FOIA case processing system was not available by remote access; and 9 percent said that electronic records were not accessible remotely.
As shown in Table 1, the comments of the RMSA respondents who answered, “Other, please be specific,” centered on four themes: the ability of staff to access records, agency staffing, resources, and the technology agencies use to respond to requests.
Table 1: Selected Data from 2020 RMSA “Other, please explain,” Responses to Question 53 “Which of the following explains why FOIA has been impacted? (Choose all that apply)”
Theme | Agency Comments |
---|---|
Access Issues |
|
Personnel |
|
Resources |
|
Technology |
|
In response to Question 54, “Which of the following actions did your agency's FOIA program take in response to the COVID-19 pandemic? (Choose all that apply),” the majority of respondents reported that their agencies worked directly with requesters to tailor their request for more efficient processing (72 percent). Fifty-six percent reported including information about any anticipated delays in requester communications, including acknowledgment letters. Half of all respondents reported that their agencies posted a notice on the FOIA website informing requesters of the most efficient way to make a request, and 43 percent posted a notice on the FOIA website informing requesters of any anticipated delays. Twenty percent of respondents answered “other, please explain” in response to Question 54.
Table 2: 2020 RMSA Responses to Question 54, “Which of the following actions did your agency's FOIA program take in response to the COVID-19 pandemic? (Choose all that apply) (DOJ, Guidance for Agency FOIA Administration in Light of COVID-19 Impacts, https://www.justice.gov/oip/guidance-agency-foia-administration-light-covid-19-impacts, updated May 28, 2020.)”
Answer Options | Count per answer option |
Percentage to total responses |
---|---|---|
Worked directly with requesters to tailor their requests for most efficient processing |
187 | 72% |
Included information about any anticipated delays in requester communication, including acknowledgment letters | 146 | 56% |
Assessed technology to ensure most efficient administration of FOIA | 135 | 52% |
Posted a notice on the FOIA website informing requesters of most efficient way to make a request | 130 | 50% |
Posted a notice on the FOIA website informing requesters of any anticipated delays |
113 | 43% |
Used multitrack processing to further triage requests that could be processed more efficiently remotely | 100 | 38% |
Other, please explain | 51 | 20% |
Posted additional proactive disclosures for high public interest topics related to the COVID-19 pandemic | 48 | 18% |
Total number of responses to this question | 261 |
As shown in Table 3, the “Other, please be specific” comments focused on four particular themes: access, communication with requesters, process, and the technology agencies use to respond to requests.
Table 3: Selected Data from 2020 RMSA “Other, please explain,” Responses to Question 54 “Which of the following actions did your agency's FOIA program take in response to the COVID-19 pandemic? (Choose all that apply) (DOJ, Guidance for Agency FOIA Administration in Light of COVID-19 Impacts, https://www.justice.gov/oip/guidance-agency-foia-administration-light-covid-19-impacts, updated May 28, 2020.)”
Theme |
Agency Comments |
---|---|
Access Issues |
|
Communication |
|
Process |
|
Technology |
|
Observations:
- The COVID-19 pandemic forced maximum telework across the government and challenged agencies and FOIA programs in unexpected ways. The 2020 RMSA responses demonstrate the ways that agencies adapted their communication strategies, work processes and technology to keep staff safe while continuing to respond to requests.
- While we recognize that there is no one-size-fits-all approach to administering FOIA, agencies that reported that they effectively managed their FOIA process, resources and technology, and communicated with requesters were more likely to report that the pandemic did not disrupt their ability to respond to FOIA requests.
- Agency responses to the 2020 RMSA survey offer insight into the challenges agencies face as they transition to fully electronic recordkeeping per Office of Management and Budget (OMB)/NARA Memorandum M-19-21, Transition to Electronic Records, and manage their FOIA programs during a pandemic.[1]
E-Discovery Tools
E-discovery is the process of identifying, locating, preserving, collecting, preparing, reviewing, and producing electronically stored information (ESI) in the context of the legal process.[2] Federal agencies use e-discovery software to meet legal requirements by facilitating the collection of agency records for such purposes as internal investigations and complaints, civil and criminal litigation discovery requests, and responding to FOIA and Privacy Act requests. This includes facilitating the review, redaction and production of ESI such as emails, computer files, and databases.
A total of 266 respondents answered the 2020 RMSA question on their agency’s use of e-discovery tools to search for records when responding to FOIA and/or legal discovery. The majority of respondents—72 percent—said that their agencies use e-discovery tools to conduct searches of ESI. Twenty-four percent of the respondents indicated that their agencies do not use e-discovery tools to conduct searches and 4 percent answered that they did not know whether their agency uses such tools.
One hundred eighty-nine respondents answered the question about the purposes for which their agencies use e-discovery tools. Ninety-one percent of the respondents said that their agencies use e-discovery tools for FOIA responses involving requests for email records. Sixty-nine percent reported that their agencies use e-discovery tools for lawsuit-related requests. Sixty-two percent said that they use e-discovery tools to respond to legal discovery, including third-party subpoena requests. Fifty-eight percent said that their agencies use e-discovery tools to manage legal holds while 54 percent use them for FOIA responses not involving requests for email records.[3] Nearly half of the respondents said that their agencies use e-discovery tools congressional requests (49 percent). Respondents also reported that their agencies use e-discovery tools to de-duplicate records in response to requests (45 percent), internal research for or by staff (44 percent), and knowledge management (17 percent).
Table 4: 2020 RMSA Responses to Question 50, “For what purposes are e-Discovery tools used? (Choose all that apply)?”
Answer Options |
Count per answer option |
Percentage to total responses |
---|---|---|
FOIA responses involving requests for email records |
172 |
91% |
Lawsuit-related requests |
130 |
69% |
Legal discovery or third-party subpoena requests |
118 |
62% |
Managing legal holds |
109 |
58% |
FOIA responses NOT involving requests for email records |
102 |
54% |
Congressional requests |
93 |
49% |
De-duplication of records in responding to requests |
85 |
45% |
Internal research for or by staff |
83 |
44% |
Knowledge management |
33 |
17% |
Total number of responses to this question |
189 |
|
Of the 63 respondents whose agencies do not use e-discovery tools to search for records, 51 percent said that e-discovery tools are not available at their agencies, 29 percent cited the cost of e-discovery tools, and 6 percent said that there are a limited number of licenses available.
Respondents who answered “other, please be specific,” (56 percent) cited the following as reasons why their agencies do not use e-discovery tools to search for records.
- Agency does not need to use e-discovery tools
- due to limited resources, agency does not have e-discovery tools;
- due to size of agency and the number of requests received per year, the agency cannot justify the cost;
- staff are able to conduct searches without e-discovery tools.
- Agency expects a new system this year.
- Agency is currently working on obtaining e-discovery tools.
- E-discovery tools are solely used by the agency’s Office of General Counsel.
- E-discovery tools process has not been created yet.
The 2018-2020 term of the FOIA Advisory Committee recommended that the Office of Information Policy (OIP) at the Department of Justice provide guidance to agencies on the use of e-discovery tools to assist their searches of electronic records in response to FOIA requests. To better understand the e-discovery landscape throughout the federal government and inform OIP’s future guidance, OGIS used the 2020 RMSA to ask agencies about their use of e-discovery tools when searching for records to respond to FOIA requests and/or legal discovery and, if applicable, why agencies do not use e-discovery tools to search for records.
Observations:
- A significant majority of agencies that receive more than 50 FOIA requests annually use e-discovery tools to search for records when responding to FOIA and/or legal discovery. More work across the disciplines of FOIA, records management and technology is necessary. To that end, in 2020, the Chief FOIA Officers Council’s Technology Committee established seven interagency working groups to examine how agencies are using various tools to process FOIA requests and provide best practices and recommendations on leveraging technology to improve the FOIA process for all stakeholders. In addition, the 2020-2022 term of the FOIA Advisory Committee established a Technology Subcommittee “to prepare baseline standards and best practice recommendations to ensure that federal agencies have up-to-date access and impartial information on the functionality and operation of technology-driven solutions to ensure that the selection and implementation of new FOIA tools meet current and future needs of federal agencies and the requester community.”
- While e-discovery tools typically require a significant investment of resources, these tools may already be in use at agencies’ General Counsel offices, and FOIA offices may be able to leverage existing resources and licenses within their agencies to use these tools to process FOIA requests more efficiently.
- Smaller agencies, agencies that receive few FOIA requests per year and agencies that do not yet have a large volume of electronic records may not desire or need e-discovery software to assist in conducting FOIA searches and in processing and responding to FOIA requests.
Chief FOIA Officers and Agency Records Officers
The final FOIA question in the 2020 RMSA survey asked respondents to describe the working relationship between the Agency Records Officer (ARO) and Chief FOIA Officer (CFO) at their agency by choosing all of the statements that applied. A majority of respondents (51 percent) reported that their ARO and CFO work together on IT requirements that benefit both programs, and 50 percent reported that their training programs address the importance and relationship between FOIA and records management. Respondents also reported that the ARO and CFO provide training on records management and FOIA to each other's staff (46 percent), work together on high-profile or complex FOIA requests (45 percent), and identify programs or offices most likely to have responsive records (44 percent). Thirty-five percent said that the ARO and CFO coordinate search terms to identify responsive records, 20 percent said “Other, please explain,” and 7 percent answered, “None of the above.”
Table 5: 2020 RMSA Responses to Question 55, “Which of the following describes the working relationship between the Agency Records Officer and the Chief FOIA Officer? (Choose all that apply).”
Answer Options |
Count per answer option |
Percentage to total responses |
---|---|---|
Work together on Information Technology (IT) requirements that benefit both programs | 135 | 51% |
Training programs include the importance and relationship between FOIA and records management |
131 |
50% |
Provide training on records management and FOIA to each other's staff |
122 |
46% |
Work together on high-profile or complex FOIA requests |
119 |
45% |
Identify programs or offices most likely to have responsive records | 115 | 44% |
Coordinate search terms to identify responsive records | 93 | 35% |
Other, please explain |
54 |
20% |
None of the above |
18 |
7% |
Total number of responses to this question |
264 |
Table 6: Selected Data from 2020 RMSA “Other, please explain,” Responses to Question 55, “Which of the following describes the working relationship between the Agency Records Officer and the Chief FOIA Officer? (Choose all that apply).”
Theme | Agency Comments |
---|---|
FOIA |
|
Frequency |
|
Management/Organization |
|
Records Management |
|
Training |
|
Technology |
|
Observations:
- OGIS has long recognized the importance of strong records management as a solid foundation for an efficient and compliant FOIA program and smoother FOIA process.
- While the placement of FOIA and records management programs within agency organizational structures varies, records management and FOIA staff should work together to procure and deploy IT systems that will help agencies to carry out their missions and comply with the FOIA.
Conclusion
OGIS’s participation in the CRO’s annual RMSA helps us fulfill our statutory mission to “identify procedures and methods for improving compliance” under FOIA. We are hopeful that agency efforts to implement OMB/NARA Memorandum M-19-21, which encourages agencies to fully transition to electronic records management systems, will improve FOIA administration throughout the federal government.
Finally, we hope answers to questions regarding agency use of e-discovery tools to search for electronic records in response to FOIA requests will inform OIP guidance on the use of e-discovery tools to respond to FOIA requests and will assist OGIS, the Chief FOIA Officers Council, and the FOIA Advisory Committee in working to improve the FOIA process so that it works for all.
Methodology
The CRO’s office opened the 2020 RMSA on January 19, 2021, with a response deadline of March 19, 2021. The CRO’s office conducts the RMSA via an online survey tool that creates a unique link used to submit survey responses. Ninety-eight percent of agencies that received the 2020 RMSA link completed the assessment in accordance with NARA’s responsibility to report on the state of federal records management. Each year, Federal agencies are required to conduct an RMSA and submit the findings to NARA. The goal of the self-assessments is to determine whether Federal agencies are compliant with statutory and regulatory records management requirements.
A total of 266 respondents answered the FOIA questions included in the survey, though not all respondents answered every FOIA question. The respondents included records officers at all Cabinet-level departments, departmental components, and independent agencies. Seven non-executive branch agencies that are not subject to FOIA voluntarily took part in the assessment. CRO instructed agency Records Managers to consult with their agency FOIA Officers to answer the FOIA-related questions. (Percentages are rounded up and may not equal 100 percent.)
The full 2020 RMSA report and survey results will be included in the Federal Agency Records Management Annual Report, published on the CRO’s RMSA web page.
Questions 49-55 pertained to FOIA.
Q49. Does your agency use e-Discovery tools to search for records when responding to FOIA and/or Legal Discovery?
Answer Options |
Count per answer option |
Percentage to total responses |
---|---|---|
Yes |
191 |
72% |
No, please explain |
65 |
24% |
Do not know |
10 |
4% |
Total number of responses to this question |
266 |
100% |
Q50. For what purposes are e-Discovery tools used? (Choose all that apply)
Answer Options |
Count per answer option |
Percentage to total responses |
---|---|---|
Congressional requests |
93 |
49% |
De-duplication of records in responding to requests |
85 |
45% |
FOIA responses NOT involving requests for email records |
102 |
54% |
FOIA responses involving requests for email records |
172 |
91% |
Internal research for or by staff |
83 |
44% |
Knowledge management |
33 |
17% |
Lawsuit-related requests |
130 |
69% |
Legal discovery or third-party subpoena requests |
118 |
62% |
Managing legal holds |
109 |
58% |
Total number of responses to this question |
189 |
|
Q51. Please explain why e-Discovery tools are not used to search for records. (Choose all that apply)
Answer Options |
Count per answer option |
Percentage to total responses |
---|---|---|
Cost |
18 |
29% |
E-discovery tools are not available at my agency |
32 |
51% |
Other, please be specific: |
35 |
56% |
There are a limited number of licenses available |
4 |
6% |
Total number of responses to this question |
63 |
|
Q52. Has the COVID-19 pandemic disrupted your agency's ability to respond to FOIA requests?
Answer Options |
Count per answer option |
Percentage to total responses |
---|---|---|
Yes |
128 |
49% |
No |
127 |
48% |
Do not know |
8 |
3% |
Total number of responses to this question |
263 |
100% |
Q53. Which of the following explains why FOIA has been impacted? (Choose all that apply)
Answer Options |
Count per answer option |
Percentage to total responses |
---|---|---|
Agency staff are not available to conduct searches |
59 |
46% |
Electronic records are not accessible remotely |
11 |
9% |
FOIA case processing system is not available by remote access |
15 |
12% |
Paper records are inaccessible due to office closure |
103 |
80% |
Other, please be specific: |
46 |
36% |
Total number of responses to this question |
128 |
|
Q54. Which of the following actions did your agency's FOIA program take in response to the COVID-19 pandemic? (Choose all that apply) (DOJ, Guidance for Agency FOIA Administration in Light of COVID-19 Impacts, https://www.justice.gov/oip/guidance-agency-foia-administration-light-covid-19-impacts, updated May 28, 2020.)
Answer Options |
Count per answer option |
Percentage to total responses |
---|---|---|
Worked directly with requesters to tailor their requests for most efficient processing |
187 |
72% |
Posted a notice on the FOIA website informing requesters of most efficient way to make a request |
130 |
50% |
Posted a notice on the FOIA website informing requesters of any anticipated delays |
113 |
43% |
Included information about any anticipated delays in requester communication, including acknowledgment letters |
113 |
43% |
Used multitrack processing to further triage requests that could be processed more efficiently remotely |
146 |
56% |
Posted additional proactive disclosures for high public interest topics related to the COVID-19 pandemic |
48 |
18% |
Assessed technology to ensure most efficient administration of FOIA | 135 | 52% |
Other, please explain | 51 | 20% |
Total number of responses to this question | 261 |
Q55. Which of the following describes the working relationship between the Agency Records Officer and the Chief FOIA Officer? (Choose all that apply)
Answer Options |
Count per answer option |
Percentage to total responses |
---|---|---|
Work together on high-profile or complex FOIA requests |
119 |
45% |
Work together on Information Technology (IT) requirements that benefit both programs |
135 |
51% |
Training programs include the importance and relationship between FOIA and records management |
131 |
50% |
Provide training on records management and FOIA to each other's staff |
122 |
46% |
Other, please explain |
54 |
20% |
None of the above |
18 |
7% |
Identify programs or offices most likely to have responsive records |
115 |
44% |
Coordinate search terms to identify responsive records |
93 |
35% |
Total number of responses to this question |
264 |
|
References
[1] See OMB/NARA Memorandum M-19-21, “Transition to Electronic Records,” dated June 28, 2019.
[2] The Sedona Conference Glossary: eDiscovery & Digital Information Management, Fifth Edition, 21 SEDONA CONF. J. 263 (2020) at 303.
[3] A legal hold is "[a] communication issued as a result of current or reasonably anticipated litigation, audit, government investigation, or other such matter that suspends the normal disposition or processing of records." See id at 330.