OGIS Issue Assessment: Leveraging Technology to Improve Freedom of Information Act (FOIA) Searches

Published July 31, 2019    |    Download the assessment

A central part of processing a Freedom of Information Act (FOIA) request is searching for responsive records. FOIA requires Federal agencies to conduct searches "reasonably calculated to uncover all relevant documents.”¹ However, there is no central government records clearinghouse or standard way for agencies to search for records. Each agency creates, maintains, stores and retrieves its own records as it carries out its mission. Indeed, strong records management is essential to an efficient and effective FOIA program that fully complies with the statute, thereby ensuring a smoother FOIA process.  

Recognizing there is no one-size-fits-all approach to administering FOIA, the FOIA Advisory Committee² recommended in 2018 that the Archivist of the United States, David S. Ferriero, take seven actions and promote 43 best practices to improve the administration of FOIA.  

This report focuses on three of the seven recommended actions. The three recommendations, aimed at improving searches, are:  

• that the Office of Information Policy (OIP) at the U.S. Department of Justice (DOJ) collectdetailed information in Chief FOIA Officer (CFO) reports regarding specific methods andtechnologies that agencies are using to search their electronic records, including email;
   
• that the CFO Council partner with the Chief Information Officers (CIO) Council to establish asubcommittee to study the use of FOIA technology across the government and identify bestpractices and recommendations that can be used across agencies; and
   
• that a modification to the Federal Acquisition Regulation (FAR) require all agencies to considerFOIA obligations when acquiring electronic records management software.³

CFO Reports: Leveraging Technology in FOIA Searches

The 2019 CFO reports show that FOIA processors at some Federal agencies are working across disciplines, collaborating with agency technology experts and leveraging technology tools to conduct more efficient searches for records, including emails. The findings show that the FOIA process works best when agency FOIA programs cooperate with other program offices to improve searches for responsive records. In its guidelines for the 2019 CFO reports, OIP asked agencies that received more than 50 FOIA requests in Fiscal Year (FY) 2017 the following question:⁴ 

Enlarge

Figure 1

Pie Chart for responses to the CFO 2019 question, "Is your agency leveraging technology to facilitate efficiency in conducting searches, including for emails?"

Is your agency leveraging technology to facilitate efficiency in conducting searches, including search for emails? If so, please describe the type of technology used. If not, please explain why and please describe the typical search process used instead.” 

The responses show that agencies are using technology in various ways to search for responsive records, including through e-discovery tools and collaborative platforms that foster document storage and sharing. Sixty-two of 72 agencies responded that they use technology to facilitate efficiency in conducting searches while two agencies responded that they do not.⁵  

In response to the follow-up prompts, about one in five agencies reported collaborating with colleagues with technology expertise who are outside of the FOIA office.  

The answers, provided in short narrative, illustrate that some agencies are fulfilling one or more of the FOIA Advisory Committee’s three best practices regarding records management and search. 

Those best practices are:  

• designating a point of contact to approve search requests within records management systems; 
   
• creating add-on IT systems for exporting records; and 
   
• making the end-goal of responding to FOIA requests a major component when developing the agency’s records management system and workflows. 

Best Practice: Designate points of contact for search requests

To improve the quality of searches for responsive records and to promote efficiency, the FOIA Advisory Committee suggested that agencies designate a point of contact to approve search requests within their respective records management systems. Some of the agencies responding to the FOIA search question in the CFO reports noted collaboration with non-FOIA colleagues.  

• The Office of the Chief Information Officer (OCIO) at the U.S. Department of Agriculture (USDA) conducts FOIA searches for responsive electronic records for several of the Department’s FOIA components. The searches, conducted by the “Client Experience Center,” include electronic messaging and electronic calendars.
   
• The Corporation for National and Community Service’s Office of Information Technology and the Environmental Protection Agency’s eDiscovery Division use e-discovery tools to conduct extensive searches. 
   
• At the Federal Energy Regulatory Commission (FERC), IT experts and FOIA professionals developed guidance for staff to conduct more effective and efficient key word searches.   
   
• The Merit Systems Protection Board uses contractors to teach FOIA professionals how cloud-based software tools can improve the FOIA process, including searches.  
   
• The Commodity Futures Trading Commission FOIA staff works with its forensics team to conduct searches for records, including email.  
   
• FOIA professionals work with IT professionals to conduct electronic records searches at about a dozen agencies, including the National Endowment for the Humanities, the National Labor Relations Board, the National Science Foundation, the National Transportation Safety Board, Occupational Safety and Health Review Commission, Overseas Private Investment Corporation, Peace Corps, Small Business Administration, the U.S. Postal Service, and the U.S. Agency for Global Media (formerly the Broadcasting Board of Governors).  
   
• Many DOJ components use e-discovery software particularly for searches involving email, which increases efficiency; however, IT support offices “can often have a very long backlog of searches needing to be conducted due to the volume of requests and the time it takes to conduct a pull of records for each request.”  

Best Practice: Create add-ons to IT systems for exporting records

The FOIA Advisory Committee suggested that agencies create add-ons to IT systems for exporting records. Such add-ons ensure agencies export records in usable formats, streamline the FOIA search process, promote efficiency, and reduce costs by reducing search times. Two responses in the CFO Reports addressed this best practice.  

• USDA purchased a tool that, when added to the department-wide FOIA tracking system, enables FOIA components to search, sort and select large numbers of responsive records in preparation for review. USDA noted that budget constraints limited the number of licenses for the tool that the agency purchased.  
   
• Although FERC did not specifically mention an add-on tool, FERC IT experts frequently assistwith email searches and produce the responsive records in formats that are usable and efficient to
  review. 

Best Practice: Incorporate FOIA into agency records management systems and workflows

The FOIA Advisory Committee suggested that agencies make the end goal of responding to FOIA requests a major component when developing the agency’s records management system and workflows. Increasing communication and coordination between records management and FOIA offices streamlines the process of responding to FOIA requests, and increases transparency by improving agencies’ ability to proactively release records. 

• The National Indian GamingCommission moved its FOIA Office from the Office of General Counsel to the Division of Information Technology to ensure that FOIA professionals have access to IT tools and that FOIA is considered during IT decision-making. “Already this year, for example, the [FOIA Office] has identified significant FOIA-related deficiencies in the agency’s email search and retrieval tools and these concerns are now incorporated into IT resource planning.” 
   
• FOIA professionals at several Department of Defense components, including the Air Force, the National Geospatial-Intelligence Agency, and the National Security Agency, worked with their records managers and IT officials to incorporate electronic records management technology to improve search capabilities.  

Several other agencies reported working across disciplines to ensure better FOIA processes, including searches. At the Equal Employment Opportunity Commission, for example, a Senior Attorney Advisor for Technology is dedicated to finding technological solutions to inefficiencies in the FOIA process. And at the Department of Homeland Security, a FOIA Technology System Requirements Working Group is writing requirements for a department-wide FOIA processing and case management system.
 

CFO Technology Subcommittee

The CFO Council, chaired by the directors of OGIS and OIP, established the Technology Subcommittee in September 2018. FOIA professionals from 11 departments and agencies are on the Subcommittee, which is co-chaired by Eric Stein, Director of the Office of Information Programs and Services at the Department of State, and Michael Sarich, FOIA Director at the Department of Veterans Affairs’ Veterans Health Administration. The Subcommittee reviewed the annual CFO reports and used other methods to assess the FOIA IT landscape, and is identifying best practices and recommendations for agencies in accordance with the FOIA Advisory Committee’s recommendation. The co-chairmen will present an update of their work to the CFO Council on August 5, 2019, and to the third term of the FOIA Advisory Committee on September 5, 2019.

Federal Acquisition Regulation 

OGIS is working with the NARA representative to the Federal Acquisition Regulation (FAR) Council and NARA’s Office of General Counsel to draft a business case regarding FOIA and IT procurement to submit to the Council.⁶ The business case would justify including in the FAR a clause in solicitations and contracts for the design, development and operation of a system of records to address an agency’s obligation under FOIA to provide access to Federal agency records.  

Conclusion 

With regard to the 2019 CFO Reports and leveraging technology to improve FOIA searches, we have learned the following:  

• Agencies across the government are using a wide range of technology tools.
   
• The FOIA process works best when FOIA processors work with agency technology experts and leverage technology tools.  
   
• Only one in five agencies reported such collaboration. More work across the disciplines of FOIA and technology is necessary.  

Work continues on both the CFO Technology Subcommittee and the FAR recommendations. OGIS will issue updates as necessary. 

References

¹ See Weisberg v. U.S. Dep't of Justice, 705 F.2d 1344, 1351 (D.C. Cir. 1983).

² Chartered in 2014 by the National Archives and Records Administration (NARA), the FOIA Advisory Committee serves as a deliberative body to foster dialogue between the Federal government and the requester community, solicit public comments, and develop recommendations for improving FOIA administration and proactive disclosures. The Committee reports to the Archivist of the United States. 

³ The other four recommendations pertain to standard requirements for FOIA processing tools; the intersection of FOIA and Section 508 of the Rehabilitation Act regarding document access for individuals with disabilities; methods used by agencies to prepare documents for posting on agency FOIA reading rooms; and FOIA performance standards. https://www.archives.gov/files/final-report-and-recommendations-of-2016-2018-foia-advisory-committee.pdf

⁴ U.S. Department of Justice, Office of Information Policy. “Guidelines for 2019 Chief FOIA Officer Reports” (January 29, 2019) available at https://www.justice.gov/oip/guidelines-2019-chief-foia-officer-reports.

⁵ One agency that received more than 50 FOIA requests in FY 2017 used OIP’s CFO report template for agencies receiving 50 or fewer requests, which did not specifically ask the leveraging technology question. Reports for seven agencies, including three Cabinet-level agencies, had not been posted to OIP’s CFO reports website or the respective agency FOIA websites at the time of our analysis.

⁶ The FAR Council “assist[s] in the direction and coordination of Government-wide procurement policy and Government-wide procurement regulatory activities in the Federal Government.” See https://www.acquisition.gov/far-council.