The way people communicate has evolved and diversified considerably over the past two years. Gone are the days when emails and phone calls were the primary forms of communication for many. In both personal and professional settings, people are increasingly relying on messaging apps, social media, and video conferencing.
Meanwhile, many federal agencies are just beginning to tackle digitizing their paper records. M-19-21 requires all federal agencies to make their records fully electronic by December 2022. The difficulty for federal agencies to manage their paper document stores associated with these new emerging data sources has had major repercussions. For example, many agencies are late in responding to Freedom of Information Act requests, and Congress will sanction some of these agencies for their failure to meet these basic requests.
My data has data!
While many agencies are just beginning to address their paper record issues, the data landscape is rapidly shifting to new media – from instant messages and Zoom transcripts to surveillance, CCTV and camera footage. bodily – further complicating the digital data frontier. Federal agencies must capture, manage and glean information from these new sources while extracting decades of paper documents. This is a daunting task, but completing it is essential for tracking FOIA requests, maintaining compliance with congressional oversight, and staying on top of litigation.
Cloud solutions provide the agility and scalability to do this while ensuring agencies can take full advantage of tools like artificial intelligence that can help make sense of their data and use resources efficiently. taxpayers. Here’s how agencies can think about extracting valuable insights from their paper and digital resources to stay on top of their duties.
How can we even begin?
Artificial intelligence can streamline the management of paper records and provide valuable insights, including identifying what should be kept and what can be discarded, while helping to make sense of data in digital records. However, AI technology is only as good as it was trained to be. It requires human input to be effective. Agencies need to identify where the relevant data resides, identify appropriate AI technology that can work with various types of data, train the tool with the correct data, and then verify that the results are valid. Even the most powerful AI tools developed to date must be carefully managed to function properly.
This process may involve reconsidering agency collection requirements and strategies to account for new types of data. For example, agencies should create a data retention plan for digital records based on each device’s information policies and policies as well as regulations. This helps ensure that the proper records are kept for technologies like AI to extract insights from.
Rely on other agencies and organizations
One solution to this daunting challenge may be to identify a technology or consulting partner (or even collaborate with another agency) that has experience implementing effective AI data management solutions. . While the process of procuring and implementing AI can be daunting, many federal agencies are tackling this challenge together. They can look to each other for support and learning
Partners must be able to keep pace with agency needs and deliver scalable solutions so agencies can avoid having to repeatedly go through procurement, transition, and onboarding cycles. A useful step will be to develop evaluation criteria during the market research phase to identify vendors that prioritize innovation and integration and have a strong track record with other vendors. Because the procurement process tends to be lengthy in the federal space, it’s important to think long-term and find a partner who can grow with the needs of the agency for years to come.
Bells and whistles should come last
Once the agency has completed these first two steps, it can begin to migrate native paper and digital documents into a data lake where they can be stored, sorted, and searched in a uniform manner. At this stage, they should begin to assess the tools and processes that will make the act of search and retrieval easy and accurate. Metadata is an essential aspect that makes this information searchable. For example, scanned PDFs do not contain easily sortable metadata, so searching them is just as slow and wasteful as browsing through paper records.
Once the data has been mapped and stored, agencies can begin applying AI tools to derive insights from it and respond to FOIA requests, congressional oversight, and litigation. Here’s how a few key AI tools could come into play:
- Data visualization is an overview of an entire data lake. Tools with interactive visuals can help quickly sort through huge amounts of data by quickly providing information about things like file type, custodians, conversation parts and more.
- Predictive coding sorts and tags documents much faster than a human can. Predictive coding technology amplifies an agency’s resources by learning from the decisions people have made to surface related data without the continual resource drain of manual review.
- Clustering groups documents by conceptual similarity and generates information without user intervention. For example, FOIA requests submitted to Immigration and Customs Enforcement tend to relate to similar topics (border crossing, border retention, etc.). Identifying clusters of data on these keywords reduces the number of documents ICE would have to review when responding to a request, while ensuring that no relevant information is missing.
The new data frontier may seem daunting, but if agencies take it step by step, future-proof data management is possible. The benefits are many: helping them stay compliant, be more efficient with taxpayers’ money, and fulfill their duties to serve citizens.
Angela Kovach is Director of Government Solutions and Operations at Everlaw.