Equipping Modern Command and Control Centers in the Age of PAI

Command and control centers support a variety of critical government operations—like geopolitical risk assessments, situational awareness, crisis management, counterterrorism, cybersecurity, and military/tactical operations.

In the last few years, demand for open-source data has exploded, covering about 80% of data used for intelligence reports. These sources offer valuable, timely insights, but command centers are also ingesting more data than ever.

Data abundance creates its own set of challenges in addition to emerging risks sparked by the COVID-19 pandemic and evolving global powers. Command and control centers must evaluate their toolkit to ensure that analysts are equipped to handle big data, improve their open-source tradecraft, and provide timely insights for emerging national security risks.

This article covers:

• What the rise of open-source data means for command centers
• Emerging challenges for command centers
• Software capabilities to support command centers in the modern information space

Command and control centers and the rise of PAI: OSINT

According to the IDC, 463 exabytes of online data will be created by 2025. Thanks to widespread digital transformation and social media use, much of this includes publicly-available information (PAI). 

PAI can come from easily accessible sources like the indexed, surface web, as well as social media and hard-to-access deep web and dark web content. Once this raw data is processed and analyzed, it becomes open-source intelligence (OSINT).

In the last few years, intelligence professionals have come to rely on PAI and OSINT to find security risks and access accurate insights quickly. Why?

• Public information sources often contain insights not available through proprietary or classified feeds. For example:
  • Data breaches are often disclosed on public paste sites before they are detectable within an organization. 
  • Social media can provide on-the-ground insights in areas not accessible to personnel or field sensors on the battlefield.
  • Classified information sources often focus on specific targets, whereas PAI can provide broader insights into populations and information environments.
• PAI is logistically easier to share between partners and government allies than classified information.
• Online sources like social media improve speed-to-information for breaking events, providing real-time updates. 

In February 2021, the CSIS Technology and Intelligence Task Force called for OSINT to be upgraded to “core” intelligence alongside classified INTs, like signals intelligence (SIGINT). The United States Defense Intelligence Agency—which relies on open-source data for 80% of its reports—has also been designated as a lead agency for OSINT.

The rise of PAI and OSINT has implications for command and control centers—which must adapt their processes and software accordingly.

Five challenges for command and control centers

While an influx of open-source data creates its own set of challenges, command centers also face a range of other security concerns as the post-pandemic threat landscape evolves. Here’s a rundown of some common challenges for command and control centers.

1. Data overload

Command centers—whether they’re focused on cybersecurity or generating geopolitical intelligence—are plagued by data overload. This is partly due to the sheer volume of available open-source data. Simply put, analysts don’t lack data. They lack the resources to process, analyze, and triage it efficiently.

This causes alert fatigue for analysts, making them more likely to overlook important data and take longer to transform raw data into actionable insights. According to Forrester research, security teams get over 11,000 daily alerts on average. Almost a third of them are false positives, and only 72% are addressed at all due to time constraints.

This cycle sets command centers up for a reactive rather than a proactive approach. Higher up the chain of command, this can cause misinformed or delayed decision-making—putting people, assets, data, and national security at risk.

2. Skills shortages

Advanced data-handling capabilities using AI can address data abundance challenges. But this requires skilled data scientists, which are in short supply in the US—especially when it comes to filling public sector roles.

Other factors can compound skills gaps for command and control centers. For one, intelligence and security software is often complex, taking time and skill to master. For entry-level analysts—who are easier to hire than advanced data professionals—this typically means longer training times and less consistent engagement with their toolkit. 

This is not ideal if the goal is faster, more accurate intelligence gathering, especially when entry-level analysts often turn their seats over in less than a year.

3. Adversary Capabilities

AI is considered an indicator of global power and influence. Beyond addressing skills shortages and data overload, AI is crucial for supporting national security initiatives—like cybersecurity, logistics, and counter-terrorism—in the coming decades.

These are technologies that global powers like China have made focused efforts to develop. By 2030, China aims to close the AI gap with the US with its growing research production and talent pool. China currently has 4 times as many STEM students and 3 times as many computer scientists as the US. Command and control centers will need to contend with more advanced adversary capabilities while filling skills gaps and navigating higher data volumes.

4. Cybersecurity

The COVID-19 pandemic has elevated cyber risks and dramatically shifted the cybersecurity landscape as attackers leverage public fear and capitalize on a less-secure, home-based workforce.

Ransomware attacks are also becoming more sophisticated and higher impact. Ransomware groups—many of which are highly skilled, backed by nation-states, and well-funded—have increasingly impacted critical infrastructure and government agencies (for example, the SolarWinds and Colonial Pipeline breaches). 

In 2022, the Russia-Ukraine war is also sparking political motivations for ransomware groups, which could cause higher risks for Western governments and critical infrastructure. Command and control centers must maintain agile processes and software to keep pace with evolving cybersecurity tactics.

5. Data management

We know that command centers now handle a lot of data in a variety of formats. Data management challenges are commonplace for data-reliant organizations, manifesting as data silos, low data quality, security issues, and lack of end-user accessibility. Downstream, this can produce slower insights and information gaps that result in poor decision-making and higher security risks.

Data management practices are also becoming a higher priority as jurisdictions implement data privacy regulations like GDPR and CCPA. According to Varonis, $63M worth of regulatory fines were issued in the GDPR’s first year. Command and control centers need to develop robust data governance solutions to ensure that their workforce maximizes data value and avoids costly regulatory violations.

Ten software capabilities command centers need

An effective operations center starts with effective data management and intelligence software. According to a Forrester report, security teams use 10 different security tool categories, on average, to do their jobs. In the modern information environment—where open-source data is king—what software capabilities do command and control centers need in their toolkit?

1. Managed attribution

Analysts navigate the web to gather open-source data. This can expose vulnerable information about the analyst and their web activities. Managed attribution capabilities enable users to blend into online spaces to stay secure and conceal their footprint from potential adversaries.

Common features:

• One-time-use browser instance generation
• Online persona customization
• Third-party integrations

2. Open source intelligence (OSINT)

To cope with the growing demand for open-source data, OSINT capabilities help analysts gather, process, and analyze social media and surface, deep, and dark web data. There are a variety of free OSINT tools available for specific functions—like reverse image search or maps like Google’s live traffic feature. More advanced tools consolidate several feeds and make unindexed content easily searchable.

Common features:

• Advanced search filters and data analytics
• Real-time data access
• AI and ML for assisted data analysis

3. Geographic information system (GIS)

According to Esri, a geographic information system (GIS) “connects data to a map, integrating location data (where things are) with all types of descriptive information (what things are like there).” Integrating location data with PAI has immense value for analysts, helping them understand relationships between online activities and their geography. 

Common features:

• Geofencing
• Different map views (satellite, street, topography, etc.)
• Maps geotagged open-source content like social media posts

4. Visualization

This capability aids analysis by visualizing patterns and relationships between data points. Visualizations can look like charts and word clouds, or link entities on node-based graphs. Visualization capabilities are present in some capacity in most intelligence software.

Common features:

• Public, commercial, and proprietary data inputs
• Dashboards and interactive visualizations by time, location, user, topic, etc.
• Automatic pattern recognition and visualization

5. Investigation and case management

Investigative capabilities are valuable for online investigations targeting specific individuals or entities of interest. The goal is to efficiently resolve online identities, which is common for command and control centers investigating crime.

Common features:

• Link analysis for identifying information (names, emails, aliases, etc.)
• Social media and surface, deep, and dark web data coverage
• User-dependent access control for secure investigations

6. Critical event management (CEM)

CEM-focused tools are designed to predict and detect critical events and support response and recovery. CEM capabilities are ideal for command and control centers responding to crises like cyber attacks and natural disasters. They also promote cross-departmental communications, since multiple teams are typically involved in crisis response.

Common features: 

• Real-time data access
• Location-based searches and alerts for physical risks
• Mobile-compatible for any time alerts
• Mass notification system

7. Artificial intelligence (AI)

For command and control centers, AI capabilities support data management and analysis at scale. While human expertise is irreplaceable for analysis, AI techniques like natural language processing can offload straightforward but time-consuming tasks. 

Common features:

• Anomaly detection
• Content classification and analysis
• Analytics and report generation

8. Evidence capture

As analysts gather data, they navigate dozens of sites per session. Evidence capture capabilities automatically document web pages during an investigation. This saves analysts time, resources, and confusion when they revisit their findings later. 

Common features:

• Automatic webpage documentation and annotation for time, location, etc. of capture
• Image and EXIF metadata extraction
• Search bar for fast evidence retrieval

9. Information management and sharing

Information management and sharing capabilities go a step further than evidence capture, providing a virtual analyst notebook to track, organize, and share investigations. This is valuable for ensuring data provenance, especially if personnel higher up the chain of command need to verify the accuracy of an intelligence report. 

Common features:

• Search capability across logged investigations
• Information sharing and collaboration features
• Reporting and export for auditing

10. Data governance

According to Talend, data governance is “a collection of processes, roles, policies, standards, and metrics that ensure the effective and efficient use of information in enabling an organization to achieve its goals.” Data governance software supports these processes and ultimately keeps data secure, ensures data quality, and maximizes data’s value for end-users.

Common features:

• Secure information sharing
• AI and ML-powered features for streamlined workflows
• Audit capabilities

How to successfully adopt

In an information environment overloaded with data, analysts need to generate insights quickly without sacrificing accuracy. Regardless of a tool’s core capabilities, usability, real-time data access, and comprehensive coverage address common resource and information gaps. To help address these challenges, a control room should prioritize tools that:

• Are simple and easy to use. User-friendly PAI/OSINT software drives adoption and alleviates the training burden. Some commercial tools are powerful but take weeks or months to learn. Software with a simple UI, intuitive workflow, and no click-heavy processes helps address the data overwhelm and skills shortages faced by command centers.
• Provide real-time information. Open-source data is valuable partly because it can provide updates faster than other information sources. However, some PAI tools have slower latency times, displaying data long after queries. Software that offers real-time or near real-time data access can expedite workflows and help command and control centers respond faster to risks.
• Have adequate data coverage. Information gaps can misinform decision-makers, potentially harming assets and national security interests. Intelligence professionals can enable more accurate, comprehensive intelligence with tools that prioritize data variety. For command centers relying on OSINT, this means a wide range of surface, deep, and dark web sources, as well as regional networks relevant in areas of interest.

Command centers are overloaded with data in an intelligence landscape that demands open-source content. Simply put, they need a toolkit that enables robust data management and efficient collection, analysis, and sharing. 

As command and control centers ingest more data, these capabilities will ensure faster, more informed policies and decision-making for national security.