Oversight of Intelligence Operations Begins with Collections Management

Intelligence requirements (IRs) prioritize a threat intelligence program’s needs, but without collections management, your teams won’t be able to determine or assess the resources you need to satisfy those IRs.

It’s a mistake to overlook or misunderstand collections management, and often, operations suffer for it. Collections management is crucial because it enables practitioners to identify and access the critical data and information on which their operations rely. It can also provide invaluable insight into how each component of an intelligence operation is functioning and performing.

Let’s dig inside two important facets: collection requirements management, and collection operations management.

Collection Requirements Management (CRM)

This part of collection management is where an operation identifies, prioritizes, and documents the observables and inputs—known as collection requirements—needed to fulfill IRs. The collection requirements should map to and reflect the priority of each IR.

If an intelligence operation wants to reduce fraud losses and prioritizes determining which tactics, techniques, and procedures (TTPs) fraudsters are using to commit account takeover fraud, the mapping of an IR to CR—ranked highest to lowest in priority—could look like this:

•  Monitor Russian-language deep & dark web (DDW) forums X, Y, and Z.

•  Identify emerging TTPs and mitigation tactics for account takeover fraud.

•  Monitor reputable open-web resources such as sites X, Y, and Z, to identify emerging TTPs and mitigation tactics for account takeover fraud.

Prioritization is essential, and making the determination as to which CRs are critical to your highest-priority IRs can help you to allocate your resources effectively.

Collection Operations Management (COM)

COM specifies how to satisfy collection requirements and operationalizes collection resources and activities. This function typically entails:

•  Auditing existing resources to identify gaps that could impede operations with respect to CRs. Satisfying the first example CR specified above would require access to the right DDW forums, as well as personnel with fluency in Russian, an intimate familiarity with the Russian-language underground, and expertise in account takeover fraud TTPs.

If no such sources are accessible and/or no such personnel available, COM would likely need to re-evaluate the CR’s priority and, if necessary, seek out suitable third-party vendors to provide the source accessibility and subject matter expertise needed to satisfy the CR.

•  Monitoring for source outages. While any online source can experience an outage, DDW sources tend to be particularly volatile. Sites crash, markets shut down, and forum URLs change without warning. COM needs to identify and provide access to alternative sources and help teams adjust course to satisfy their CRs.

•  Identifying redundancy in source coverage, which tends to be more common when an intelligence operation works with third-party vendors to augment its collection sources. Different vendors may offer access to the same or similar sources, but regardless of whether redundancy occurs within in-house or third-party sources, COM needs to identify it to determine whether resources need to be reallocated.

But in many cases, redundancy can be beneficial—especially for sources that satisfy high-priority CRs deemed integral to the lifeblood of your organization. These CRs should be supported by more than one source when possible; if an outage occurs, having a suitable backup source can help keep your operation on track.

•  Working with the OPSEC team to ensure in-house collection personnel can safely access and gather observables from the sources specified by the CRs. OPSEC controls can include persona management, privacy and obfuscation measures, and awareness training for personnel who access deep & dark web sources, as well as service-level agreements and traffic light protocol (TLP) classifications for those who engage in external information sharing, for example.

Collections Managers: More Than Just Administrators

A collections manager oversees CRM and COM functions, and they should be perceived as more than just providing administrative oversight. An effective collections manager is integral to the success of an intelligence operation. Not only does this person plan and integrate all collection processes, they are also your best resource to make an informed evaluation of your effectiveness in both CRM and COM and can help to inform:

Health and Performance

An effective collections manager can provide fast and accurate insight into the health of an intelligence operation by evaluating each IR and CR with respect to collection resources and activities. This process typically aims to determine:

•  The status and progress to-date of the operation’s IRs and corresponding CRs

•  How well the progress of each CR maps to its IR

•  How well collection resources and activities were able to fulfill each CR

•  The extent to which resource gaps and/or redundancy were present and, if so, how they impacted the operation

Using the previously mentioned account-takeover fraud IR and corresponding CRs for reference, an operational health assessment might seek to answer questions such as:

•  How many new TTPs have been identified since the operation began?

•  How does this number compare to the operation’s and anti-fraud team’s expectations?

•  What collection resources and activities were most valuable for identifying the specified TTPs? Least valuable?

•  What additional collection resources and activities could have enabled the operation to identify more TTPs more efficiently and effectively?

Answering these types of questions enables the collection manager to update stakeholders and decision-makers on not just the overall health of the intelligence operation, but also on the individual value and performance of each collection resource and activity.

Return on Investment

Collection managers eventually must answer the dreaded return-on-investment question with regard to the overall intelligence operation. This is complex, but a manager armed with insights gleaned from the operational health assessment should be able to estimate the value provided by individual collection resources and activities.

For example, let’s say an intelligence operation invests $50,000 in a third-party vendor to gain access to a dark web forum deemed critical to identifying the emerging account-takeover fraud TTPs needed to satisfy a high-priority IR. If data obtained from this forum does satisfy the IR, the collections manager can then estimate whether $50,000 is more or less than the reduction in fraud losses that resulted from satisfying the specified IR.

This approach can also help justify budgetary allocations and expansions. If an operational health assessment reveals a gap in a critical resource, the collection manager can estimate the potential impact of that gap and use it to justify the additional budget needed to procure said resource.

Ultimately, it’s important to remember that regardless of the size, objective, or sophistication of your intelligence operation, collection management is a must-have. And while the above crash course introduces some of the key components of an effective collection management program, it is neither comprehensive nor prescriptive. Collection resources and activities can and do vary substantially from operation to operation and organization to organization. Given the complexity and resource-intensive nature of initiating and developing a collection management program, less-experienced teams looking to do so are advised to seek the support of trusted peers and reputable third-party vendors accordingly.