XKEYSCORE – Digital Citizenship and Surveillance Society

XKEYSCORE, ACLU document archive, slide #11.

Purpose:

XKEYSCORE is an NSA search and analysis system for data collected by other surveillance programmes. The system is described by Snowden as a search engine that provides a “one-stop shop” for access to content, metadata and real-time tracking and monitoring of user activities (COU01). Access to XKEYSCORE is shared with a number of other intelligence agencies including GCHQ (COU01, GUA01). In 2012, GCHQ’s TEMPORA programme was the largest source of XKEYSCORE data (EFF01).

The system incorporates user interfaces, databases and algorithms to select specific types of content and metadata that have already been collected by other surveillance programmes. Data can be retrieved using “strong selectors” such as email addresses and “soft selectors” such as keywords (ACU01). Rules for identifying particular kinds of data can be created and stored in the system. For example, analysts can target Tor users through rules that select web searches related to Tor and connections to the Tor network (NDR01). XKEYSCORE also has the ability to alert analysts to the activities of specific email and IP addresses (GUA02).

In 2008, the system included over 700 servers at approximately 150 locations around the world (ACU01). Content remains in the XKEYSCORE environment for three to five days, while metadata is stored for 30 days.

Capabilities (ACU01, EFF01):

Ingestion of “full take” from NSA and partner agency bulk collection programmes.
Federated query mechanism allows analysts to search multiple databases with a single query.
Content and metadata can be searched using “strong selectors” and “soft selectors”.
Rules for matching particular kinds of data can be created and stored in the system.
Computer systems that are vulnerable to attack can be identified by monitoring network traffic.
Documents can be traced back to their authors.
Pattern-of-life analysis can develop profiles of individuals or find individuals matching a profile.

Data sources (ACU01, ELE01, SES01, WEE01):

CIA/NSA Special Collection Service (F6).
NSA Special Source Operations (such as PRISM, MUSCULAR and INCENSER).
Foreign satellite data (FORNSAT).
MARINA metadata repository.
TRAFFICTHIEF metadata repository.

Related programmes (ACU01, EFF01, ELE01, SES01):

PRISM – NSA programme for content and metadata collection from service providers via the FBI.

MUSCULAR – GCHQ programme for bulk data collection from service provider data centres.

INCENSER – GCHQ programme for bulk data collection from fibre-optic cables.

TEMPORA – GCHQ programme for bulk data collection and buffering.

TRAFFICTHIEF – NSA repository for metadata about selected targets.

MARINA – NSA repository for bulk Internet metadata.

PINWALE – NSA repository for selected content.

Layers of operation:

Network layer, transport layer and application layer: Matching content and metadata against rules defined by analysts.
Social layer: Aggregation of content and metadata from multiple sources, pattern-of-life analysis.

Background:

XKEYSCORE training materials detail how analysts can use it and other systems to mine enormous agency databases by filling in a simple on-screen form giving only a broad justification for the search (GUA02). Requests are not reviewed by a court or any NSA personnel before being processed. The programme covers “nearly everything a typical user does on the internet”, including the content of emails, websites visited and searches, as well as their metadata (GUA02). The programme also allows for on-going “real-time” interception of an individual’s Internet activity (GUA02).

Data storage is an issue. According to leaked documents, “At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for as little as 24 hours” (GUA02). In response, the NSA has created a multi-tiered system that allows analysts to store “interesting” content in other databases, such as one named PINWALE, which can store material for up to five years (GUA02).