← all systems · see it in The System
Signals intelligence
Intelligence-gathering by intercepting signals: communications between people (COMINT) and electronic signals not used for communication (ELINT, FISINT), with cryptanalysis where traffic is encrypted and traffic analysis of who signals to whom. Walked as the practice and its institutions at the level the public literature describes: the signals in the ether and on the wire, collection, processing, analysis, reporting and sharing, direction and oversight, and the decisions it informs; its history from wireless (1900-1904) through the world wars, the Cold War agencies and alliances (GC&CS/GCHQ, NSA, UKUSA, ECHELON) to the internet era (PRISM, Tempora) and the law and oversight around it (FISA, the Church Committee, the IPA, European courts).
Properties published scienceour proposal
Sourced fields, and our own readings.
| born at | 1904 (Russo-Japanese War), with interceptions as early as 1900 |
| born precision | about 5 years |
| boundary | Inside: the interception of other parties' signals for intelligence and everything that turns it into intelligence: collection platforms and access, processing (including cryptanalysis and direction finding), analysis, reporting and sharing, direction and the oversight that bounds it, and the agencies and alliances that run it. Outside: the signals' media (the electromagnetic field, cables, carriers, providers), the targets and their cryptography, the consumers (governments, commanders) who act, the law and courts, the press and the public, and the people whose traffic is taken. |
| nested level | A state's intelligence apparatus (part of the state); built from collection, cryptanalysis, traffic analysis and analysts; carried on the electromagnetic field and the telecommunications networks; its output runs into decisions of state and war. |
| scale | technological |
Measures (24) published science
| characteristic clock: HF/DF fix time | on the order of one minute (rotating loops: a minute or more; HF/DF: essentially instantaneous bearings) |
| Tempora retention | content three days, metadata thirty days |
| Zimmermann telegram: time to full decryption | partially decoded the next day; completed in about three weeks; sent 17 January 1917, shown to the US embassy 19 February 1917 days |
| Venona project duration | 1 February 1943 to 1 October 1980 (37 years) |
| German wireless traffic intercepted in WWI | over 80 million words (the totality) |
| EU data retention period (directive struck down 8 April 2014) | up to two years |
| Room 40 decrypts in WWI | around 15,000 messages |
| Venona decryption yield | under 3,000 of some hundreds of thousands (claimed) messages |
| U-boats sunk with an HF/DF contribution | 24 % |
| Venona duplicate key pages | around 35,000 pages |
| share of women at Bletchley Park | 75 % |
| war shortening credited to Ultra (Hinsley) | two to four years (others: as little as three months) |
| FISC warrant applications approved, 1979-2006 | 22,985 of 22,990; 5 definitively rejected applications |
| NSA employees | roughly 32,000 people |
| documents Snowden copied (NSA estimate) | 50,000 to 200,000 documents |
| IPCO judicial commissioners | fifteen senior judges, around 50 staff people |
| immediate drop in privacy-sensitive Wikipedia views after June 2013 | 995,085 of 2,960,778 (over 30%) views per month |
| Brazilian Internet paths traversing the US | 84% (37% with relay-based avoidance) |
| false evaluations at 99.9% accuracy over 10 million people | 10,000 people |
| share of intercept reports read by analysts | about 50% without the triage tool, about 20% with it (a 60% reduction); scored 18-22% vs 65% |
| SIGINT satellite antenna gain | 31.6 dBi at 10 GHz; tested 2-18 GHz |
| US adults aware of the collection (Pew 2014, cited) | 87% (43% 'a lot', 44% 'a little'); 80% agree Americans should be concerned |
| Firms relying on Privacy Shield when invalidated (health sectors) | 137 health IT, 90 biopharma, 56 medical devices, 36 health care services firms |
| Thunderstorm detection range in Watson-Watt's lightning work | 2500 km |
Open contentions (5) open contention
Where sources or readings disagree. Both stay on the map.
- C-1741 Does collecting everyone's data in bulk catch terrorists?
booked: The map leaves it contested: researchers doubt it works, and the agencies' own case was not read.
the other reading: bulk collection and machine learning unlikely to pinpoint terrorists (Verhelst et al.); agencies' own case for bulk powers (the IPA's operational review) not read here - C-1761 Is data about your messages as sensitive as the messages?
booked: The map books metadata as revealing, and notes that law protects it less than content.
the other reading: metadata receives substantially less protection than content in US law and abroad - C-1764 Is internet surveillance done everywhere or at a few chokepoints?
booked: The map books monitoring at a few central points, with the 'pervasive monitoring' description alongside.
the other reading: pervasive monitoring (RFC 7258's term) - C-1776 When did signals intelligence begin?
booked: The map dates modern signals intelligence to the Russo-Japanese War, with earlier Boer War interception alongside.
the other reading: electronic interception as early as 1900, in the Boer War; the encyclopedia flags 'first in history' as ambiguous - C-1781 Did breaking Enigma shorten World War II by years?
booked: The map keeps it contested: two to four years, or as little as three months.
the other reading: as little as about three months (Keegan and others); Hinsley and others stress the difficulty of counterfactuals
Partners and parts published science
Built from / builds (9)
- Cryptanalysis (the attack side of the practice)
- Communications intelligence (COMINT)
- Electronic intelligence (ELINT)
- Traffic analysis
- Direction finding (HF/DF)
- Listening stations and collection platforms
- Intelligence analysis and analysts
- Signals agencies (NSA, GCHQ, ASD, CSE, GCSB, Unit 8200 and others)
- Interception against communications security (top-level)
Sits inside / holds (2)
Trades with (valence) (18)
- Radio via sys:electromagnetic-field · Signals intelligence gain
- Electromagnetic field via sys:electromagnetic-field · Signals intelligence gain
- Human (Homo sapiens) via sys:electromagnetic-field · Signals intelligence gain
- State (polity); the nation state its predominant modern form via sys:government · Signals intelligence gain
- Law (rules and law) via sys:law · Signals intelligence mixed
- Internet via sys:internet · Signals intelligence gain
- Mass media via sys:mass-media · Signals intelligence loss
- Computer via sys:computer · Signals intelligence gain
- Cryptography via sys:cryptography · Signals intelligence mixed
- World wars (the two world wars of the 20th century: World War I, 1914-1918, and World War II, 1939-1945) via sys:electromagnetic-field · Signals intelligence gain
- Artificial intelligence via sys:intelligence-information · Signals intelligence mixed
- Organization via sys:telephone-network · Signals intelligence gain
- Privacy via sys:telephone-network · Signals intelligence gain
- Cyberwarfare via sys:internet · Signals intelligence mixed
- Cyber threat intelligence via sys:intelligence-information · Signals intelligence even
- Surveillance via sys:electromagnetic-field · Signals intelligence gain
- Information warfare via sys:intelligence-information · Signals intelligence even
- Telephone via sys:electromagnetic-field · Signals intelligence gain
expressed by (26)
- Interoperability/Transduction (Bridging)
- Amplification/Scale-Up (Manufacture & Distribution)
- Symbiosis/Co-Creation (Collaborative Specialization)
- Internal Coordination/Regulation
- Sensing/Scanning
- Hypothesis/Goal Selection
- Mapping/Modeling
- Uncertainty Reduction/Learning
- Multi-Scale Data Fusion / Triangulation
- Trail-Marking / Externalized Memory (Stigmergy)
- Surprise / Change-Point Detection
- Frontier Outposts / Sensing Extension
- Evade Detection
- Overcome Barrier
- Neutralize Defenses
- Subversion & Co-optation
- Prepositioning & Latency
- Battlespace Shaping
- Leverage Host/Amplification
- Physical/Virtual Barrier
- Technique-Based Identification (a.k.a. Technique-Based Detection)
- Self-Monitoring
- Validation of Identity/Intention
- Deception
- Triage & Prioritized Allocation
- Quality Control & Error Correction
floor filled by (23)
- Cryptography
- Electromagnetic field
- Human (Homo sapiens)
- State (polity); the nation state its predominant modern form
- Government
- Law (rules and law)
- Legislatures and statutes
- Internet
- Computer
- Submarine communications cables
- Cryptanalysis (the attack side of the practice)
- Artificial intelligence
- Organization
- Telephone network (circuit-switched carrier)
- Traffic analysis
- Direction finding (HF/DF)
- Listening stations and collection platforms
- Intelligence analysis and analysts
- Signals agencies (NSA, GCHQ, ASD, CSE, GCSB, Unit 8200 and others)
- SIGINT satellite
- GCHQ
- Five Eyes (UKUSA)
- Foreign Intelligence Surveillance Court
Kinds (6)
- Intelligence information
- Information gathering
- Communications intelligence (COMINT)
- Electronic intelligence (ELINT)
- FISINT
- Cyber spectrum intelligence (SpecInt)
wired (1)
Notes published science
- The direction finder that hunted U-boats began as a lightning detector: Robert Watson-Watt built it from 1926 to locate lightning (he tracked thunderstorms 2500 km away), and in WWII high-frequency direction finding contributed to about a quarter (24%) of all U-boats sunk (src:wp:huff-duff).
- Britain's first naval act of WWI was cutting Germany's undersea cables, which forced German traffic onto British lines or radio; that is why the Zimmermann telegram, sent inside neutral diplomats' cables, ran over British wires and helped bring the US into the war (src:wp:zimmermann-telegram, src:danezis-clayton-2007-traffic-analysis).
- Codebreaking gave us the computer: the ten Colossi built to break German codes were the first programmable electronic computers, made by a Bletchley team that was 75% women (src:wp:signals-intelligence, src:wp:bletchley-park). And after the June 2013 revelations, views of privacy-sensitive Wikipedia articles dropped by over 30% at once (src:penney-2016-chilling-effects).
Layers
The ways it takes in its world, where they meet, and how it acts back, each from L1 (where it meets the world) up.
signals intelligence as intelligence historians and the signals agencies describe it: signals in the ether and on the wire up to the decisions they inform (reader's view)
- L1: Signals: radio, radar, telemetry, satellite links, cables and the records carriers keep; what can be taken depends on the medium
- L2: Collection: intercept stations, ships, aircraft, satellites, cable access, demands on providers
- L3: Processing: detection, direction finding, decryption, translation, filtering
- L4: Analysis: traffic analysis and content analysis; the electronic order of battle
- L5: Reporting and sharing: summaries to command, partners, cover for the source
- L6: Direction and oversight: requirements, priorities, law, warrants, oversight bodies
- L7: Decisions by consumers: governments and commanders
the boardroom (shared floors)
- L3: Processing: signal detection, direction-finding fixes, demodulation, decryption (cryptanalysis), translation, filtering and buffering
- L4: Analysis: traffic analysis, content analysis, the electronic order of battle, fusion; first- and second-line analysts; machine filtering
- L5: Reporting and sharing: intelligence summaries, controlled distribution (Ultra, cover stories), exchange with partners
- L6: Direction and oversight: requirements and targeting, priorities, the agencies' two missions (read others', secure one's own), law, warrants and oversight bodies
- L7: Decision and action by the consumers: commanders and governments act on what was read (a war entered, a convoy rerouted, an admiral's aircraft intercepted)
way in: Captured or passed cipher material (codebooks, machine details)
- L1: Codebooks and keys taken from wrecks, expeditions and allies: the SKM from SMS Magdeburg, Wassmuss's codebook, the Poles' Enigma details
- L2: Passed to the codebreakers and set beside the intercepts
way in: Communications satellite downlinks
- L1: International telephone, fax and data traffic relayed by geostationary satellites from 1966
- L2: Ground stations under the footprints (Yakima, Menwith Hill, Misawa): in principle all such traffic
way in: Consumers' needs and feedback (requirements)
- L1: An intelligence need of a commander or agency
- L2: A validated collection requirement
- L3: Interception points coordinated under limited resources
way in: Foreign instrumentation signals (telemetry, tracking, video links)
- L1: Telemetry, tracking systems and video data links from foreign weapons and tests
- L2: Collection for national technical means of verification of arms control
way in: Law, courts and oversight
- L1: Statutes and rulings: FISA 1978, Section 702 (2008), the Investigatory Powers Act 2016; courts holding that metadata collection itself interferes with privacy
- L2: Warrants: the FISC; IPC judges reviewing warrants before they come into force
- L3: Appeal courts and European courts testing programs: Second Circuit 2015, CJEU 2014 and 2020
- L4: Standing oversight bodies: the Senate Select Committee on Intelligence; IPCO with fifteen judicial commissioners
way in: Own side's emissions (communications security monitoring)
- L1: One's own radio traffic and emissions, which the enemy can hear too
- L2: Security monitors listening to friendly nets
- L3: BEADWINDOW categories: position, capabilities, operations, EW, key personnel, COMSEC, wrong circuit
way in: Partner agencies' collection (UKUSA, third parties)
- L1: Partners' own collection: the BND's intercepts, Sweden's FRA cable collection
- L2: Agreements: UKUSA (1946), automatic sharing among the Five Eyes, third parties in associate capacities
- L3: The exchange network (STONEGHOST)
way in: Radar and other non-communication emissions (ELINT)
- L1: Foreign non-communication electromagnetic radiation: radars, weapons control, IFF responses
- L2: Collection platforms: ground stations near the border, ships, aircraft, SIGINT satellites; each transmitter's hardware fingerprint
- L3: Signal identification: parameters matched against device libraries, or recorded as a new emitter
way in: Radio communications of the target (voice, Morse, radioteletype)
- L1: The target's radio transmissions in the open ether: no one's property, heard by anyone in range
- L2: Intercept receivers and operators: Y-stations, ships, aircraft; operators who copy Morse and know each sender's 'fist'
way in: Revelations, leaks and public reaction
- L1: Disclosures published: press reports before 1975, the 2005 New York Times story, Snowden's documents from June 2013
- L2: Public awareness: 87% of US adults had heard of the collection (Pew 2014)
- L3: Changed behaviour: an immediate drop of over 30% in views of privacy-sensitive Wikipedia articles
- L4: Investigations: troubling press reports led to the Church Committee (1975)
- L5: Reform statutes: the USA Freedom Act (June 2015) ended bulk phone collection
way in: Stored communications compelled from service providers
- L1: Stored internet communications held by companies (Microsoft, Google, Apple, Facebook and others)
- L2: Demands under Section 702 for data matching court-approved search terms
- L3: Already decrypted and easier to handle: what was encrypted on the backbone or discarded by filters
way in: Traffic data and call records (metadata)
- L1: Call detail records kept for billing: who called whom, when, for how long; densely interconnected, reidentifiable, revealing
- L2: Orders to carriers for all records (Verizon, 25 April 2013, 'on an ongoing daily basis')
- L3: Retention: carriers obliged to keep all customers' metadata up to two years (EU directive, struck down 2014)
way in: Traffic on wires and cables (telegraph, telephone, fibre)
- L1: Cable traffic, which needs physical access; in 1914 Britain cut Germany's cables so its traffic ran over British lines or radio
- L2: Access through the carriers or a tap: Western Union to the Black Chamber (1919-), SHAMROCK (1945 to early 1970s), Tempora on fibre
- L3: Buffering: content held three days, metadata thirty, while selectors pick what is kept
way in: Where a transmitter is (direction finding)
- L1: The emission itself, whenever the target transmits; ships weigh the value of sending against being found
- L2: Direction-finding sets: rotated loops (a minute or more), HF/DF arrays read on an oscilloscope (essentially instantaneous), Wullenweber 'elephant cages', time-of-arrival and interferometry
way out: Declassification and public release
- L1: Texts published
- L6: Decision to release: UKUSA's text (25 June 2010); FISC opinions under the USA Freedom Act
way out: Demands served on carriers and providers
- L1: Records and stored content turned over
- L2: Directive or order served (Verizon order; PRISM demands)
- L6: Court-approved search terms, orders
way out: Electronic attack: jamming, spoofing, meaconing
- L1: Radars jammed or deceived ('soft kill'), beacons retransmitted with false information
- L4: ESM gives the information electronic attack needs
way out: Protecting the source (cover stories)
- L1: Reconnaissance flights flown so the enemy sees a plausible finder
- L5: Controlled distribution; information credited to a fictional spy network (Boniface)
way out: Reports to consumers
- L1: The consumer's decision: a declaration of war, a convoy route
- L3: Shown to a foreign government when it serves (the Zimmermann telegram to the US embassy, 19 February 1917)
- L5: Intelligence summaries sent up (INTSUM)
way out: Securing one's own communications
- L1: Own traffic secured
- L3: Advice on and provision of codes and cyphers
- L6: The defensive mission: GC&CS's public function, NSA's protection task, GCHQ's NCSC
way out: Targeting for attack
- L1: Forces struck: U-boats sunk (24% with HF/DF), Admiral Yamamoto's aircraft intercepted (Operation Vengeance)
- L4: Target selection by traffic analysis; a DF fix
- L7: A decision to strike what was located or read
way out: Tasking collection (aiming and tuning)
- L1: Antennas and dishes pointed at a slice of the ether
- L2: Interceptors told where to aim antennas and how to tune receivers (PSTS)
- L6: A collection requirement and a decision to target