← all systems · see it in The System
Secure software development
The practice of developing software so that it guards against the accidental introduction of security vulnerabilities and resists misuse (src:wp:secure-coding): walked as the whole practice as practitioners and researchers describe it, from threat modelling and secure design through secure coding, memory-safe languages, code review, static analysis and fuzzing, to protecting the release and responding to the vulnerabilities that remain (NIST SSDF's four groups), with the people, organisations, incentives and laws that carry it.
Properties published scienceour proposal
Sourced fields, and our own readings.
| born at | 1974-1975 (Saltzer and Schroeder; manuscript received October 1974, revised 1975) |
| born precision | year |
| boundary | Inside: the practices that build software to resist misuse: security requirements and threat modelling, secure design principles, secure coding and memory-safe languages, code review, static analysis, fuzzing and other security testing, protecting the build and release (signing, bills of materials), responding to residual vulnerabilities and fixing their root causes, and the organisational processes and training that carry them (SSDF, SDL, DevSecOps). Outside: the software itself, the vulnerabilities in it, the attackers, the operating systems and languages it builds on, the computer and information security operations that detect and respond after deployment, markets and law. |
| nested level | A practice carried by people and organisations (rung 12, technological and social), working on software; the build-time part of computer security, framed by information security's goals and risk methods. |
| scale | technological |
Measures (27) published science
| characteristic clock: average vulnerability lifetime in code (half-life shape) | ~4 (Chromium ~2, OpenSSL ~7) years |
| median time-to-fix, OSS-Fuzz bugs | 5.3 (flaky bugs 43) days |
| unfixed fuzzer bugs: median age at collection | 437 (95% between 90 and 1275) days |
| disclosure deadline (OSS-Fuzz) | 90 days after discovery, or 30 days after the patch |
| OSS-Fuzz bugs studied | 23,907 in 316 projects bugs |
| memory-safety share of Android vulnerabilities | 76% (2019) to 24% (2024) |
| memory-safety share of Chromium high-severity security bugs | ~70 % |
| vulnerability density, 5-year-old code vs new code | 3.4x to 7.4x lower ratio |
| Rust memory-safety bugs studied (all Rust memory-safety CVEs to 2020-12-31 and more) | 186; all but one compiler bug need unsafe code bugs |
| commit-to-merge time vs vulnerability lifetime | ~20 days vs years |
| Copilot programs vulnerable in high-risk CWE scenarios | ~40 of 1,689 programs (89 scenarios) % |
| AI-assistant user study | 47 participants, 5 tasks; 87% of secure answers needed significant edits participants / % |
| single SAST tool: vulnerable commits with a warning in the vulnerable function | 52 (combined tools 78; at least 76% of warnings irrelevant; 22% undetected) |
| formal inspection latent-defect discovery rate (testing ~30%) | 60-65 % |
| optimal code review rate | 200-400 lines of code per hour |
| code review: respondents who say finding vulnerabilities is their responsibility | 81 (126 of survey) % |
| malicious packages: share typosquatting; mean edit distance to target | 61%; 2.3 % ; Levenshtein distance |
| malicious package availability before public report (mean; median 67) | 209 days |
| open source in commercial codebases | 96% contain it; 70-90% of a typical codebase |
| malicious packages studied | 174 (npm, PyPI, RubyGems, Nov 2015 - Nov 2019) packages |
| SolarWinds malicious update installs | >18,000 organisations |
| Miller's 1990 fuzz study: utilities crashed | 25-33 % |
| bugs expected in a 35-million-line program (security-critical share) | 1,000,000 (perhaps 1%) bugs |
| SSDF practice groups | 4 (PO, PS, PW, RV) groups |
| position papers received for the SSDF update | >150 papers |
| secure development methodologies reviewed | 28 methodologies |
| kinetic proofreading cost (cross-domain, tRNA charging): ATP per charge, right vs wrong amino acid | 1.5 vs 270; error reduced by a factor of 1/180 ATP per tRNA |
Open contentions (4) open contention
Where sources or readings disagree. Both stay on the map.
- C-413 Where do thoughts come from?
booked: No theory is proven, so the map lists the main ones side by side and doesn't pick one. It shows where thinking is handled in the brain, not where thoughts start.
the other reading: see Human walk: where thoughts come from is contested - C-1515 Do code-scanning security tools mostly raise false warnings?
booked: The map books most warnings as unhelpful, and keeps a study that found them precise.
the other reading: SAST tools show high precision and low recall on a synthetic Java codebase (src:esposito-2024-sast: 'In contrast to prevailing wisdom') - C-1518 Does secure software development work?
booked: The map books measured successes, and keeps the view that the methods as a whole are unproven.
the other reading: methodology-level effectiveness unproven: 'lack of evidence supporting effectiveness of these methodologies, often based on the belief that they reduce the number of vulnerabilities' (src:kudriavtseva-2022-ssdm); the Android decline is the language shift, not the methodology as a whole - C-1519 When did secure software development begin?
booked: The map books the 1975 design principles, and keeps Microsoft's 2004 process and the fuzzing of 1988-90 as other starts.
the other reading: born as a named practice with Microsoft's SDL (2004) and secure coding standards; born when defenders took up the attackers' tools (fuzzing 1988-1990)
Partners and parts published science
Built from / builds (13)
- The race to find a program's bugs first
- Secure coding
- Secure by design
- Code review
- Vulnerability disclosure and response
- Software supply-chain security
- Threat modelling
- Fuzzing
- Static application security testing
- Secure design principles
- Memory safety (memory-safe languages)
- The war in memory (offence vs defence over memory-corruption bugs)
- Competition for trust in package registries (malicious packages vs developers and registries)
Sits inside / holds (2)
Trades with (valence) (14)
- Threat actor via sys:software · Secure software development mixed
- Human (Homo sapiens) via sys:human · Secure software development gain
- Law (rules and law) via sys:software · Secure software development gain
- Software via sys:software · Secure software development gain
- Large language model via sys:software · Secure software development mixed
- Operating system via sys:software · Secure software development gain
- Market and trade via sys:software · Secure software development loss
- Programming language via sys:software · Secure software development gain
- Computer security via sys:software · Secure software development gain
- Information security via sys:software · Secure software development gain
- Organization via sys:software · Secure software development gain
- Software vulnerability via sys:software · Secure software development gain
- Supply chain security via sys:software · Secure software development gain
- Vulnerability management via sys:software · Secure software development gain
expressed by (22)
- Scaffolding/Enablement (Meta-Tooling)
- Templating / Standardization
- Quality Improvement/Refinement
- Encapsulation & Modularity (Abstraction)
- Sensing/Scanning
- Mapping/Modeling
- Uncertainty Reduction/Learning
- Active Experimentation / Probing (Optimal Experimental Design)
- Trail-Marking / Externalized Memory (Stigmergy)
- Social Learning / Public Information Use
- Reactive Response
- Proactive Defense
- Physical/Virtual Barrier
- Technique-Based Identification (a.k.a. Technique-Based Detection)
- Self-Monitoring
- Validation of Identity/Intention
- Validation of Effectiveness
- Enclave/Segregation
- Resilience & Repair
- Redundancy & Diversity
- Triage & Prioritized Allocation
- Quality Control & Error Correction
floor filled by (15)
- Human (Homo sapiens)
- Law (rules and law)
- Software
- Large language model
- Operating system
- Programming language
- Computer security
- Threat actor
- Information security
- Organization
- Common Weakness Enumeration (CWE)
- Threat modelling
- Fuzzing
- Static application security testing
- Supply chain security
Kinds (6)
- Software engineering
- Secure by design
- Microsoft Security Development Lifecycle
- NIST SSDF (SP 800-218)
- DevSecOps
- Safe Coding (Google)
Notes published science
- The flaw that secure coding fights hardest is a carry gone wrong: a buffer overflow is data that won't fit in its slot and spills into the next one, exactly like a sum spilling into the next column, except here the spill overwrites something it shouldn't (src:wp:secure-coding).
- Google found that old code heals itself: vulnerabilities in a code base decay exponentially with a half-life, so when Android stopped writing new memory-unsafe code, its memory-safety bugs fell from 76% to 24% of all vulnerabilities in six years even though the old unsafe code kept growing (src:google-2024-android-memsafe).
- In 1984 Ken Thompson showed that a compiler can plant a back door in every program it builds, including new copies of itself, with no trace in any source code; his moral, 'No amount of source-level verification or scrutiny will protect you from using untrusted code', is a hard ceiling on what code review can ever know (src:thompson-1984).
Layers
The ways it takes in its world, where they meet, and how it acts back, each from L1 (where it meets the world) up.
secure software development as practitioners describe it: the code, components and threats it takes in, up to software that resists misuse (reader's view)
- L1: The material the practice faces: code being written (by people and AI assistants), third-party components, inputs that cross trust boundaries, and attackers
- L2: Taking it in: review by people, static analysis, fuzzing, threat elicitation, disclosure programmes, dependency intake
- L3: Findings: weaknesses, warnings, crashes and threats, triaged; most come from a few common error kinds
- L4: The shared risk picture: threat model and security requirements
- L5: Design and implementation decisions: principles (least privilege, fail-safe defaults, complete mediation), memory-safe languages, safe APIs, mitigations
- L6: The organisation's process: SSDF, SDL, DevSecOps (shift left), training, protecting the software
- L7: Purpose: software that resists misuse; residual vulnerabilities answered and their root causes removed
the boardroom (shared floors)
- L4: The shared picture of risk: the threat model and security requirements, where findings from review, tools, fuzzing, reports and attack research meet
- L5: Design and implementation decisions: secure design principles, memory-safe languages, safe APIs, mitigations, fixes
- L6: The organisation's process: SSDF practice groups, life-cycle methods (SDL, DevSecOps, 28 methodologies), training, gates, segmentation; obligations from law and market
- L7: Purpose and response: software that resists misuse, residual vulnerabilities answered, root causes fixed so they do not recur
way in: Attacks and attack research
- L1: Attackers and offensive researchers develop new attacks; penetration exercises beat every general-purpose system tried (1970s)
- L2: Published as attack techniques and incidents (1972 overflow study; Morris worm 1988)
- L3: Modelled: attack building blocks toward exploitation
way in: Code review (people reading code)
- L1: The change put up for review; at least one reviewer is not the author
- L2: Read by people at 200-400 lines per hour (sight, thought)
- L3: Findings: formal inspections find 60-65% of latent defects; security rarely the first focus unless asked
way in: Code suggested by AI assistants
- L1: Suggestions from a model trained on unvetted public code
- L2: Accepted or edited by the developer: 87% of secure answers needed significant edits
- L3: In the code: about 40% of 1,689 Copilot programs vulnerable; assisted users wrote less secure code and believed it secure
way in: Fuzzing (the running program probed with inputs)
- L1: Inputs that cross a trust boundary
- L2: Generated and mutated inputs; grey-box fuzzers evolve a seed pool by coverage
- L3: Crashes triaged into bugs: 23,907 bugs in 316 OSS-Fuzz projects, found in punctuated spikes
way in: Money, markets and law
- L1: Markets and law: bad security products drive out good; liability largely ineffective for software
- L2: Rules arrive: Executive Order 14028 led to the SSDF update; liability waivers for audited vendors proposed
- L3: Obligations: the vendor faces an audit or liability
way in: New code: what developers write
- L1: Source code as it is written: in memory-unsafe languages (C, C++) pointers are raw addresses with no bounds checks; a 35-million-line program may hold a million bugs
- L2: The change: a commit or diff; vulnerabilities sit mostly in new or recently changed code
- L3: Its flaws age out: vulnerability density falls exponentially with code age (half-life); average lifetime about 4 years
way in: Static analysis (SAST)
- L1: Source code and components, white box
- L2: Rules matched against the code
- L3: Warnings: one tool warns in 52% of vulnerable commits, 76% of warnings irrelevant, combined tools 78%
way in: Third-party components (the supply chain)
- L1: Components, libraries, tools and processes; 96% of commercial codebases contain open source, 70-90% of a typical codebase
- L2: Resolved and installed by dependency managers, install scripts run automatically; 61% of malicious packages typosquat
- L3: Inventory and detection: a bill of materials; malicious packages live 209 days on average before reported
way in: Threat modelling (the system and its likely attackers)
- L1: The system as designed, its assets, the probable attacker
- L2: Threats elicited: STRIDE checklists, attack trees, data-flow diagrams; with people, a set of evolving practices
- L3: Threats enumerated and prioritised
way in: Vulnerability reports from outside
- L1: Information from acquirers, users and public sources; in Chromium about 70% of high-severity bugs are memory unsafety
- L2: Received through a vulnerability disclosure programme
- L3: Analysed for root causes
way in: What developers know (inner)
- L1: Training and knowledge: most companies give no security training yet expect security in review
- L2: Recall while writing: insecure practices and their secure alternatives taught to developers
- L3: Choices made: most vulnerabilities come from a small number of common programming errors
way out: Release protected software
- L1: Delivered to acquirers through trusted channels (which supply-chain attackers also use)
- L3: Signed and verifiable: code signing and release integrity
- L6: Protect the software: all components kept from tampering
way out: Respond: patch and advise
- L1: The code on machines updated (the first generation: reactive patching)
- L5: Disclosure by deadline: fixed bugs disclosed 30 days after the patch or at 90 days
- L7: Respond to residual vulnerabilities
way out: Train the people
- L1: What developers know (the c-knowledge channel)
- L3: Insecure practices and secure alternatives taught
- L6: Role-based training for all who contribute
way out: Write and fix the code
- L1: The code changed: bounds checked, sizes limited (strncpy for strcpy), stack layout randomised
- L3: Merged: fuzzer-found bugs fixed with a median of 5.3 days
- L4: Enforced by the toolchain: the borrow checker at compile time; unsafe code marked and owned by the developer
- L5: Decision: a safe construct, a memory-safe language for new code, a fix