EVOPEDIA

← all systems · see it in The System

Penetration testing

An authorised simulated cyberattack on a computer system, performed live to evaluate its security (src:wp:penetration-test); in NIST's words, security testing in which assessors mimic real-world attacks to identify methods for circumventing the security features of an application, system, or network (src:nist-sp800-115). Walked as testers, their standards and researchers describe it: the engagement (scope, rules of engagement, timebox), reconnaissance and scanning, vulnerability analysis, exploitation and pivoting, reporting and re-testing; its kinds (vulnerability assessment, application and network tests, internal 'assumed breach' tests, OT tests, red teaming, bug bounties, automated and LLM-driven testing); its ethics and law; the people who do it.

Properties published scienceour proposal

Sourced fields, and our own readings.

born at1965 (first formal request at the SDC conference); tiger teams in the 1970s
born precisionyear
boundaryInside: the authorised engagement (scope and rules of engagement, the testers and their tools, reconnaissance, scanning, exploitation, pivoting, logs, the report and the re-test), and the standing practice (methodologies, checklists, training, the profession and its ethics). Outside: the client organisation and its systems and people (the target), the defenders it tests (blue team, SOC), real attackers it emulates, the vulnerability management that acts on its findings, vendors, law and regulators.
nested levelA practice inside computer and information security, a component of a full security audit (src:wp:penetration-test); one technique among NIST's target vulnerability validation techniques (src:nist-sp800-115); run by people (white-hat hackers) on an organisation's request.
scalesocial

Measures (21) published science

characteristic clock: penetration test cycle (NIST: annual may suffice)1 year
mandated minimum test frequency (FISMA)no less than annually per year
engagement effort: penetration test5-10 (1-2 testers) days
engagement effort: vulnerability assessment2-4 (1 tester) days
engagement effort: internal network and OT tests7-10 (1-2 testers) days
engagement effort: red teaming30+ (3-4 testers) days
tester attention span on one web test3-4 days before motivation degrades (one tester)
interviewed testers' work experiencemean 9.0, sd 6.5, median 8 (n = 12) years
time for injection attacks to become rare (testers' report)about 10 years
cybersecurity workforce growth vs gap growth11.1% vs 26.2% per year (ISC2 2022, as cited)
PentestGPT sub-task completion gain228.6% over GPT-3.5; 58.6% over GPT-4 percent
PentestGPT benchmark13 targets, 182 sub-tasks count
PentestGPT real-world machines solved4 of 10 HackTheBox machines for 131.5 USD of API use count
PentestGPT CTF placing24th of 248 teams (1500 of 4200 points in one passage, 1400 points in another) rank
bug bounty discovery decay exponent-0.40 (p < 0.001, R2 = 0.79; 35 HackerOne programs) power-law exponent of awards over time
bug bounty programs on the two largest platformsabout 3,300 programs
Forbes 500 without a disclosure program80 percent
bug bounty contributors studied2,504 across 82 programs count
AI red-team attacks released38,961 attacks
Hobbs's 1851 lock-pick payment20,000 (adjusted for inflation) USD
interviewed testers who named intuition when choosing where to look8 of 12 testers

Open contentions (6) open contention

Where sources or readings disagree. Both stay on the map.

Partners and parts published science

Built from / builds (7)

Sits inside / holds (3)

Trades with (valence) (14)

expressed by (20)

floor filled by (12)

Kinds (21)

Notes published science

Layers

The ways it takes in its world, where they meet, and how it acts back, each from L1 (where it meets the world) up.

penetration testing as testers and their standards describe it: the client's systems, people and premises in scope, up to the assurance the test gives (reader's view)

  1. L1: The target in scope: the client's systems, applications, people and premises, with their weaknesses
  2. L2: Reconnaissance and scanning: information gathering, port and service identification
  3. L3: Vulnerability analysis: scanned hosts compared with vulnerability databases and the testers' own knowledge; hypothesised flaws ranked
  4. L4: The attack: exploitation to confirm, chaining and pivoting, looping back to discovery
  5. L5: The engagement: scope, rules of engagement, approval, timebox, stop points, notification and escalation
  6. L6: The report and risk rating; re-test; the testers' quality control
  7. L7: Assurance: what the client learns of its security posture, bounded by presence-not-absence; the profession's integrity

the boardroom (shared floors)

  1. L4: The attack: exploitation to confirm vulnerabilities, chaining flaws and pivoting, attack choice (most cost-effective for an attacker first), mental models and the red team's map of the company
  2. L5: The engagement: scope and rules of engagement, management approval, timebox, stop points, coordination with the client, escalation of detected test activity (the white team)
  3. L6: Reporting and risk rating: logs and periodic reports during the test, the final report (vulnerabilities, risk rating, mitigation guidance), the re-test report; the testers' own quality control
  4. L7: Assurance and purpose: assurance about the system's security for the client, within the limit that a test shows presence not absence; the profession's integrity

way in: Known vulnerabilities and exploits

  1. L1: Public vulnerability databases (NVD) and exploit collections (Bugtraq, Metasploit's database of known exploits)
  2. L2: Versions of scanned services and software looked up, automatically by scanners or by hand
  3. L3: A list of hypothesised flaws ranked by the chance the flaw exists and the ease of exploiting it (flaw hypothesis methodology)

way in: Law and professional ethics

  1. L1: The law on computer access (the CFAA's 'exceeds authorized access'); consent is what separates white hats from others
  2. L2: Legal safe harbours valued by hunters; non-disclosure and limited liability in contracts
  3. L3: Professional ethics built on integrity: serve and protect the client, uphold the profession, avoid conflicts of interest and false results

way in: Mandates and standards

  1. L1: FISMA periodic testing no less than annually; PCI DSS penetration tests on a schedule and after changes; GSA's pre-vetted HACS services
  2. L2: The organisation's assessment policy: requirements, roles, methodology, frequency, documentation
  3. L3: Tests scheduled; methodologies (PTES, OSSTMM, NIST SP 800-115, ISSAF, OWASP) chosen

way in: People: the human element

  1. L1: Employees and their awareness of security
  2. L2: Pretexts, phone calls, phishing; staff names found in reconnaissance
  3. L3: Information revealed (for example a password), used to attack systems

way in: Reconnaissance: public information about the target

  1. L1: Open information about the target: host names and IP addresses, employee names and contact details on web and directory servers
  2. L2: Gathering: DNS interrogation, WHOIS queries, search engines (open source intelligence), and network sniffing on internal tests
  3. L3: A target profile: hosts and people, data usable later, for example in a social engineering attack

way in: Scanning: hosts, ports and services

  1. L1: The target's hosts, open ports and services
  2. L2: Port and service identification (Nmap) and vulnerability scanners; slowed to stay under the radar in covert tests
  3. L3: Scanned hosts compared with vulnerability databases; no tool covers an area, so several are run and merged by hand

way in: The application's behaviour: requests and responses

  1. L1: HTTP requests and responses, API operations and their documentation
  2. L2: An intercepting proxy (Burp, used by every web tester interviewed), crawlers, one test user per access group
  3. L3: A model of expected behaviour and access rules; deviations classified (vulnerability, input filter or harmless error?)

way in: The client's fixes, seen by re-testing

  1. L1: Items remediated by the client according to its risk appetite
  2. L2: The reported vulnerabilities attacked again
  3. L3: Remediation confirmed or not

way in: The client's request and what it hands over

  1. L1: The client's goals and written permission; what it provides (nothing but scope, credentials and architecture, or full access and source code)
  2. L2: The contract: scope, emergency contacts, source IP addresses, failsafe procedures; scope is what bounty hunters weigh most
  3. L3: The box chosen (black, gray, white); rate limits or firewalls disabled; dedicated test environments
  4. L4: Shortcuts built into the plan: breach assumed, local admin accounts provided, anti-virus disabled

way in: The community: new techniques and tools

  1. L1: Twitter/X, YouTube channels, blog posts, Reddit, GitHub, courses, conferences and colleagues
  2. L2: Reading, CTFs and write-ups; skills flow both ways between CTFs and work
  3. L3: New attack classes from security researchers trickle down to practitioners; community-extended tools evolve faster

way in: The defenders' response: detection by the blue team

  1. L1: Defenders' ability to detect attacks and respond
  2. L2: Red teams watch whether they are detected; honey-tokens on shares; some automated attacks 'too loud'
  3. L3: Tools and moves re-judged by their chance of detection; test traffic told apart from attacks by the named source IPs

way in: The premises: physical security

  1. L1: Fences, cameras, alarms, locks and employee behaviour at the site
  2. L2: Physical reconnaissance (people, places, security devices, weather); hardware tools for RFID, NFC, Sub-GHz radio and side channels
  3. L3: Weaknesses identified and an operational plan written, sent to the client for approval

way in: The target's condition during the test

  1. L1: Systems may be damaged or rendered inoperable; OT side effects can be catastrophic
  2. L2: Written logs kept; periodic reports to system administrators and management
  3. L3: Stability of the target judged before each exploit; 'unstable' attacks held back
  4. L4: The point where an additional action would cause damage

way in: The tester's experience and intuition (inner)

  1. L1: Experience from previous tests, CTFs, the same client or industry, and former work as a developer; where intuition comes from is contested
  2. L2: Intuition felt in the body: 'some organ that tingles if an operation looks fishy'; triggered by recently read exploits
  3. L3: Preconceptions about features (uploads, XML) and technologies (PHP) as likely weak

way out: Disclose new flaws to vendors

  1. L1: The flaw reported to the vendor
  2. L5: Responsible disclosure requests, met with resistance by some ICS suppliers; testers averse after time-wasting responses
  3. L6: A new vulnerability found in a vendor's product

way out: Exploit and pivot

  1. L1: The target's state changed: access held, or the network's security state altered
  2. L2: Access gained, privileges escalated, other machines reached (pivoting)
  3. L3: Payload delivered (Metasploit automates attacks on known vulnerabilities)
  4. L4: An attack chosen to verify a hypothesised flaw

way out: Leave things as found

  1. L1: Lights, locks, keyboards and chairs left as they were found
  2. L3: All equipment brought in is listed and checked out
  3. L5: Clean-up part of the plan

way out: Notify at once

  1. L1: The client acts before the test ends
  2. L3: Notification with countermeasures sent to the client's emergency contact
  3. L5: A highly vulnerable flaw found, for example one endangering human life

way out: Physical entry

  1. L1: Entry to a restricted area (a server room, an executive's office)
  2. L2: Security devices identified and defeated with tools and techniques
  3. L4: Operational plan approved by the client

way out: Raise the blue team's skills

  1. L1: Blue team skills raised; beachheads found and attackers evicted
  2. L2: Detection rehearsed against repeated attacks
  3. L6: Findings on detection shared; purple teams repeat attacks so the blue team can tune

way out: Report and recommend

  1. L1: The client remediates by its risk appetite
  2. L5: Results presented to the organisation's managers
  3. L6: Report: executive summary, vulnerabilities classified by risk matrix, remediation recommendations; social engineering results only in summary

way out: Social engineering

  1. L1: An employee tricked into revealing information
  2. L3: Delivered by hand: fully automated tools avoided for fear of detection and ethical qualms
  3. L4: A pretext chosen from reconnaissance

Sources (21) published science

Penetration test (Wikipedia, via Wikimedia Enterprise snapshot 2026-11-01)
Scarfone K, Souppaya M, Cody A, Orebaugh A (2008). NIST SP 800-115: Technical Guide to Information Security Testing and Assessment. NIST (behind its links)
Happe A, Cito J (2023). Understanding Hackers' Work: An Empirical Study of Offensive Security Practitioners. ESEC/FSE '23. doi:10.1145/3611643.3613900 (arXiv 2308.07057, author's version) (behind its links)
Pierce JD, Jones AG, Warren MJ (2006). Penetration Testing Professional Ethics: a conceptual model and taxonomy. Australasian Journal of Information Systems 13(2). doi:10.3127/ajis.v13i2.52 (behind its links)
Maillart T, Zhao M, Grossklags J, Chuang J (2017). Given enough eyeballs, all bugs are shallow? Revisiting Eric Raymond with bug bounty programs. Journal of Cybersecurity 3(2):81. doi:10.1093/cybsec/tyx008 (arXiv 1608.03445v2 read; journal page refused the request) (behind its links)
Happe A, Cito J (2023). Getting pwn'd by AI: Penetration Testing with Large Language Models. ESEC/FSE '23. doi:10.1145/3611643.3613083 (arXiv 2308.00121) (behind its links)
Deng G et al. (2024). PentestGPT: Evaluating and Harnessing Large Language Models for Automated Penetration Testing. USENIX Security 24 (arXiv 2308.06782v2) (behind its links)
Akgul O et al. (2023). Bug Hunters' Perspectives on the Challenges and Benefits of the Bug Bounty Ecosystem. USENIX Security '23 (arXiv 2301.04781) (behind its links)
Hata H, Guo M, Babar MA (2017). Understanding the Heterogeneity of Contributors in Bug Bounty Programs. ESEM 2017 (arXiv 1709.06224) (behind its links)
Ganguli D et al. (2022). Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned (arXiv 2209.07858) (behind its links)
Bug bounty program (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Red team (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Computer Fraud and Abuse Act (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Social engineering (security) (Wikipedia, via Wikimedia Enterprise snapshot 2026-11-01) (behind its links)
Dittrich D, Kenneally E et al. (2012). The Menlo Report: Ethical Principles Guiding Information and Communication Technology Research. US DHS Science and Technology Directorate (behind its links)
NIST (2022). SP 800-53A Rev. 5: Assessing Security and Privacy Controls in Information Systems and Organizations. doi:10.6028/NIST.SP.800-53Ar5 (behind its links)
Feffer M, Sinha A, Deng WH, Lipton ZC, Heidari H (2024). Red-Teaming for Generative AI: Silver Bullet or Security Theater? (arXiv 2401.15897v3) (behind its links)
Basiri A et al. (2016). Chaos Engineering. IEEE Software 33(3):35-41. doi:10.1109/MS.2016.60 (arXiv 1702.05843) (behind its links)
Klemme I, Hyvärinen P, Karvonen A (2025). Survival benefits of antipredator training in captive-reared salmon: impact of behaviour, health, and genetics. Oecologia (PMC12592309) (behind its links)
Iwasaki A, Omer SB (2020). Why and How Vaccines Work. Cell 183(2):290-295 (PMC7560117) (behind its links)
Capture the flag (cybersecurity) (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)