EVOPEDIA

← all systems · see it in The System

Security audit and compliance

The checking of an organisation's or product's security against standards, laws and contracts by a party that did not build it: an independent review and examination of system records, activities and related documents (src:wp:information-security-audit), done by examining, interviewing and testing (src:nist-sp800-53a-r5), reported as findings, and turned into a decision others rely on: a certificate (ISO/IEC 27001, Common Criteria), an attestation report (SOC 1/2/3), an authorization to operate (src:nist-sp800-37-r2), a compliance validation (PCI DSS) or a regulator's fine (GDPR). Compliance is the goal it checks for: 'conforming to a rule, such as a specification, policy, standard or law' (src:wp:regulatory-compliance). Walked as auditors, assessors, certification schemes and regulators describe it, with the research on whether it works kept beside it.

Properties published scienceour proposal

Sourced fields, and our own readings.

born atcontested (see C-1775)
born precisioncontested
boundaryInside: the auditors and assessors (internal auditors, external audit firms, certification bodies and their audit teams, evaluation laboratories, approved scanning vendors, inspectors of regulators), their methods (examine, interview, test; sampling; depth and coverage), the criteria they check against once selected (scope, objectives, determination statements), their findings and reports, and the decision bodies that turn findings into certificates, attestations, authorizations or penalties. Outside: the audited organisation and its people (each a person with their own channels), the systems and records examined, the standards bodies and legislatures that write the criteria, accreditation bodies that check the checkers, and the relying parties (customers, acquirers, card brands, regulators, the public) who act on the result.
nested levelA practice inside the oversight floor of governance: part of the monitor-and-evaluate loop of security policy and governance (its audit channel), and of the audit-review step of information security's monitoring. Law, standards and contracts shape it from outside; certification schemes and regulators run it at the scale of whole markets.
scalesocial

Measures (23) published science

characteristic clock: ISO/IEC 27001 surveillance auditat least 1 per calendar year (first within 12 months)
characteristic clock: certification cycle (ISO/IEC 27001; UN R155 CSMS)3 years
clock: PCI DSS self-assessment questionnaire and external scan4 per year (quarterly, all merchant levels)
Common Criteria certificates crawled1631 active, 3725 archived certificates
time and cost to obtain a product certificate (CC, FIPS 140)months or even years; hundreds of thousands of dollars or more time; USD
CC certificates with at least one known vulnerability88 of 1631 active; 274 of 3725 archived certificates
correlation of EAL with average CVE base score-0.28 Spearman rho (no positive association found)
delay of one fix after certification3 years (maintenance update fixing 3 CVEs disclosed before certification)
PCI scanners fully compliant with ASV guidelines0 of 6 scanners
PCI scanner prices$250 to $2,995 USD per year
e-commerce sites with at least one PCI DSS violation94 % of 1,203
e-commerce sites with a violation that should disqualify them86 % of 1,203
per-hand-off efficiency, merchant compliance claim re-checked independently (Payment to Entropy)about 0.14 share of scanned sites whose compliance survives (derived: 100% - 86%)
organisations not compliant at interim validation (Verizon 2018)47.5 %
Heartland breach (validated compliant)more than one hundred million card numbers
sum of GDPR fines analysedabout 1,839 million (max 746 million; mean 2,149,000) EUR
GDPR fines analysed856 fines
GDPR fines rooted in organisational vs technical problems601 vs 314 fines
ISO/IEC 27001 studies citing outcomes of certification26 (half with empirical evidence) % of reviewed studies
ISO/IEC 27001:2022 Annex A controls93 controls in four themes
reduction in incident probability from adopting six basic precautions (riskiest Israeli firms)80 to 42 % probability
CISOs and executives interviewed on investment drivers40 people
one ISO certification practice in its first yearover 1 million USD revenue; more than 20 certifications USD; certifications

Open contentions (5) open contention

Where sources or readings disagree. Both stay on the map.

Partners and parts published science

Built from / builds (8)

Sits inside / holds (3)

Trades with (valence) (15)

expressed by (14)

floor filled by (12)

Kinds (14)

wired (2)

Notes published science

Layers

The ways it takes in its world, where they meet, and how it acts back, each from L1 (where it meets the world) up.

security audit and compliance as auditors, assessors and regulators describe it: the audited organisation's systems, people and records, up to the assurance others rely on (reader's view)

  1. L1: The audited organisation's systems, people, records, facilities and products: technical, physical and administrative controls
  2. L2: Evidence taken in by three methods: examine (documents, records, configurations), interview (people), test (mechanisms exercised, scans, penetration tests); plus evidence reused from earlier assessments and operations
  3. L3: Criteria, scope and plan: preliminary assessment of time, cost and scope; audit objectives; criteria from standards, laws and contracts; depth and coverage (basic, focused, comprehensive) chosen to the assurance required
  4. L4: Evidence weighed against determination statements: each yields 'satisfied' or 'other than satisfied'; sampling and point-in-time readings
  5. L5: Findings reported: exit meeting with the audited, the report (procedures, findings, recommendations, rankings high/medium/low), corrective actions and plans of action and milestones
  6. L6: The decision: a certification body's certification decision, an authorizing official's authorization to operate, an attestation opinion, a validation of compliance, or a regulator's fine
  7. L7: Assurance relied on by others: customers, acquirers, card brands, procurers, regulators and the public act on the certificate or report instead of checking for themselves; accountability for claims

the boardroom (shared floors)

  1. L4: Evidence weighed against criteria: each determination statement yields 'satisfied' or 'other than satisfied' (which can also mean the assessor could not obtain enough information); sampling at a point in time
  2. L5: Findings reported: exit meeting, report dated at completion, recommendations ranked, corrective action plans; the audited answers
  3. L6: Decision: certify or not, authorise to operate (accepting residual risk), issue the attestation opinion, validate compliance, or fine; the decision maker also judges whether assessor independence suffices
  4. L7: Assurance and accountability: relying parties (customers, acquirers, card brands, procurers, the public) act on the certificate or report; certification confers market access or liability shields; failed claims bring penalties

way in: Audit trails and operational evidence (logs, incident records, continuous monitoring)

  1. L1: Audit trails: chronological records written by a privileged process that users cannot stop or change
  2. L2: Operational evidence collected: remediation records, incident reports, continuous monitoring results
  3. L3: Read at organisation-defined frequencies for ongoing assessment

way in: Complaints and breach reports (the regulator's way in)

  1. L1: Data subjects, employees and other companies complain; breaches become public
  2. L2: The data protection authority receives and opens an investigation
  3. L3: The issue categorised (organisational or technical; which GDPR article)

way in: Earlier assessments and certificates reused (product certificates, third-party evaluations, previous reports)

  1. L1: Prior assessment results, validation reports and certificates for components
  2. L2: Accepted when owners and authorizing officials agree; reciprocal acceptance across agencies
  3. L3: References between certificates (composite and updated products) carry earlier findings forward, including vulnerabilities

way in: Examine: documents, policies, records, configurations

  1. L1: System records, activities and related documents; policies and procedures documented at the site
  2. L2: The examine method: reviewing, inspecting, observing, studying or analysing assessment objects
  3. L3: Depth and coverage of examination set to the assurance required

way in: Interview: hearing the people (the audit as hearing, audire)

  1. L1: The organisation's people and what they say about how the work is done
  2. L2: The interview method: discussions with individuals or groups to understand, clarify or obtain evidence
  3. L3: What is heard is checked against records and observation (meet IT management, review job descriptions)

way in: Laws and regulations that mandate audits or set the rules checked (HIPAA, SOX, GDPR, UN R155, state laws)

  1. L1: Statutes and regulations: in certain regulated industries audits are mandated by law
  2. L2: The organisation and its auditors become aware of the requirement (regulatory compliance as the goal of being aware and taking steps)
  3. L3: The requirement turned into audit criteria and scope (e.g. SOX section 404 internal control; UN R155 CSMS requirements)

way in: Observe: the auditor's site visit (sight)

  1. L1: Premises and equipment: locked cages, man traps, single entrances, environmental controls
  2. L2: Travelling to the site and observing processes
  3. L3: Equipment inspected and checked against utilization reports and downtime records

way in: Standards, control catalogues and contracts (ISO/IEC 27001 Annex A, PCI DSS, Trust Services Criteria, protection profiles)

  1. L1: Published standards and contracts: PCI DSS v1.0 (2004) by the card brands; ISO/IEC 27001 (2005, 2013, 2022); protection profiles for classes of product
  2. L2: Selected for the engagement: Security Target with chosen SFRs and SARs; Statement of Applicability against Annex A
  3. L3: Turned into determination statements and assessment procedures tailored to the system

way in: Test: mechanisms exercised (scans, penetration tests, configuration checks)

  1. L1: The live systems and products: networks, servers, web applications, certified devices
  2. L2: Tests run: approved scanning vendors' external scans (quarterly for PCI), lab evaluation of products, penetration testing
  3. L3: The non-intrusive, low-interaction constraint limits what can be tested on live production sites

way in: The auditor's own judgement and independence (inner channel)

  1. L1: The auditor as a person with interests: paid by the audited, competing for clients
  2. L2: Independence held by professional standards (IIA, AICPA) and judged by the authorizing official

way in: The engagement request (client, acquirer, procurer or authorizing official asks for the audit)

  1. L1: A relying party requires a report: a merchant must submit its compliance report to its acquirer; CC certification specified in procurement
  2. L2: Preliminary assessment of maturity, time, cost and scope
  3. L3: Planning and preparation: objectives, manpower, timeline; Stage 1 readiness review agrees the audit plan

way out: Certificate, attestation report or authorization issued to relying parties

  1. L1: Relying parties keep accounts active, buy, procure or admit to market (a merchant keeps its account with the acquirer)
  2. L3: Report written for its audience: SOC 1/2 for a limited audience, SOC 3 for the public; review reports state 'limited assurance'
  3. L6: Certification decision or authorization to operate

way out: Certification refused, withdrawn or left to expire

  1. L1: The product or organisation loses the standing the certificate gave (procurement, market access)
  2. L6: The scheme ends a certificate's validity (archived certificates; recertification needed before expiry)

way out: Findings and corrective actions back to the audited

  1. L1: Controls changed in the audited systems
  2. L3: Owners decide: remediate, or accept a finding that presents no significant risk; plans of action and milestones
  3. L5: Findings and suggested corrective actions communicated in a formal meeting

way out: Fines and penalties

  1. L1: Organisations adjust how they handle data, organisationally and technically
  2. L3: Fines published (enforcement trackers) and passed along contracts
  3. L6: A fine imposed: by a data protection authority (GDPR), or by card brands to the acquirer who passes it to the merchant

Sources (25) published science

Information security audit (encyclopedia)
ISO/IEC 27001 (encyclopedia) (behind its links)
NIST (2018). SP 800-37 Rev. 2: Risk Management Framework for Information Systems and Organizations. doi:10.6028/NIST.SP.800-37r2 (behind its links)
Saemann M., Theis D., Urban T., Degeling M. (2022). Investigating GDPR Fines in the Light of Data Flows. Proceedings on Privacy Enhancing Technologies 2022(4):314-331 (behind its links)
Rahaman S., Wang G., Yao D. (2019). Security Certification in Payment Card Industry: Testbeds, Measurements, and Recommendations. CCS '19. doi:10.1145/3319535.3363195 (behind its links)
LSE (n.d.). Reforming auditing and risk management to improve governance (research impact case study on Michael Power's The Audit Explosion, 1994, and The Audit Society: Rituals of Verification, 1997) (behind its links)
Sarbanes-Oxley Act (encyclopedia) (behind its links)
NIST (2022). SP 800-53A Rev. 5: Assessing Security and Privacy Controls in Information Systems and Organizations. doi:10.6028/NIST.SP.800-53Ar5 (behind its links)
Conformity assessment (encyclopedia) (behind its links)
Audit (encyclopedia) (behind its links)
Janovsky A., Jancar J., Svenda P., Chmielewski L., Michalik J., Matyas V. (2024). sec-certs: Examining the security certification practice for better vulnerability mitigation. arXiv:2311.17603v2 (behind its links)
Payment Card Industry Data Security Standard (encyclopedia) (behind its links)
Culot G., Nassimbeni G., Podrecca M., Sartor M. (2021). The ISO/IEC 27001 information security management standard: literature review and theory-based research agenda. The TQM Journal 33(7):76-105. doi:10.1108/TQM-09-2020-0202 (CC BY 4.0) (behind its links)
Gandal N., Moore T., Riordan M., Barnir N. (2022). Empirically Evaluating the Effect of Security Precautions on Cyber Incidents (WEIS 2022; CEPR DP17605; NSF PAR copy) (behind its links)
Moore T., Dynes S., Chang F. R. (2015). Identifying How Firms Manage Cybersecurity Investment. Darwin Deason Institute, SMU (interview study of 40 executives) (behind its links)
Sonkar N. (2025). Scaling an ISO Compliance Practice: Strategic Insights from Building a $1M+ Cybersecurity Certification Line. arXiv:2505.11583 (practitioner case study) (behind its links)
System and Organization Controls (encyclopedia) (behind its links)
Blocki J., Christin N., Datta A., Procaccia A. D., Sinha A. (2013). Audit Games. arXiv:1303.0356 (IJCAI 2013) (behind its links)
Regulatory compliance (encyclopedia) (behind its links)
Audit trail (encyclopedia) (behind its links)
US Department of Defense (1985). Trusted Computer System Evaluation Criteria (DoD 5200.28-STD, the Orange Book) (behind its links)
General Data Protection Regulation (Wikipedia, via Wikimedia Enterprise) (behind its links)
Oberti F., Abrate F., Savino A., Parisi F., Di Carlo S. (2024). Navigating the road to automotive cybersecurity compliance. arXiv:2407.00483 (behind its links)
Common Criteria (encyclopedia) (behind its links)
Information technology audit (encyclopedia) (behind its links)