Every SaaS vendor’s security page says the same thing: “We’re SOC 2 certified.” Sometimes they add the ISO 27001 badge for extra credibility. Procurement teams see the badges, check the box, and move on. Nobody reads the actual report. And if they did, most wouldn’t understand what it says — or more importantly, what it doesn’t say. A security audit report is not a guarantee of security. It’s an opinion, issued by an auditor, about whether specific controls were in place during a specific period. That’s a much weaker statement than “this company is secure,” and the gap between the two is where the real risk lives.

The TLDR

Security audits are systematic, independent evaluations of an organization’s security controls against a defined standard or framework. SOC 2, ISO 27001, PCI DSS, HITRUST — each has its own scope, methodology, and deliverable. A passing audit means the organization demonstrated that specific controls existed and operated effectively during the audit period, as verified through sampling. It does not mean every control works perfectly all the time. It does not mean the organization is immune to breaches. And it absolutely does not mean the auditor tested every system, reviewed every configuration, or validated every claim. Understanding what audits actually assess — and what they miss — is essential to making informed trust decisions.

The Reality

SolarWinds was SOC 2 compliant when attackers compromised its Orion build system and distributed malicious updates to 18,000 organizations. Equifax maintained security certifications when 147 million records walked out the door. The audit passed. The breach happened anyway. This isn’t a contradiction — it’s a feature of how audits work.

Audits are sample-based. An auditor doesn’t review every access control decision your organization made in the last twelve months. They pull a sample — maybe 25-50 items from a population of thousands — and verify that those samples comply with the stated control. If the sample passes, the control passes. Statistically, that’s reasonable. Practically, it means a control can be failing 10% of the time and still pass the audit if the sample happens to miss the failures.

Audits also test against defined criteria. SOC 2 evaluates your controls against the AICPA Trust Services Criteria. ISO 27001 evaluates against Annex A controls from ISO/IEC 27001:2022. If your biggest risk isn’t covered by the criteria — or if the auditor’s scope didn’t include the system where the risk lives — the audit won’t find it. Auditors test what’s in scope. Attackers don’t care about scope.

How It Works

SOC Reports: The Alphabet Soup

SOC — System and Organization Controls — reports are issued under the AICPA framework. There are three types, and they serve different audiences.

SOC 1 evaluates controls relevant to financial reporting. This is for your financial auditor, not your security team. If a service provider processes transactions or handles financial data, their SOC 1 report tells your auditors whether the provider’s controls support accurate financial statements. Security folks can mostly ignore SOC 1 unless financial data integrity is the specific concern.

SOC 2 evaluates controls against five Trust Services Criteria: Security, Availability, Processing Integrity, Confidentiality, and Privacy. Security is always included. The other four are optional and selected based on the service’s nature. This is the report that actually matters for security assessments.

SOC 3 is a SOC 2 report stripped down for public consumption. It contains the auditor’s opinion but none of the detailed control descriptions or test results. It’s the marketing version. If a vendor hands you a SOC 3 instead of a SOC 2, push back. You need the details.

Type I vs. Type II

This distinction is critical and frequently misunderstood.

Type I evaluates the design of controls at a specific point in time. “As of December 31, 2025, the following controls were suitably designed.” This tells you the controls existed on paper on that date. It says nothing about whether they actually worked over any period of time. A company could have designed beautiful controls on December 30 and gotten the audit on December 31. Type I is a starting point, not evidence of operational security.

Type II evaluates the design and operating effectiveness of controls over a period of time — typically 6-12 months. “During the period January 1 through December 31, 2025, the following controls operated effectively.” This is what you want. It means the auditor tested whether the controls actually worked, not just whether they existed. If you’re evaluating a vendor and they only have a Type I, ask when the Type II is coming.

ISO 27001 Certification

ISO/IEC 27001 takes a different approach. Instead of evaluating specific controls against predefined criteria, it evaluates whether an organization has implemented an Information Security Management System (ISMS) — a structured framework for managing information security risk.

The certification process involves:

  1. Establishing the ISMS. Define the scope, conduct a risk assessment, select controls from Annex A (93 controls across organizational, people, physical, and technological domains), and produce a Statement of Applicability explaining which controls apply and why.

  2. Stage 1 Audit. The certification body reviews documentation. Is the ISMS designed properly? Are policies in place? Is the risk assessment methodology sound? This is the paperwork check.

  3. Stage 2 Audit. On-site assessment of whether the ISMS is actually implemented and operating. Interviews, evidence review, observation. This is where the auditor verifies that the documented processes match reality.

  4. Surveillance Audits. Annual check-ins to verify the ISMS is maintained. The full recertification happens every three years.

ISO 27001 certification means an accredited auditor verified that your ISMS meets the standard’s requirements. It doesn’t mean every system is perfectly secured. It means you have a system for managing security that meets a defined bar. That’s valuable but different from “we tested your firewall rules.”

PCI DSS: The Payment Card Standard

If you handle credit card data, PCI DSS isn’t optional. Version 4.0 (effective March 2025) includes over 250 requirements across 12 high-level categories — from firewall configuration to encryption to access control to monitoring.

PCI audits (conducted by Qualified Security Assessors, or QSAs) are among the most technically rigorous because the standard is prescriptive. It doesn’t say “implement appropriate access controls” — it says “limit access to cardholder data by business need to know” and specifies exactly how to validate that. The tradeoff is rigidity. PCI is excellent at protecting cardholder data and mediocre at everything else. A PCI-compliant organization has strong controls around the cardholder data environment. The rest of the network? PCI doesn’t care.

How Audit Sampling Works

This is where folks need to pay attention, because audit sampling is the mechanism that determines what actually gets tested.

Auditors don’t review everything. They can’t — the populations are too large. Instead, they use statistical or judgmental sampling to select items for testing. AICPA audit sampling standards (AU-C 530) govern this process.

For a control that operates daily (like access reviews or change management approvals), the auditor might sample 25-40 instances from a twelve-month period. If all 25 pass, the control is effective. If one fails, the auditor expands the sample. If multiple fail, it’s a finding.

The math works statistically. But it means a control could fail 5% of the time and still pass the audit with a clean sample. It also means the auditor is testing the control’s operation, not its effectiveness at stopping attacks. A control can operate exactly as designed and still be insufficient to prevent a breach — because the control itself was inadequate, not because it failed.

What the Auditor Didn’t Test

Every SOC 2 report includes a section called “Complementary User Entity Controls” — controls that the customer is responsible for implementing. Multi-factor authentication configuration. User access provisioning. Endpoint security. These are carved out of the vendor’s audit scope because the vendor can’t control what you do on your end. If you don’t implement your complementary controls, the vendor’s clean audit report doesn’t protect you.

Auditors also rely heavily on management representations. The organization tells the auditor “here’s how this control works” and provides evidence. The auditor tests the evidence. If the organization is selectively presenting evidence or the auditor doesn’t dig deep enough, things get missed. This isn’t a conspiracy theory — it’s an acknowledged limitation of the audit model documented in every auditing standard.

How It Gets Exploited

Audit scope exclusion. Organizations can — and do — exclude their riskiest systems from the audit scope. A SOC 2 report’s “System Description” section defines what’s in scope. If the legacy application with known vulnerabilities isn’t in scope, it doesn’t appear in the report. The clean report covers the shiny new systems. The risk lives in the excluded ones.

Point-in-time preparation. Some organizations treat audits like finals week. They cram — remediating findings, tightening access, writing policies — in the weeks before the auditor arrives. After the audit, things slide back. Type II audits reduce this risk (they cover a full period) but don’t eliminate it, because the auditor is still sampling from the period, not monitoring continuously.

Report age. A SOC 2 report from two years ago tells you what the controls looked like two years ago. Organizations change. People leave. Systems get replaced. Configurations drift. Always check the report period and consider what might have changed since.

Badge theater. Displaying “SOC 2 Certified” and “ISO 27001” badges on the website without making the actual reports available for review. The badge tells you the audit happened. The report tells you what was tested, what was found, and what was excluded. The badge is marketing. The report is evidence. Demand the report.

What You Can Do

Sources & Further Reading