Module: S12 — Advanced Smart Contract Harnesses · Duration: ~60 minutes
Format: Verbatim transcript. Cues [SLIDE N] map to 03-slide-deck.html.
[SLIDE 1 — Title]
This is Module S Twelve: Advanced Smart Contract Harnesses. Sixty minutes. The second and final module of Pillar Four. The audit harness from S Eleven is operational — it detects, exploits, and patches. This module makes it measurable, portable across chains, and deliverable. Three sub-sections: benchmarking against EVMbench, Solana and cross-chain security, and the audit as a client-ready deliverable.
[SLIDE 2 — S12.1: Benchmarking Against EVMbench]
Sub-section one. Benchmarking against EVMbench. EVMbench is the standard for measuring smart contract audit harnesses. It is not enough to claim "our harness finds bugs" — the claim must be scored against the same one hundred seventeen vulnerabilities, across the same forty repositories, using the same three-mode evaluation. EVMbench makes smart contract audit performance comparable, reproducible, and falsifiable. That last word matters — falsifiable. A benchmark you cannot fail is not a benchmark.
[SLIDE 3 — The three scores that matter]
EVMbench scores an agent across three modes, each producing an independent score.
Detect recall. Of the one hundred seventeen known vulnerabilities, how many did the harness find? A harness that finds one hundred of one hundred seventeen has eighty-five percent Detect recall. This is the most commonly reported score, but it is the least complete. Detection without exploitation or patching is a hypothesis, not a delivered result.
Patch quality. For each detected vulnerability, did the harness generate a patch that removes the vulnerability and preserves all intended behavior? Patch quality is not binary. A patch can remove the bug but break a test. A patch can fix the specific PoC but leave the underlying flaw — the trap from S Eleven point four. EVMbench scores patch quality against both criteria.
Exploit success rate. For each detected vulnerability, did the harness construct a working PoC that runs on forked mainnet and successfully exploits the target? This is the hardest score. A harness can detect a vulnerability it cannot exploit — meaning it identified the pattern but cannot construct the attack transaction.
A harness that scores ninety percent Detect, thirty percent Patch, thirty percent Exploit is a detection engine, not an audit harness. A harness that scores eighty percent across all three is a genuine audit tool. The three scores must be reported together. Reporting only Detect recall hides weaknesses in the Patch and Exploit pipelines.
[SLIDE 4 — The 92% vs 34% gap]
Purpose-built smart contract agents achieve approximately ninety-two percent detection on DeFi-specific vulnerability classes. General-purpose frontier LLMs — GPT-five-point-one used as a baseline — achieve approximately thirty-four percent on the same classes. This gap is not a marginal improvement. It is the difference between a tool an audit team can rely on and a tool that misses two-thirds of DeFi bugs.
The gap has a structural explanation. DeFi-specific vulnerability classes — flash loan manipulation, oracle dependency, rebase token accounting, governance execution paths, liquidation logic — require domain knowledge that general LLMs lack. A general LLM can identify a reentrancy pattern — that is syntactic. It struggles with "this protocol assumes the spot price is honest, but a flash loan can manipulate it in one transaction" — that is semantic and economic.
Purpose-built agents embed this domain knowledge via four mechanisms. Heuristic scaffolds — Heimdallr's approach — that encode DeFi-specific safety properties. Tool integration with Slither, Mythril, and Foundry, so the LLM reasons over structured tool output, not raw source. Context engineering — function-level reorganization — that presents the right code slice for the vulnerability class. And cascaded verification that filters false positives specific to DeFi patterns.
The benchmark gap is the quantified value of these engineering choices. When a team considers building a smart contract audit harness, the ninety-two versus thirty-four gap is the business case. A purpose-built harness finds nearly three times as many DeFi bugs as pointing a general LLM at the code.
[SLIDE 5 — Benchmark as product differentiator]
EVMbench results are not just an internal metric. They are a marketing artifact. An audit firm that publishes "our harness scores eighty-eight percent Detect, seventy-two percent Patch, sixty-eight percent Exploit on EVMbench" makes a falsifiable claim that competitors can compare against. This is how audit quality becomes legible to clients who cannot evaluate Solidity security themselves. The benchmark converts "trust us, we're good" into "here is the scored evidence."
The lab for this section configures the S Eleven harness to run against an EVMbench subset and produces a scored results table — the same table that would appear in a product pitch or a competitive comparison.
[SLIDE 6 — S12.2: Solana and Cross-Chain]
Sub-section two. Solana and cross-chain considerations. The EVM is not the only chain. Solana's account model, the Anchor framework, and cross-chain bridges create distinct vulnerability classes that require harness adaptations. The harness architecture — Detect, Patch, Exploit, cascaded verification — ports. The tools and heuristics do not.
[SLIDE 7 — Solana vulnerability classes]
Solana differs from the EVM at the execution-model level. The EVM has a single global state tree keyed by contract address. Solana has accounts that are explicitly passed into each transaction instruction. This creates a class of vulnerabilities with no EVM analog.
Account confusion is the signature Solana bug. An Anchor program receives accounts as parameters. If it does not check that an account is owned by the expected program or is the expected type, an attacker can pass a malicious account that satisfies the struct layout but behaves differently. The defense is Anchor's account constraints and explicit owner checks — but these must be applied correctly, and missing them is a common audit finding.
Missing signer check. A program performs a privileged action without verifying the caller signed the transaction. In the EVM, this is like a missing only-owner modifier, but Solana signer checks are explicit per-account, which means they can be missed per-account.
Integer arithmetic errors. Solana and Rust default to checked arithmetic, but unchecked blocks or casting can introduce overflows. Same class as the EVM, different defaults.
PDA misuse. Program-Derived Addresses are deterministic addresses controlled by the program. They can be mis-derived, collision-attacked, or used without verifying the seeds. There is no EVM equivalent.
The Anchor framework reduces account-confusion bugs when used correctly. It provides account validation macros and a constraint system. But it introduces its own footguns: a derive-accounts struct that omits a has-one constraint is a silent security gap. The harness checks Anchor programs by static-analyzing the derive-accounts blocks for missing constraints — the Solana equivalent of Slither's access-control detectors.
[SLIDE 8 — Cross-chain bridge security]
Cross-chain bridges are the highest-value attack surface in crypto. Bridges lock assets on one chain and mint representations on another. Compromising the bridge's verification logic lets an attacker mint unbacked assets. The Wormhole hack in twenty twenty-two — three hundred twenty-five million dollars. The Nomad hack in twenty twenty-two — one hundred ninety million. And the twenty twenty-four and twenty twenty-five bridge exploits collectively caused billions in losses.
Bridge security is a distinct discipline because the vulnerability is often not in a single contract but in the cross-chain verification protocol. How does Chain B know that Chain A actually locked the assets? Common failure modes: signature verification flaws — the bridge accepts invalid signatures, wrong signer set, replayable signatures. Message-passing manipulation — the bridge trusts a message format an attacker can forge. Relayer centralization — a single relayer can submit fraudulent messages. Wrapped-asset accounting — minting more wrapped assets than the locked collateral.
The harness adapts for bridge audits. It models the cross-chain message flow as a state machine — the bridge protocol, not just one contract. It traces asset flows across both chains. It checks the verification invariants: minted amount must be less than or equal to locked amount. This is beyond single-contract auditing. It requires modeling the protocol as a distributed system.
[SLIDE 9 — Harness adaptations for Solana]
The three-mode architecture ports to Solana. The tools change.
Static analysis: Slither and Mythril do not work on Solana. The harness uses Solana-specific tools — solana-security-txt, Anchor's anchor test with fuzzing, and custom Semgrep rules for Rust and Anchor patterns.
Exploit PoCs: instead of Foundry forked mainnet, the harness uses Solana's bankrun or solana-test-validator with a mainnet snapshot. The PoC structure — setup, attack, verify — is identical.
Invariant extraction: the same strategy applies. Extract the protocol's intended invariants and check them. Solana invariants look different — "the vault token account's balance equals the sum of user deposits" — but the method is the same.
[SLIDE 10 — S12.3: Audit as a Deliverable]
Sub-section three. Smart contract audit as a deliverable. The audit harness produces findings. The deliverable is the audit report. A finding that lives only in a tool's JSON output is not a deliverable — it is raw material. The audit report is what the client reads, what the remediation team acts on, and what becomes part of the protocol's security record.
[SLIDE 11 — Report structure and severity]
A client-ready smart contract audit report has a consistent structure across top firms — Trail of Bits, Consensys Diligence, OpenZeppelin. Six sections.
Scope. Which contracts were audited, at which commit hash, over what dates. The scope defines what was reviewed and, by omission, what was not. This is the legal and operational boundary of the audit.
Methodology. How the audit was conducted — manual review, static analysis tools used, fuzzing, formal verification. This establishes that the audit was rigorous and reproducible.
Findings table. A summary table listing every finding: ID, title, severity, status, location. This is the executive view — the client's first stop.
Severity breakdown. A count of findings by severity.
Detailed findings. For each finding: description, impact, proof of concept — the exploit from S Eleven point three — recommendation — the patch from S Eleven point four — and status. This is the section the remediation team works from.
Remediation roadmap. Prioritized action items — which findings to fix first, which are acceptable risk, and the verification path for each fix.
Severity classification. Critical — funds at immediate risk, exploitable now without special conditions. High — funds at risk under conditions, exploitable but requires a specific state or trigger. Medium — logic error without immediate financial impact, may cause incorrect behavior but not direct loss. Low — best-practice violation, no direct risk but indicates code quality issues. Informational — optimization or style suggestion, no security relevance.
The severity is the finding's most consequential attribute. It drives remediation priority and the client's risk communication. The harness assigns severity via the LLM triage step, but every severity is human-confirmed before the report ships. The harness's severity is a draft. The auditor's is final.
The report generator converts structured finding records into the report format. The human auditor reviews, edits, and finalizes. The generator does the assembly. The auditor does the judgment.
[SLIDE 12 — Audit as product]
An AI-assisted smart contract audit is a product. The client pays for the scope, the methodology, the findings, and the remediation support. The benchmark scores are the quality proof. The report is the deliverable. The remediation support is the ongoing relationship.
Pricing an AI-assisted audit differs from a traditional manual audit. The marginal cost of running the harness is low — Heimdallr's two dollars and thirty-one cents per ten thousand lines. But the value delivered is high — ninety-two percent detection. The pricing model captures the gap between cost and value. The client pays for the finding quality, not the compute time. This is the audit-as-product positioning: the harness is the engine, the report is the product, the benchmark scores are the proof, and the remediation support is the retention.
Next is the capstone — Capstone One for bug bounty or appsec, Capstone Two for smart contract or cloud. That is where the audit harness is applied to a full engagement.
[End of script]
# Teaching Script — Module S12: Advanced Smart Contract Harnesses **Module**: S12 — Advanced Smart Contract Harnesses · **Duration**: ~60 minutes **Format**: Verbatim transcript. Cues `[SLIDE N]` map to `03-slide-deck.html`. --- [SLIDE 1 — Title] This is Module S Twelve: Advanced Smart Contract Harnesses. Sixty minutes. The second and final module of Pillar Four. The audit harness from S Eleven is operational — it detects, exploits, and patches. This module makes it measurable, portable across chains, and deliverable. Three sub-sections: benchmarking against EVMbench, Solana and cross-chain security, and the audit as a client-ready deliverable. [SLIDE 2 — S12.1: Benchmarking Against EVMbench] Sub-section one. Benchmarking against EVMbench. EVMbench is the standard for measuring smart contract audit harnesses. It is not enough to claim "our harness finds bugs" — the claim must be scored against the same one hundred seventeen vulnerabilities, across the same forty repositories, using the same three-mode evaluation. EVMbench makes smart contract audit performance comparable, reproducible, and falsifiable. That last word matters — falsifiable. A benchmark you cannot fail is not a benchmark. [SLIDE 3 — The three scores that matter] EVMbench scores an agent across three modes, each producing an independent score. Detect recall. Of the one hundred seventeen known vulnerabilities, how many did the harness find? A harness that finds one hundred of one hundred seventeen has eighty-five percent Detect recall. This is the most commonly reported score, but it is the least complete. Detection without exploitation or patching is a hypothesis, not a delivered result. Patch quality. For each detected vulnerability, did the harness generate a patch that removes the vulnerability and preserves all intended behavior? Patch quality is not binary. A patch can remove the bug but break a test. A patch can fix the specific PoC but leave the underlying flaw — the trap from S Eleven point four. EVMbench scores patch quality against both criteria. Exploit success rate. For each detected vulnerability, did the harness construct a working PoC that runs on forked mainnet and successfully exploits the target? This is the hardest score. A harness can detect a vulnerability it cannot exploit — meaning it identified the pattern but cannot construct the attack transaction. A harness that scores ninety percent Detect, thirty percent Patch, thirty percent Exploit is a detection engine, not an audit harness. A harness that scores eighty percent across all three is a genuine audit tool. The three scores must be reported together. Reporting only Detect recall hides weaknesses in the Patch and Exploit pipelines. [SLIDE 4 — The 92% vs 34% gap] Purpose-built smart contract agents achieve approximately ninety-two percent detection on DeFi-specific vulnerability classes. General-purpose frontier LLMs — GPT-five-point-one used as a baseline — achieve approximately thirty-four percent on the same classes. This gap is not a marginal improvement. It is the difference between a tool an audit team can rely on and a tool that misses two-thirds of DeFi bugs. The gap has a structural explanation. DeFi-specific vulnerability classes — flash loan manipulation, oracle dependency, rebase token accounting, governance execution paths, liquidation logic — require domain knowledge that general LLMs lack. A general LLM can identify a reentrancy pattern — that is syntactic. It struggles with "this protocol assumes the spot price is honest, but a flash loan can manipulate it in one transaction" — that is semantic and economic. Purpose-built agents embed this domain knowledge via four mechanisms. Heuristic scaffolds — Heimdallr's approach — that encode DeFi-specific safety properties. Tool integration with Slither, Mythril, and Foundry, so the LLM reasons over structured tool output, not raw source. Context engineering — function-level reorganization — that presents the right code slice for the vulnerability class. And cascaded verification that filters false positives specific to DeFi patterns. The benchmark gap is the quantified value of these engineering choices. When a team considers building a smart contract audit harness, the ninety-two versus thirty-four gap is the business case. A purpose-built harness finds nearly three times as many DeFi bugs as pointing a general LLM at the code. [SLIDE 5 — Benchmark as product differentiator] EVMbench results are not just an internal metric. They are a marketing artifact. An audit firm that publishes "our harness scores eighty-eight percent Detect, seventy-two percent Patch, sixty-eight percent Exploit on EVMbench" makes a falsifiable claim that competitors can compare against. This is how audit quality becomes legible to clients who cannot evaluate Solidity security themselves. The benchmark converts "trust us, we're good" into "here is the scored evidence." The lab for this section configures the S Eleven harness to run against an EVMbench subset and produces a scored results table — the same table that would appear in a product pitch or a competitive comparison. [SLIDE 6 — S12.2: Solana and Cross-Chain] Sub-section two. Solana and cross-chain considerations. The EVM is not the only chain. Solana's account model, the Anchor framework, and cross-chain bridges create distinct vulnerability classes that require harness adaptations. The harness architecture — Detect, Patch, Exploit, cascaded verification — ports. The tools and heuristics do not. [SLIDE 7 — Solana vulnerability classes] Solana differs from the EVM at the execution-model level. The EVM has a single global state tree keyed by contract address. Solana has accounts that are explicitly passed into each transaction instruction. This creates a class of vulnerabilities with no EVM analog. Account confusion is the signature Solana bug. An Anchor program receives accounts as parameters. If it does not check that an account is owned by the expected program or is the expected type, an attacker can pass a malicious account that satisfies the struct layout but behaves differently. The defense is Anchor's account constraints and explicit owner checks — but these must be applied correctly, and missing them is a common audit finding. Missing signer check. A program performs a privileged action without verifying the caller signed the transaction. In the EVM, this is like a missing only-owner modifier, but Solana signer checks are explicit per-account, which means they can be missed per-account. Integer arithmetic errors. Solana and Rust default to checked arithmetic, but unchecked blocks or casting can introduce overflows. Same class as the EVM, different defaults. PDA misuse. Program-Derived Addresses are deterministic addresses controlled by the program. They can be mis-derived, collision-attacked, or used without verifying the seeds. There is no EVM equivalent. The Anchor framework reduces account-confusion bugs when used correctly. It provides account validation macros and a constraint system. But it introduces its own footguns: a derive-accounts struct that omits a has-one constraint is a silent security gap. The harness checks Anchor programs by static-analyzing the derive-accounts blocks for missing constraints — the Solana equivalent of Slither's access-control detectors. [SLIDE 8 — Cross-chain bridge security] Cross-chain bridges are the highest-value attack surface in crypto. Bridges lock assets on one chain and mint representations on another. Compromising the bridge's verification logic lets an attacker mint unbacked assets. The Wormhole hack in twenty twenty-two — three hundred twenty-five million dollars. The Nomad hack in twenty twenty-two — one hundred ninety million. And the twenty twenty-four and twenty twenty-five bridge exploits collectively caused billions in losses. Bridge security is a distinct discipline because the vulnerability is often not in a single contract but in the cross-chain verification protocol. How does Chain B know that Chain A actually locked the assets? Common failure modes: signature verification flaws — the bridge accepts invalid signatures, wrong signer set, replayable signatures. Message-passing manipulation — the bridge trusts a message format an attacker can forge. Relayer centralization — a single relayer can submit fraudulent messages. Wrapped-asset accounting — minting more wrapped assets than the locked collateral. The harness adapts for bridge audits. It models the cross-chain message flow as a state machine — the bridge protocol, not just one contract. It traces asset flows across both chains. It checks the verification invariants: minted amount must be less than or equal to locked amount. This is beyond single-contract auditing. It requires modeling the protocol as a distributed system. [SLIDE 9 — Harness adaptations for Solana] The three-mode architecture ports to Solana. The tools change. Static analysis: Slither and Mythril do not work on Solana. The harness uses Solana-specific tools — solana-security-txt, Anchor's anchor test with fuzzing, and custom Semgrep rules for Rust and Anchor patterns. Exploit PoCs: instead of Foundry forked mainnet, the harness uses Solana's bankrun or solana-test-validator with a mainnet snapshot. The PoC structure — setup, attack, verify — is identical. Invariant extraction: the same strategy applies. Extract the protocol's intended invariants and check them. Solana invariants look different — "the vault token account's balance equals the sum of user deposits" — but the method is the same. [SLIDE 10 — S12.3: Audit as a Deliverable] Sub-section three. Smart contract audit as a deliverable. The audit harness produces findings. The deliverable is the audit report. A finding that lives only in a tool's JSON output is not a deliverable — it is raw material. The audit report is what the client reads, what the remediation team acts on, and what becomes part of the protocol's security record. [SLIDE 11 — Report structure and severity] A client-ready smart contract audit report has a consistent structure across top firms — Trail of Bits, Consensys Diligence, OpenZeppelin. Six sections. Scope. Which contracts were audited, at which commit hash, over what dates. The scope defines what was reviewed and, by omission, what was not. This is the legal and operational boundary of the audit. Methodology. How the audit was conducted — manual review, static analysis tools used, fuzzing, formal verification. This establishes that the audit was rigorous and reproducible. Findings table. A summary table listing every finding: ID, title, severity, status, location. This is the executive view — the client's first stop. Severity breakdown. A count of findings by severity. Detailed findings. For each finding: description, impact, proof of concept — the exploit from S Eleven point three — recommendation — the patch from S Eleven point four — and status. This is the section the remediation team works from. Remediation roadmap. Prioritized action items — which findings to fix first, which are acceptable risk, and the verification path for each fix. Severity classification. Critical — funds at immediate risk, exploitable now without special conditions. High — funds at risk under conditions, exploitable but requires a specific state or trigger. Medium — logic error without immediate financial impact, may cause incorrect behavior but not direct loss. Low — best-practice violation, no direct risk but indicates code quality issues. Informational — optimization or style suggestion, no security relevance. The severity is the finding's most consequential attribute. It drives remediation priority and the client's risk communication. The harness assigns severity via the LLM triage step, but every severity is human-confirmed before the report ships. The harness's severity is a draft. The auditor's is final. The report generator converts structured finding records into the report format. The human auditor reviews, edits, and finalizes. The generator does the assembly. The auditor does the judgment. [SLIDE 12 — Audit as product] An AI-assisted smart contract audit is a product. The client pays for the scope, the methodology, the findings, and the remediation support. The benchmark scores are the quality proof. The report is the deliverable. The remediation support is the ongoing relationship. Pricing an AI-assisted audit differs from a traditional manual audit. The marginal cost of running the harness is low — Heimdallr's two dollars and thirty-one cents per ten thousand lines. But the value delivered is high — ninety-two percent detection. The pricing model captures the gap between cost and value. The client pays for the finding quality, not the compute time. This is the audit-as-product positioning: the harness is the engine, the report is the product, the benchmark scores are the proof, and the remediation support is the retention. Next is the capstone — Capstone One for bug bounty or appsec, Capstone Two for smart contract or cloud. That is where the audit harness is applied to a full engagement. --- [End of script]