PHASE 7 TESTING STRATEGY — PROVING CONTROL PLANE SAFETY¶
Project: AeroDB Phase: Phase 7 — Control Plane, Admin UI, and Operator Tooling Status: AUTHORITATIVE · NON-NEGOTIABLE
1. Purpose of This Document¶
This document defines how Phase 7 is tested.
The goal of Phase 7 testing is not feature validation or UI polish. The goal is to prove that the control plane cannot violate correctness, determinism, or auditability guarantees.
Testing exists to demonstrate that Phase 7:
- Cannot act autonomously
- Cannot partially execute actions
- Cannot hide failures
- Cannot interfere with the kernel
If a test cannot fail when an invariant is violated, the test is invalid.
2. Testing Philosophy¶
Phase 7 tests are adversarial.
Tests must assume:
- Operator error
- Network failure
- Control-plane crash
- Duplicate requests
- UI misbehavior
Happy-path testing alone is insufficient.
Every test must ask:
What happens if this goes wrong?
3. Test Scope¶
Phase 7 tests MUST cover:
- Command execution semantics
- Confirmation enforcement
- Failure handling
- Determinism
- Auditability
- Non-interference with kernel behavior
Tests outside this scope are optional and must not weaken guarantees.
4. Test Categories¶
4.1 Invariant Enforcement Tests¶
Purpose:
- Prove Phase 7 invariants cannot be violated
Required tests:
- No execution without confirmation
- No execution without authority
- No retries on failure
- No background execution
Each invariant MUST have at least one negative test.
4.2 Command Semantics Tests¶
Purpose:
- Prove one command maps to one kernel action
Required tests:
- Duplicate command rejection
- No implicit chaining
- Explicit rejection propagation
4.3 Failure Injection Tests¶
Purpose:
- Prove fail-closed behavior
Required tests:
- Client crash before confirmation
- Client crash after confirmation
- Control-plane crash during dispatch
- Network timeout before kernel response
Expected result:
- No partial execution
- Deterministic outcome
4.4 Determinism Tests¶
Purpose:
- Prove identical input yields identical outcome
Required tests:
- Same command + same kernel state → same result
- Same error conditions → same error output
No timing-based divergence allowed.
4.5 Auditability Tests¶
Purpose:
- Prove nothing executes without audit records
Required tests:
- Command attempt creates audit record
- Confirmation creates audit record
- Failure creates audit record
- Crash does not erase records
Missing audit record = test failure.
4.6 Non-Interference Tests¶
Purpose:
- Prove Phase 7 does not affect kernel behavior
Required tests:
- Kernel behavior identical with Phase 7 disabled
- Observability-only operations cause no mutation
5. Forbidden Testing Patterns¶
Phase 7 tests MUST NOT:
- Mock kernel correctness decisions
- Bypass confirmation logic
- Assume retries
- Use timing sleeps for correctness
If a test relies on timeouts or race conditions, it is invalid.
6. Test Environment Requirements¶
Tests MUST:
- Run against real kernel components
- Use real storage where applicable
- Simulate crashes via process termination
In-memory substitutes are allowed only if they do not weaken guarantees.
7. Test Coverage Requirements¶
Coverage is measured by invariant coverage, not line coverage.
Phase 7 MUST demonstrate:
- 100% invariant coverage
- Explicit tests for every command
- Explicit tests for every failure class
Uncovered invariant = incomplete Phase 7.
8. Test Determinism¶
All tests MUST:
- Be repeatable
- Produce identical results across runs
- Avoid nondeterministic dependencies
Flaky tests are correctness failures.
9. Regression Testing¶
Any change to Phase 7 MUST:
- Add or update tests
- Demonstrate no regression in invariants
Removing tests requires reopening Phase 7.
10. Acceptance Criteria¶
Phase 7 passes testing only if:
- All invariants are enforced
- All failure paths are tested
- All audit records are validated
- No kernel interference is observed
11. Final Statement¶
Phase 7 testing exists to prove nothing bad can happen, even when everything goes wrong.
If a bug escapes Phase 7 testing, it means the tests were insufficient.
A control plane that cannot be proven safe must not exist.
This testing strategy is authoritative.
END OF PHASE 7 TESTING STRATEGY