Skip to content

Commit bfe35e3

Browse files
committed
docs: enhance testing strategy with load, security, accessibility, fuzz, and CI coverage (#1062)
- Add Cypress Smoke Suite, Accessibility Testing, Security Testing, Load & Performance Testing, Fuzz & Property-Based Testing, CI Pipeline Integration, Coverage Thresholds, and Tools & Frameworks Overview sections - Update validation script with 8 new required sections and 7 reference checks (axe-core, k6, proptest, cargo-fuzz, etc.) - Update validator tests to match new requirements - All validations and tests pass
1 parent ffce149 commit bfe35e3

3 files changed

Lines changed: 175 additions & 7 deletions

File tree

docs/TESTING_STRATEGY.md

Lines changed: 127 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,9 @@ This document defines how testing is split across unit, integration, and end-to-
1616
| Unit | Pure logic, rendering branches, validation, math, and state reducers/hooks | The feature owner | `frontend/src/**/*.test.ts(x)`, `backend/src/__tests__/**/*.test.ts`, `contracts/vault/src/*_tests.rs`, `contracts/vault/src/test.rs` | `cd frontend && npm run test:run`, `cd backend && npm test`, `cargo test -p vault` |
1717
| Integration | Module-to-module behavior, HTTP handlers, provider wiring, contract scenarios with real Soroban test env | The service or feature owner | `backend/src/__tests__/*.test.ts`, `frontend/src/tests/*.test.tsx`, `frontend/src/components/*.test.tsx`, `frontend/src/pages/*.test.tsx`, `contracts/vault/src/test.rs` | Same commands as unit, plus focused suite runs |
1818
| E2E | Real browser journeys through the running app | The frontend feature owner, with backend support when the journey crosses APIs | `frontend/e2e/*.spec.ts` | `cd frontend && npm run test:e2e` |
19+
| Smoke (Cypress) | Fast critical-path verification in CI before Playwright suite; wallet connect, deposit/withdraw access, transaction history | The frontend feature owner | `frontend/cypress/e2e/*.cy.ts` | `cd frontend && npm run test:cypress` |
20+
| Load | API throughput, latency budgets, and degradation behavior under concurrent traffic | The platform/backend owner | `tests/load/*.test.js` | `k6 run tests/load/vault-load.test.js` (CI: `load-tests.yml`) |
21+
| Contract Fuzz | Invariant-guided random input generation for Soroban contract math and state transitions | The contract feature owner | `contracts/vault/fuzz/`, `contracts/vault/src/fuzz_math.rs`, `contracts/vault/src/deposit_withdraw_props.rs` | `cargo test -p vault`, `cargo fuzz run share_price_math` |
1922

2023
## Ownership Rules
2124

@@ -99,10 +102,14 @@ Use E2E tests only for user journeys that must prove the app works in a real bro
99102
- Cross-layer behavior has at least one deterministic integration test.
100103
- Browser-only flows have at least one Playwright test.
101104
- New feature work adds coverage in the layer that owns the behavior, not just in the widest suite.
105+
- New UI components include an `axe-core` accessibility audit in their test suite.
106+
- Security-sensitive changes add or update tests in the relevant security test files.
107+
- Contract math changes include proptest or fuzz coverage for the affected invariants.
108+
- Load test thresholds are reviewed when API contracts change (endpoint shape, latency budgets).
102109

103110
## Repository Enforcement
104111

105-
This strategy is enforced with the repository validator at `npm run validate:testing-strategy`. The command checks that the strategy document still covers the required testing layers, layer-specific guidance, recommended commands, and Playwright-based E2E coverage expectations.
112+
This strategy is enforced with the repository validator at `npm run validate:testing-strategy`. The command checks that the strategy document still covers the required testing layers, layer-specific guidance, recommended commands, Playwright-based E2E coverage expectations, accessibility testing, security testing, load testing, fuzz/property-based testing, CI pipeline integration, coverage thresholds, and the tools & frameworks overview.
106113

107114
## Core Playwright User Flows
108115

@@ -119,6 +126,125 @@ Canonical browser journeys live under `frontend/e2e/` and run with `cd frontend
119126

120127
Shared stubs and Freighter mocking belong in `frontend/e2e/fixtures.ts` so every core flow stays deterministic without a live backend.
121128

129+
## Cypress Smoke Suite
130+
131+
Cypress smoke tests (`frontend/cypress/e2e/smoke.cy.ts`) provide a lightweight first-pass verification that critical user journeys are not broken. They run faster than the full Playwright suite and are intended as a CI gate before heavier E2E work.
132+
133+
| Scenario | What it proves |
134+
| --- | --- |
135+
| Wallet connection | The Freighter message protocol stub returns a connected state visible in the UI |
136+
| Deposit navigation | The deposit CTA is reachable from the dashboard |
137+
| Withdrawal navigation | The withdrawal CTA is reachable from the dashboard |
138+
| Transaction history | The `/transactions` route renders a table, empty state, or wallet prompt |
139+
140+
Unlike Playwright, Cypress tests use `cy.intercept()` for API mocking and run inside the same browser event loop. Use Cypress for fast smoke-gating; use Playwright for full multi-tab, multi-origin browser journeys.
141+
142+
## Accessibility Testing
143+
144+
Accessibility tests live in `frontend/src/tests/accessibility.test.tsx` and use `axe-core` to audit rendered component trees for WCAG violations. These tests are part of the frontend unit/integration suite (`cd frontend && npm run test:run`).
145+
146+
- Every new UI component that renders interactive elements must include an `axe-core` audit in its test suite.
147+
- Focus on critical violations (`critical` and `serious` impact levels).
148+
- Use `@testing-library/react` queries that mirror real user interactions (role-based selectors, accessible names).
149+
- The accessibility test suite covers: dashboard, deposit/withdraw forms, transaction history, settings, and navigation.
150+
151+
## Security Testing
152+
153+
Security-focused tests validate defenses against common vulnerability classes:
154+
155+
| Test file | Coverage |
156+
| --- | --- |
157+
| `frontend/src/tests/xss-prevention.test.tsx` | XSS vectors in user-supplied input, URL parameters, and rendered output |
158+
| `frontend/src/lib/security.test.ts` | Input sanitization, CSP header validation, secure storage patterns |
159+
| `frontend/src/lib/maskSensitiveValues.test.ts` | Privacy-preserving display of wallet addresses and balances |
160+
| `contracts/vault/tests/security_tests.rs` | Contract access control, reentrancy guards, overflow protection |
161+
| `contracts/vault/tests/guard_checks_test.rs` | Authorization guard correctness for admin and strategy operations |
162+
| `contracts/vault/tests/access_control_test.rs` | Role-based permission enforcement on vault methods |
163+
164+
Security tests run as part of the standard test suites (`npm run test:run`, `npm test`, `cargo test -p vault`).
165+
166+
## Load & Performance Testing
167+
168+
Load tests use [k6](https://k6.io) and target the staging backend. They validate latency budgets and error rates under concurrent traffic.
169+
170+
| File | Target | Scenarios |
171+
| --- | --- | --- |
172+
| `tests/load/vault-load.test.js` | Backend deposit/withdrawal endpoints | 200 VU ramp-up over 2 min, sustained for 5 min |
173+
174+
**Thresholds:**
175+
- `http_req_duration`: p95 < 500 ms
176+
- `http_req_failed`: rate < 0.1%
177+
178+
**Execution:**
179+
- Scheduled: Mondays at 03:00 UTC (`.github/workflows/load-tests.yml`)
180+
- Manual: `workflow_dispatch` trigger in the Actions tab
181+
- Run locally: `k6 run tests/load/vault-load.test.js`
182+
183+
Load tests use idempotency keys to allow safe re-runs and target the staging environment only — never production.
184+
185+
## Fuzz & Property-Based Testing (Contracts)
186+
187+
### Cargo-Fuzz (Coverage-Guided Fuzzing)
188+
189+
Coverage-guided fuzz targets live under `contracts/vault/fuzz/fuzz_targets/` and use `cargo-fuzz` (libFuzzer).
190+
191+
| Target | What it exercises |
192+
| --- | --- |
193+
| `share_price_math` | Deposit, withdraw, yield accrual, and fee extraction with random inputs |
194+
195+
Run with:
196+
```bash
197+
cargo fuzz run share_price_math
198+
```
199+
200+
Seed fixtures in `contracts/vault/fuzz/seed_fixtures/` bootstrap the fuzzer with known-interesting inputs.
201+
202+
### Proptest (Property-Based Testing)
203+
204+
Property-based tests use the `proptest` crate and are co-located with contract source files.
205+
206+
| File | What it covers |
207+
| --- | --- |
208+
| `contracts/vault/src/fuzz_math.rs` | Arithmetic invariants: deposit/withdraw round-trips, share price monotonicity, fee bounds |
209+
| `contracts/vault/src/deposit_withdraw_props.rs` | Multi-user share sums, yield accrual monotonicity, cooldown enforcement, batch vs individual deposit equivalence |
210+
211+
Proptest regression files are checked into `contracts/vault/proptest-regressions/` so that discovered failures are never silently lost.
212+
213+
## CI Pipeline Integration
214+
215+
| Workflow | Trigger | What runs |
216+
| --- | --- | --- |
217+
| PR checks (implicit) | Every PR | Frontend unit + integration (`npm run test:run`), Backend tests (`npm test`), Contract tests (`cargo test -p vault`) |
218+
| `.github/workflows/e2e.yml` | PRs touching `frontend/**`, pushes to `main` | Playwright E2E suite (Chromium) |
219+
| `.github/workflows/load-tests.yml` | Weekly schedule (Mon 03:00 UTC) + manual dispatch | k6 load tests against staging |
220+
221+
All CI workflows upload failure artifacts (screenshots, videos, traces) for post-mortem analysis.
222+
223+
## Coverage Thresholds
224+
225+
Coverage is enforced at the CI level for backend and tracked for frontend:
226+
227+
| Layer | Tool | Threshold | Config |
228+
| --- | --- | --- | --- |
229+
| Backend | Jest (`--coverage`) | 50% branches, functions, lines, statements | `backend/jest.config.js``coverageThreshold` |
230+
| Frontend | Vitest (`@vitest/coverage-v8`) | Tracked, not yet enforced | `cd frontend && npm run test:run -- --coverage` |
231+
| Contracts | Not yet instrumented | N/A | Future: `cargo-tarpaulin` or `grcov` |
232+
233+
## Tools & Frameworks Overview
234+
235+
| Tool | Layer(s) | Purpose |
236+
| --- | --- | --- |
237+
| [Vitest](https://vitest.dev) | Frontend unit/integration | Component, hook, utility, and page-level tests; jsdom environment |
238+
| [Testing Library](https://testing-library.com) | Frontend unit/integration | DOM queries and user-event simulation |
239+
| [Playwright](https://playwright.dev) | Frontend E2E | Real browser automation with trace, video, and screenshot capture |
240+
| [Cypress](https://cypress.io) | Frontend smoke | Fast smoke-gating of critical paths in CI |
241+
| [Jest](https://jestjs.io) | Backend unit/integration | Service, middleware, and route tests with `supertest` |
242+
| [k6](https://k6.io) | Backend load | Concurrent traffic simulation with latency/error budgets |
243+
| [proptest](https://docs.rs/proptest) | Contracts | Randomized property-based testing for arithmetic invariants |
244+
| [cargo-fuzz](https://rust-fuzz.github.io) | Contracts | Coverage-guided fuzzing via libFuzzer |
245+
| [axe-core](https://github.com/dequelabs/axe-core) | Frontend accessibility | WCAG violation detection in component render trees |
246+
| [fast-check](https://fast-check.dev) | Frontend/Backend | Property-based testing for TypeScript logic |
247+
122248
---
123249

124250
## Property-Based Tests for Deposit/Withdraw Math (Issue #962)

scripts/validate-testing-strategy.test.ts

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,32 @@ Use E2E tests for browser journeys.
4848
| Flow | Spec | What it proves |
4949
| --- | --- | --- |
5050
| Dashboard | dashboard-load.spec.ts | App loads correctly |
51+
52+
## Cypress Smoke Suite
53+
Cypress smoke tests provide lightweight first-pass verification.
54+
55+
## Accessibility Testing
56+
Accessibility tests use axe-core to audit rendered component trees.
57+
58+
## Security Testing
59+
Security-focused tests validate defenses against common vulnerability classes.
60+
61+
## Load & Performance Testing
62+
Load tests use k6 and target the staging backend.
63+
64+
## Fuzz & Property-Based Testing (Contracts)
65+
Coverage-guided fuzz targets use cargo-fuzz. Property-based tests use proptest.
66+
67+
## CI Pipeline Integration
68+
CI workflows run on every PR and on schedule.
69+
70+
## Coverage Thresholds
71+
Coverage is enforced at the CI level.
72+
73+
## Tools & Frameworks Overview
74+
| Tool | Layer(s) | Purpose |
75+
| --- | --- | --- |
76+
| Vitest | Frontend unit/integration | Component, hook, utility tests |
5177
`;
5278

5379
const result = validateTestingStrategyDoc(markdown);

scripts/validate-testing-strategy.ts

Lines changed: 22 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -17,10 +17,28 @@ const REQUIRED_SECTIONS = [
1717
'## Recommended Commands',
1818
'## Review Checklist',
1919
'## Core Playwright User Flows',
20+
'## Cypress Smoke Suite',
21+
'## Accessibility Testing',
22+
'## Security Testing',
23+
'## Load & Performance Testing',
24+
'## Fuzz & Property-Based Testing (Contracts)',
25+
'## CI Pipeline Integration',
26+
'## Coverage Thresholds',
27+
'## Tools & Frameworks Overview',
2028
];
2129

2230
const REQUIRED_LAYER_HEADINGS = ['### Unit', '### Integration', '### E2E'];
2331

32+
const REQUIRED_REFERENCES = [
33+
{ term: 'Playwright', message: 'Testing strategy doc must reference Playwright for browser E2E coverage.' },
34+
{ term: 'npm run test:e2e', message: 'Testing strategy doc must include the E2E command for browser journeys.' },
35+
{ term: 'axe-core', message: 'Testing strategy doc must reference axe-core for accessibility testing.' },
36+
{ term: 'k6', message: 'Testing strategy doc must reference k6 for load testing.' },
37+
{ term: 'proptest', message: 'Testing strategy doc must reference proptest for contract property-based testing.' },
38+
{ term: 'cargo-fuzz', message: 'Testing strategy doc must reference cargo-fuzz for coverage-guided fuzzing.' },
39+
{ term: 'Cypress', message: 'Testing strategy doc must reference Cypress for smoke testing.' },
40+
];
41+
2442
export function validateTestingStrategyDoc(markdownContent: string): ValidationResult {
2543
const errors: string[] = [];
2644
const warnings: string[] = [];
@@ -42,12 +60,10 @@ export function validateTestingStrategyDoc(markdownContent: string): ValidationR
4260
}
4361
}
4462

45-
if (!markdownContent.includes('Playwright')) {
46-
errors.push('Testing strategy doc must reference Playwright for browser E2E coverage.');
47-
}
48-
49-
if (!markdownContent.includes('npm run test:e2e')) {
50-
errors.push('Testing strategy doc must include the E2E command for browser journeys.');
63+
for (const ref of REQUIRED_REFERENCES) {
64+
if (!markdownContent.includes(ref.term)) {
65+
errors.push(ref.message);
66+
}
5167
}
5268

5369
return { valid: errors.length === 0, errors, warnings };

0 commit comments

Comments
 (0)