Commit 182b5b0
feat: S-Level anti-cheat sandbox with 7-layer defense system
* feat: add complete sandbox system with 6 defense layers
Implements full competition-grade anti-cheat sandbox from anti-cheat.md:
1. CacheIsolator (cache_isolator.py) — File system isolation:
- Isolated HOME directory per test
- Disabled triton/torch/cuda caches
- Auto cleanup via context manager
2. ImportHookSandbox (import_hook.py) — Runtime import enforcement:
- sys.meta_path hook for live import interception
- Auto-patches triton.autotune/heuristics/Config at import time
- Auto-patches torch.compile/CUDA Graph at import time
- Blocks multiprocessing.shared_memory/posix_ipc/mmap
- Secure exec/eval wrapper with keyword scanning
3. CUDALayerProtector (cuda_protector.py) — CUDA protection:
- Disables CUDA Graph capture/replay
- Resets CUDA state between tests
- Disables TF32 for consistent precision
4. BucketedShapeGenerator (shape_generator.py) — Shape randomization:
- GPU-alignment-friendly random shapes
- GEMM/Attention/Conv specialized generators
- TensorLayoutRandomizer for stride randomization
5. ProcessIsolatedEvaluator (process_isolator.py) — Process isolation:
- Each test in fresh subprocess (mp.spawn)
- All sandbox layers auto-applied in worker
- Batch evaluation support
6. StatisticalTimingValidator (timing_validator.py) — Statistical checks:
- CV/IQR/convergence scoring
- Outlier detection (1.5*IQR rule)
- Retest consistency check
FullSandbox (full_sandbox.py) ties all layers together into a single API.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: implement complete 7-layer S-Level anti-cheat sandbox
Strictly follows triton_competition_anti_cheat_guide.md:
Layer 1 - cache_isolator.py (Section 3):
CacheIsolator with HOME isolation, triton/torch cache dirs, cleanup
Layer 2 - import_hook.py (Section 4):
ForbiddenModuleLoader, DisabledCUDAGraph, ImportHookSandbox,
RuntimeSandbox, SecureBuiltins, SecurityError, enable_competition_sandbox
Layer 3 - cuda_protector.py (Section 5):
CUDALayerProtector, DisabledCUDAGraphContext,
CUDA Graph/TF32/profiler disable, CUDA state reset
Layer 4 - shape_generator.py (Section 6):
ShapeBucket, BucketedShapeGenerator (STANDARD_BUCKETS, GEMM_BUCKETS,
generate_gemm/conv/attention_shape),
TensorLayoutRandomizer (randomize_layout/contiguity/strides)
Layer 5 - process_isolator.py (Section 7):
TestConfig, isolated_test_worker (7-step isolation),
ProcessIsolatedEvaluator (evaluate_single/batch with mp.spawn)
Layer 6 - timing_validator.py (Section 8):
TimingAnomalyType, TimingValidationResult,
StatisticalTimingValidator (CV/IQR/convergence/outliers/retest),
AdvancedTimingValidator
Layer 7 - competition_evaluator.py (Section 9):
CompetitionConfig, TestCase, TestCaseGenerator,
isolated_worker_main, CompetitionEvaluator, main()
Plus full_sandbox.py convenience wrapper
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: block code-level hack vectors (input sniffing, caching, memory access)
Three new defenses against real-world competition attacks:
1. print() blocked:
- AST scan: 'print()' calls flagged as hack
- Runtime: builtins.print replaced with no-op in SecureBuiltins
2. data_ptr() / storage() blocked:
- AST scan: .data_ptr, .untyped_storage, .storage, .storage_offset
all flagged as forbidden memory access
3. Per-iteration random seeds:
- Each timed run uses a different seed -> different input values
- Kills hardcoded lookup tables and inter-iteration caching
- Fresh clone() per iteration prevents pointer equality checks
- Separate warmup seed to isolate warmup from scoring
These close the "200x speedup" hacks: input sniffing via print(),
inter-iteration result caching, and raw memory pointer reading.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: detect module-level mutable state as anti-hack vector
Adds scope tracking to AST HackDetector:
- visit_FunctionDef/AsyncFunctionDef/ClassDef -> increment scope depth
- visit_Assign/visit_AnnAssign -> at depth 0, detect:
- name = {} / name = [] / name = {...} (literals)
- name = dict() / name = list() / name = set() (constructor calls)
Module-level dict/list/set declarations are banned in competition mode
because they enable inter-iteration result caching attacks.
Local mutable state inside functions is NOT flagged (legitimate use).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* perf: optimize dual-execution check by reusing accuracy test output
Add dual_execution_check_with_ref() that accepts already-computed
reference output from the accuracy test, only running the
triton.jit-disabled execution and comparing with the saved result.
Saves 50% of Layer 2 execution time.
Also enables anti_hack (Layer 2 + Layer 3) by default in VerifyConfig.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add lightweight sandbox protections to Verifier __init__
Applied automatically to both LLM track and Agent track:
1. Env vars: TRITON_DISABLE_AUTOTUNE, TRITON_CACHE_DIR, TORCHINDUCTOR_DISABLE, CUDA_CACHE_DISABLE
2. CUDA layer protection: disable CUDA Graph, TF32, reset CUDA state
3. Runtime import hook: patch triton.autotune/torch.compile at import time
All set via Verifier._setup_sandbox() called in __init__.
Failures are non-fatal (try/except).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: wire SecureBuiltins into RuntimeSandbox, remove import hook from daily verify
1. RuntimeSandbox.enable() now auto-enables SecureBuiltins (print noop)
2. Verifier._setup_sandbox: removed import hook (too heavy for daily use,
only env vars + CUDA protector for lightweight path)
3. Both fixes verified: 8 modules all pass
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: automerge-bot <devnull@local>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>1 parent e039e53 commit 182b5b0
10 files changed
Lines changed: 1564 additions & 15 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
83 | 83 | | |
84 | 84 | | |
85 | 85 | | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
86 | 132 | | |
87 | 133 | | |
88 | 134 | | |
| |||
127 | 173 | | |
128 | 174 | | |
129 | 175 | | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
130 | 182 | | |
131 | 183 | | |
132 | 184 | | |
| |||
168 | 220 | | |
169 | 221 | | |
170 | 222 | | |
171 | | - | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
172 | 228 | | |
173 | 229 | | |
174 | 230 | | |
175 | 231 | | |
176 | 232 | | |
177 | 233 | | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
178 | 239 | | |
179 | 240 | | |
180 | 241 | | |
| |||
319 | 380 | | |
320 | 381 | | |
321 | 382 | | |
322 | | - | |
323 | 383 | | |
324 | 384 | | |
325 | 385 | | |
326 | 386 | | |
327 | 387 | | |
328 | | - | |
329 | | - | |
330 | 388 | | |
331 | 389 | | |
332 | | - | |
333 | 390 | | |
334 | 391 | | |
335 | | - | |
336 | 392 | | |
337 | | - | |
338 | 393 | | |
339 | 394 | | |
340 | 395 | | |
| |||
346 | 401 | | |
347 | 402 | | |
348 | 403 | | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
349 | 448 | | |
350 | 449 | | |
351 | 450 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
0 commit comments