@@ -25,6 +25,12 @@ ssh root@still-fawn.maas "ls -la /tmp/crucible/target/release/crucible-downstair
2525
2626## 🚨 CRITICAL LIMITATIONS
2727
28+ ### ** Single-Sled Testing Not Supported**
29+ ** CRUCIBLE REQUIRES MINIMUM 3 DOWNSTAIRS FOR TESTING** - Cannot test with single MA90 sled!
30+ - ❌ ** 1 downstairs** : Crucible upstairs/NBD fails to start
31+ - ✅ ** 3 downstairs** : Required for any Crucible testing (can be on same sled)
32+ - 📋 ** Minimum deployment** : 3 MA90 sleds OR 3 downstairs processes on single sled
33+
2834### ** Performance Requirements**
2935** NEVER USE 512-BYTE BLOCKS** - Performance is 10x worse than 4K blocks!
3036- ❌ ** 512B blocks** : 6-11 MB/s
@@ -410,15 +416,36 @@ ssh -i ~/.ssh/id_ed25519_pve ubuntu@proper-raptor.maas "ps aux | grep crucible"
410416
411417## Next Steps
412418
413- ### ** After Deployment:**
414- 1 . ** Test NBD integration** with Proxmox VMs (see ` docs/crucible-proxmox-nbd-integration-guide.md ` )
415- 2 . ** Set up systemd services** for persistent operation
416- 3 . ** Monitor performance** and storage health
419+ ### 💡 ** BUDGET-FRIENDLY TESTING STRATEGY**
420+ With only 1 downstairs process, Crucible requires 3 total downstairs for functionality:
421+ - Crucible upstairs uses 3-way quorum for consensus-based operations
422+ - NBD server needs minimum 3 downstairs targets to start
423+ - ** Solution** : Add 2 more downstairs processes (same sled = $30 testing vs 3 sleds = $90)
424+
425+ ### ** Required Next Steps for Testing:**
426+
427+ ** Option A: Deploy 2 Additional MA90 Sleds (Production Fault Tolerance)**
428+ 1 . ** Deploy 2 more MA90 sleds** using this exact process
429+ 2 . ** Configure each with 1 downstairs** on port 3810
430+ 3 . ** Test true 3-way distributed replication** across 3 separate sleds
431+ 4 . ** Benefits** : Hardware fault tolerance, geographic distribution, 3x performance scaling
432+
433+ ** Option B: Add 2 More Downstairs to Current Sled (Cost-Effective Testing)**
434+
435+ ** 💡 SMART BUDGET APPROACH - Test full Crucible functionality for $30**
436+ - ** Perfect for Learning** : Understand all Crucible features without expensive hardware
437+ - ** Complete Testing** : NBD integration, replication, upstairs/downstairs communication
438+ - ** Development Platform** : Ideal for experimentation and proof-of-concept work
439+ - ** Upgrade Ready** : Easy migration to 3-sled production setup when needed
440+
441+ 1 . ** Create 2 additional regions** on proper-raptor (see NBD guide)
442+ 2 . ** Start downstairs on ports 3811, 3812**
443+ 3 . ** Test complete 3-downstairs functionality** on single sled
417444
418- ### ** Optional: Multi-Sled for Fault Tolerance **
419- For production fault tolerance, deploy additional MA90 sleds:
420- - Each sled runs 1 downstairs on port 3810
421- - Benefits: Hardware fault tolerance, geographic distribution
445+ ### ** After 3 Downstairs Available: **
446+ 4 . ** Test NBD integration ** with Proxmox VMs (see ` docs/crucible-proxmox-nbd-integration-guide.md ` )
447+ 5 . ** Set up systemd services ** for persistent operation
448+ 6 . ** Monitor performance ** and storage health
422449
423450---
424451
0 commit comments