Skip to content

Commit a0351b0

Browse files
committed
Revert "docs: fix Crucible guide - single node works fine"
This reverts commit 3a90ee2.
1 parent 3a90ee2 commit a0351b0

1 file changed

Lines changed: 35 additions & 8 deletions

File tree

docs/ma90-crucible-deployment-complete-guide.md

Lines changed: 35 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,12 @@ ssh root@still-fawn.maas "ls -la /tmp/crucible/target/release/crucible-downstair
2525

2626
## 🚨 CRITICAL LIMITATIONS
2727

28+
### **Single-Sled Testing Not Supported**
29+
**CRUCIBLE REQUIRES MINIMUM 3 DOWNSTAIRS FOR TESTING** - Cannot test with single MA90 sled!
30+
-**1 downstairs**: Crucible upstairs/NBD fails to start
31+
-**3 downstairs**: Required for any Crucible testing (can be on same sled)
32+
- 📋 **Minimum deployment**: 3 MA90 sleds OR 3 downstairs processes on single sled
33+
2834
### **Performance Requirements**
2935
**NEVER USE 512-BYTE BLOCKS** - Performance is 10x worse than 4K blocks!
3036
-**512B blocks**: 6-11 MB/s
@@ -410,15 +416,36 @@ ssh -i ~/.ssh/id_ed25519_pve ubuntu@proper-raptor.maas "ps aux | grep crucible"
410416

411417
## Next Steps
412418

413-
### **After Deployment:**
414-
1. **Test NBD integration** with Proxmox VMs (see `docs/crucible-proxmox-nbd-integration-guide.md`)
415-
2. **Set up systemd services** for persistent operation
416-
3. **Monitor performance** and storage health
419+
### 💡 **BUDGET-FRIENDLY TESTING STRATEGY**
420+
With only 1 downstairs process, Crucible requires 3 total downstairs for functionality:
421+
- Crucible upstairs uses 3-way quorum for consensus-based operations
422+
- NBD server needs minimum 3 downstairs targets to start
423+
- **Solution**: Add 2 more downstairs processes (same sled = $30 testing vs 3 sleds = $90)
424+
425+
### **Required Next Steps for Testing:**
426+
427+
**Option A: Deploy 2 Additional MA90 Sleds (Production Fault Tolerance)**
428+
1. **Deploy 2 more MA90 sleds** using this exact process
429+
2. **Configure each with 1 downstairs** on port 3810
430+
3. **Test true 3-way distributed replication** across 3 separate sleds
431+
4. **Benefits**: Hardware fault tolerance, geographic distribution, 3x performance scaling
432+
433+
**Option B: Add 2 More Downstairs to Current Sled (Cost-Effective Testing)**
434+
435+
**💡 SMART BUDGET APPROACH - Test full Crucible functionality for $30**
436+
- **Perfect for Learning**: Understand all Crucible features without expensive hardware
437+
- **Complete Testing**: NBD integration, replication, upstairs/downstairs communication
438+
- **Development Platform**: Ideal for experimentation and proof-of-concept work
439+
- **Upgrade Ready**: Easy migration to 3-sled production setup when needed
440+
441+
1. **Create 2 additional regions** on proper-raptor (see NBD guide)
442+
2. **Start downstairs on ports 3811, 3812**
443+
3. **Test complete 3-downstairs functionality** on single sled
417444

418-
### **Optional: Multi-Sled for Fault Tolerance**
419-
For production fault tolerance, deploy additional MA90 sleds:
420-
- Each sled runs 1 downstairs on port 3810
421-
- Benefits: Hardware fault tolerance, geographic distribution
445+
### **After 3 Downstairs Available:**
446+
4. **Test NBD integration** with Proxmox VMs (see `docs/crucible-proxmox-nbd-integration-guide.md`)
447+
5. **Set up systemd services** for persistent operation
448+
6. **Monitor performance** and storage health
422449

423450
---
424451

0 commit comments

Comments
 (0)