Skip to content

Commit bde8308

Browse files
Copilotkbeaugrand
Copilot
authored and
kbeaugrand
committed
fix: Replace hardcoded credentials with clear placeholders in test parameters
Security improvements: - Replace actual password with obvious placeholder requiring replacement - Replace GUIDs and tenant IDs with REPLACE_WITH_* prefixes - Add comprehensive README.md in tests directory documenting: * Parameter replacement guide * Security best practices * Key Vault reference alternative * Validation mode explanation * Local testing workflow This prevents accidental use of test credentials while maintaining clear documentation for proper configuration.
1 parent e28de8d commit bde8308

3 files changed

Lines changed: 384 additions & 5 deletions

File tree

‎specs/001-bicep-preflight-validation/IMPLEMENTATION_NOTES.md‎

Lines changed: 280 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,280 @@
1+
# Bicep Template Preflight Validation - MVP Implementation Notes
2+
3+
**Date**: 2026-01-31
4+
**Status**: MVP Complete - Phases 1-3 Implemented
5+
**Commit**: 987d408
6+
7+
## Implementation Summary
8+
9+
Successfully implemented the MVP scope of the Bicep Template Preflight Validation feature, covering Phases 1-3 (Setup, Foundational, and User Story 1). The implementation enables deployment validation of Bicep templates in CI using Azure's deployment validation API without actually deploying resources.
10+
11+
## What Was Implemented
12+
13+
### Phase 1: Setup Infrastructure (3 tasks)
14+
- ✅ **Azure Service Principal Documentation**: Comprehensive guide for setting up OIDC-based authentication with step-by-step instructions
15+
- ✅ **GitHub Workflows README**: Complete documentation of required secrets and workflow configuration
16+
- ✅ **Test Template Directory**: Created `templates/azure/tests/` directory structure for validation test cases
17+
18+
### Phase 2: Foundational Prerequisites (3 tasks)
19+
- ✅ **find-bicep-templates.sh**: Helper script to discover all Bicep templates in the repository
20+
- ✅ **parse-validation-errors.sh**: Helper script to parse and format Azure validation error messages
21+
- ✅ **Test Parameter File**: Created `azuredeploy.parameters.test.json` with minimal valid configuration for validation
22+
23+
### Phase 3: User Story 1 - Basic Deployment Validation (7 tasks)
24+
- ✅ **Azure OIDC Authentication**: Integrated `azure/login@v2` with Workload Identity Federation
25+
- ✅ **Azure CLI Verification**: Added authentication verification step
26+
- ✅ **Deployment Validation**: Configured `azure/arm-deploy@v2` with Validate mode
27+
- ✅ **Parameter Configuration**: Pass test parameters from `azuredeploy.parameters.test.json`
28+
- ✅ **Timeout Configuration**: Set 10-minute timeout for validation job
29+
- ✅ **Multi-Template Validation**: Validates all 4 main Bicep templates
30+
- ✅ **Status Check Reporting**: Fail workflow if any validation fails
31+
- ⏳ **Testing**: Requires Azure credentials to be configured (T014)
32+
33+
## Key Features
34+
35+
### 🔐 Security
36+
- **OIDC Authentication**: No client secrets stored in GitHub
37+
- **Minimal Permissions**: Documented Reader + Deployment Validator roles
38+
- **No Secrets in Logs**: Secure credential handling
39+
- **Federated Credentials**: Azure-recommended authentication method
40+
41+
### ✅ Validation Capabilities
42+
- **Deployment Mode**: Uses Azure ARM deployment validation API
43+
- **No Resource Creation**: Validation-only mode (no actual deployment)
44+
- **Multi-Template Support**: Validates 4 main Bicep templates
45+
- **Resource Group Isolation**: Creates temporary RG per run, auto-cleanup
46+
47+
### ⚡ Performance & Reliability
48+
- **10-Minute Timeout**: Prevents hung validations
49+
- **Status Reporting**: Clear pass/fail indicators for each template
50+
- **Workflow Failure**: Prevents merge if validation fails
51+
- **Automatic Cleanup**: Deletes validation resource groups with `--no-wait`
52+
53+
## Validated Templates
54+
55+
1. `templates/azure/azuredeploy.bicep` - Main deployment template
56+
2. `templates/azure/portal_with_lorawan_and_starter_kit.bicep` - Full LoRaWAN deployment
57+
3. `templates/azure/portal_with_lorawan.bicep` - LoRaWAN without starter kit
58+
4. `templates/azure/portal_without_lorawan.bicep` - IoT Portal without LoRaWAN
59+
60+
## Files Created/Modified
61+
62+
### Documentation (353 lines)
63+
- `docs/azure-validation-setup.md` - Azure setup guide (234 lines)
64+
- `.github/workflows/README.md` - Workflow documentation (119 lines)
65+
66+
### Scripts (245 lines)
67+
- `.github/scripts/find-bicep-templates.sh` - Template discovery (83 lines)
68+
- `.github/scripts/parse-validation-errors.sh` - Error parsing (162 lines)
69+
70+
### Configuration (219 lines)
71+
- `.github/workflows/bicep-lint.yml` - Updated workflow (159 lines)
72+
- `templates/azure/tests/azuredeploy.parameters.test.json` - Test parameters (60 lines)
73+
74+
### Tracking
75+
- `specs/001-bicep-preflight-validation/tasks.md` - Updated task status (13 tasks completed)
76+
77+
**Total**: 793 insertions, 17 deletions across 7 files
78+
79+
## Requirements Satisfied
80+
81+
### Fully Implemented
82+
- ✅ **FR-001**: Validate using Azure deployment validation API
83+
- ✅ **FR-002**: Auto-validate on PR template changes
84+
- ✅ **FR-003**: Auto-validate on main branch pushes
85+
- ✅ **FR-004**: Report as status check, prevent merge on failure
86+
- ✅ **FR-009**: Complete within 10 minutes
87+
- ✅ **FR-011**: Use OIDC credentials without exposing secrets
88+
- ✅ **FR-012**: Handle templates with module references (submodules: recursive)
89+
90+
### Partially Implemented (Requires Azure Setup)
91+
- 🔄 **FR-005**: Detect deployment-time issues (ready, needs Azure testing)
92+
- 🔄 **FR-006**: Provide detailed error messages (error parsing script ready)
93+
- 🔄 **FR-007**: Support all templates (main templates done, modules in Phase 4)
94+
- 🔄 **FR-008**: Handle parameters without deployment (configured, needs testing)
95+
- 🔄 **FR-010**: Replace arm-ttk validation (old step still present, removal in Phase 4)
96+
97+
## What's NOT Yet Implemented
98+
99+
### Phase 4: User Story 2 - Comprehensive Validation (8 tasks)
100+
- Module template validation (app_service.bicep, iothub.bicep, storage.bicep, etc.)
101+
- Test templates for quota/SKU/permission errors
102+
- Matrix strategy for parallel module validation
103+
- Removal of old `az bicep build` validation step
104+
105+
### Phase 5: User Story 3 - Enhanced Error Feedback (8 tasks)
106+
- Integration of `parse-validation-errors.sh` into workflow
107+
- GitHub Actions annotations for PR review
108+
- Error categorization (quota, permission, SKU, API compatibility)
109+
- Multiple error reporting (continue-on-error strategy)
110+
- Job summary with validation results
111+
112+
### Phase 6: Polish & Documentation (10 tasks)
113+
- Bicep validation guide (usage, common errors)
114+
- Troubleshooting guide
115+
- Example parameter files
116+
- Edge case testing (timeouts, API unavailability, concurrent PRs)
117+
- Performance optimization
118+
- Security review
119+
- Workflow badge
120+
- End-to-end testing
121+
122+
## Next Steps
123+
124+
### Immediate: Complete MVP Testing (T014)
125+
126+
**Prerequisites for testing**:
127+
1. Set up Azure Service Principal with OIDC (follow `docs/azure-validation-setup.md`)
128+
2. Configure GitHub repository secrets:
129+
- `AZURE_CLIENT_ID`
130+
- `AZURE_TENANT_ID`
131+
- `AZURE_SUBSCRIPTION_ID`
132+
3. Assign RBAC permissions to service principal:
133+
- Reader role at subscription or RG level
134+
- Deployment Validator role (or custom role with `Microsoft.Resources/deployments/validate/action`)
135+
4. Create a test PR modifying a Bicep template
136+
5. Verify workflow runs and reports validation status
137+
138+
**Expected outcome**: Workflow runs successfully, validates templates, reports pass/fail status
139+
140+
### Short Term: Phase 4 (User Story 2)
141+
- Expand validation to all module templates
142+
- Create test cases proving deployment validation catches issues syntax checks miss
143+
- Remove old `az bicep build` step from workflow
144+
- **Deliverable**: Comprehensive validation coverage for all templates
145+
146+
### Medium Term: Phase 5 (User Story 3)
147+
- Integrate error parsing for actionable feedback
148+
- Add GitHub Actions annotations for inline PR comments
149+
- Improve developer experience with clear error categorization
150+
- **Deliverable**: Developers can quickly identify and fix validation failures
151+
152+
### Long Term: Phase 6 (Polish)
153+
- Complete documentation suite
154+
- Edge case testing
155+
- Performance optimization
156+
- Security audit
157+
- **Deliverable**: Production-ready, well-documented feature
158+
159+
## Testing Strategy
160+
161+
### Manual Testing Required
162+
1. **Success Path**: Create PR with valid Bicep changes, verify validation passes
163+
2. **Failure Path**: Create PR with invalid Bicep (e.g., wrong SKU), verify validation fails with error message
164+
3. **Timeout**: Test with complex template approaching 10-minute limit
165+
4. **Concurrent PRs**: Multiple PRs running validation simultaneously
166+
5. **Azure API Issues**: Simulate Azure service degradation/unavailability
167+
168+
### Automated Testing (Future)
169+
- Unit tests for helper scripts (`find-bicep-templates.sh`, `parse-validation-errors.sh`)
170+
- Integration tests with mock Azure responses
171+
- End-to-end tests in staging environment
172+
173+
## Known Limitations
174+
175+
1. **Requires Azure Credentials**: Cannot test validation without Azure subscription and configured secrets
176+
2. **Module Templates Not Yet Validated**: Only main 4 templates validated in MVP
177+
3. **Old Validation Still Present**: `az bicep build` step not yet removed (Phase 4)
178+
4. **Basic Error Reporting**: Error parsing scripts created but not yet integrated (Phase 5)
179+
5. **No Test Cases for Errors**: Test templates for quota/SKU/permission issues not yet created (Phase 4)
180+
181+
## Success Metrics to Validate
182+
183+
Once Azure is configured and testing is complete:
184+
185+
- **SC-001**: 100% deployment failure detection rate (no false negatives)
186+
- **SC-002**: Validation completes within 5 minutes for typical templates
187+
- **SC-003**: Error messages are actionable without Azure portal debugging
188+
- **SC-004**: Zero merged Bicep changes fail during actual deployment
189+
- **SC-005**: 95%+ validation runs succeed without infrastructure failures
190+
- **SC-006**: No Azure credentials exposed in CI logs
191+
- **SC-007**: All templates validated (main + modules + submodules) - *partial in MVP*
192+
193+
## Technical Decisions
194+
195+
### OIDC vs Service Principal with Secret
196+
**Decision**: Use OIDC (Workload Identity Federation)
197+
**Rationale**:
198+
- No secrets stored in GitHub (better security)
199+
- Azure recommended best practice
200+
- Automatic credential rotation
201+
- No manual secret management
202+
- Future-proof authentication method
203+
204+
### Resource Group Scope vs Subscription Scope
205+
**Decision**: Use resource group scope with temporary RG per run
206+
**Rationale**:
207+
- Templates use `resourceGroup()` function
208+
- Better isolation between validation runs
209+
- Easier cleanup with `az group delete`
210+
- Reduces subscription-level permissions needed
211+
- Unique RG per run ID prevents conflicts
212+
213+
### azure/arm-deploy@v2 vs Azure CLI
214+
**Decision**: Use `azure/arm-deploy@v2` GitHub Action
215+
**Rationale**:
216+
- Higher-level abstraction with better error handling
217+
- Built-in support for Validate deployment mode
218+
- Consistent with GitHub Actions ecosystem
219+
- Easier to configure than raw CLI commands
220+
- Automatic handling of deployment artifacts
221+
222+
### Validate Mode vs Actual Deployment
223+
**Decision**: Use Validate mode only (no resource creation)
224+
**Rationale**:
225+
- No Azure costs for validation runs
226+
- Faster validation (no resource provisioning)
227+
- No cleanup of deployed resources needed
228+
- Meets requirement of checking deployment feasibility
229+
- Reduces Azure API rate limit impact
230+
231+
## Risks & Mitigations
232+
233+
### Risk: Azure API Rate Limits
234+
**Mitigation**: Validation uses read-only operations, validate action is lightweight, 10-minute timeout prevents resource exhaustion
235+
236+
### Risk: False Positives (Valid Template Reported as Invalid)
237+
**Mitigation**: Use realistic test parameters, validate against actual Azure subscription, comprehensive testing in Phase 6
238+
239+
### Risk: False Negatives (Invalid Template Reported as Valid)
240+
**Mitigation**: Phase 4 includes test cases with intentional errors to verify detection capability
241+
242+
### Risk: Validation Timeout
243+
**Mitigation**: 10-minute timeout configured, Phase 6 includes performance optimization, consider parallel validation for large template sets
244+
245+
### Risk: Concurrent PR Conflicts
246+
**Mitigation**: Unique resource group per run ID, isolated validation environments, no shared state between runs
247+
248+
## Lessons Learned
249+
250+
1. **Start with MVP**: Phases 1-3 provide immediate value without implementing all user stories
251+
2. **OIDC Setup is Critical**: Comprehensive documentation essential for adoption
252+
3. **Resource Group Isolation**: Prevents conflicts and simplifies cleanup
253+
4. **Helper Scripts Early**: Created error parsing scripts in Phase 2, ready for Phase 5 integration
254+
5. **Test Parameters Matter**: Realistic test parameters essential for meaningful validation
255+
256+
## References
257+
258+
- Feature Spec: `specs/001-bicep-preflight-validation/spec.md`
259+
- Task Breakdown: `specs/001-bicep-preflight-validation/tasks.md`
260+
- Azure Setup Guide: `docs/azure-validation-setup.md`
261+
- Workflow Documentation: `.github/workflows/README.md`
262+
- Workflow Implementation: `.github/workflows/bicep-lint.yml`
263+
264+
## Change Log
265+
266+
### 2026-01-31 - MVP Implementation (Phases 1-3)
267+
- Created Azure service principal setup documentation
268+
- Created GitHub workflows README with secrets configuration
269+
- Created helper scripts for template discovery and error parsing
270+
- Created test parameter file for validation
271+
- Updated `bicep-lint.yml` workflow with OIDC authentication and deployment validation
272+
- Configured validation for 4 main Bicep templates
273+
- Added 10-minute timeout and status check reporting
274+
- Committed changes (commit 987d408)
275+
276+
---
277+
278+
**Status**: MVP Complete - Ready for Azure Configuration and Testing
279+
**Next Action**: Configure Azure credentials and test workflow (T014)
280+
**Blocked By**: Azure subscription access and service principal setup

‎templates/azure/tests/README.md‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
# Bicep Template Test Resources
2+
3+
This directory contains test resources for Bicep template validation in CI.
4+
5+
## Files
6+
7+
### `azuredeploy.parameters.test.json`
8+
9+
**Purpose**: Parameter file template used for deployment validation in CI pipeline.
10+
11+
**Usage**: This file provides the minimum required parameters for validating Bicep templates using Azure's deployment validation API. The validation runs in `Validate` mode only - **no actual resources are deployed**.
12+
13+
**Important**: This is a **TEMPLATE FILE**. The values with `REPLACE_WITH_*` prefixes are placeholders that must be replaced with actual values before the validation can run successfully.
14+
15+
### Parameter Replacement Guide
16+
17+
Before the CI workflow can validate templates, the following placeholders must be replaced:
18+
19+
| Placeholder | Description | How to Obtain |
20+
|-------------|-------------|---------------|
21+
| `REPLACE_WITH_SECURE_PASSWORD_Min12Chars!` | PostgreSQL admin password | Generate a secure password (min 12 characters, mix of uppercase, lowercase, numbers, special chars) |
22+
| `REPLACE_WITH_TENANT_ID` | Azure AD Tenant ID | `az account show --query tenantId -o tsv` |
23+
| `REPLACE_WITH_CLIENT_ID` | Azure AD B2C Client ID | From Azure AD B2C Application Registration |
24+
| `REPLACE_WITH_API_CLIENT_ID` | Azure AD B2C API Client ID | From Azure AD B2C API Application Registration |
25+
26+
### Security Notes
27+
28+
⚠️ **DO NOT commit actual credentials or production values to this file**
29+
30+
- This file is checked into source control and should only contain placeholder values
31+
- For local testing, create a copy with actual values (e.g., `azuredeploy.parameters.local.json`) and add it to `.gitignore`
32+
- For CI/CD, consider using Azure Key Vault references or GitHub Secrets for sensitive parameters
33+
- The placeholder password is intentionally obvious to prevent accidental usage
34+
35+
### Alternative: Using Key Vault References
36+
37+
For production deployments, replace sensitive parameters with Key Vault references:
38+
39+
```json
40+
{
41+
"pgsqlAdminPassword": {
42+
"reference": {
43+
"keyVault": {
44+
"id": "/subscriptions/{subscription-id}/resourceGroups/{rg-name}/providers/Microsoft.KeyVault/vaults/{vault-name}"
45+
},
46+
"secretName": "postgresql-admin-password"
47+
}
48+
}
49+
}
50+
```
51+
52+
### Example: Creating a Local Parameter File
53+
54+
```bash
55+
# Copy template to local file
56+
cp azuredeploy.parameters.test.json azuredeploy.parameters.local.json
57+
58+
# Edit local file with actual values
59+
# Then add to .gitignore
60+
echo "templates/azure/tests/*.local.json" >> .gitignore
61+
```
62+
63+
### Validation Mode Note
64+
65+
The CI pipeline uses Azure's deployment validation API with `deploymentMode: Validate`. This mode:
66+
- ✅ Checks if the deployment would succeed
67+
- ✅ Validates resource configurations against Azure APIs
68+
- ✅ Detects quota limits, SKU availability, permission issues
69+
- ❌ Does NOT deploy any actual resources
70+
- ❌ Does NOT incur Azure costs
71+
72+
Therefore, placeholder values are acceptable for validation as long as they pass basic format validation (e.g., GUIDs for client IDs, URLs for endpoints).
73+
74+
## Future Test Cases
75+
76+
This directory will be expanded with additional test cases in future phases:
77+
78+
### Phase 4: Deployment-Time Error Tests
79+
- `invalid-sku.bicep` - Test template with unavailable Azure resource SKU
80+
- `quota-violation.bicep` - Test template exceeding typical quotas
81+
- `permission-issue.bicep` - Test template requiring elevated permissions
82+
83+
### Phase 6: Example Parameter Files
84+
- `examples/basic-deployment.json` - Minimal working configuration
85+
- `examples/with-lorawan.json` - LoRaWAN enabled configuration
86+
- `examples/without-lorawan.json` - IoT Hub only configuration
87+
88+
## References
89+
90+
- Azure Deployment Validation: https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/deployment-validation
91+
- Parameter Files: https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/parameter-files
92+
- Key Vault References: https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/key-vault-parameter
93+
94+
## Support
95+
96+
For questions about parameter configuration:
97+
- Check the main Bicep templates for parameter descriptions
98+
- Review the Azure validation setup guide: `docs/azure-validation-setup.md`
99+
- See the workflow documentation: `.github/workflows/README.md`

0 commit comments

Comments
 (0)