feat: automated deployment, comprehensive documentation, and operational improvements - #75
Open
dfriveros11 wants to merge 370 commits into
Open
feat: automated deployment, comprehensive documentation, and operational improvements#75dfriveros11 wants to merge 370 commits into
dfriveros11 wants to merge 370 commits into
Conversation
added 30 commits
January 13, 2026 14:35
- Use SCRIPT_DIR with BASH_SOURCE to get absolute path - Fix cd command in screen session to use absolute path - Prevents deployment from running in wrong directory - Ensures client deployment runs correctly in screen sessions
- Change pre-built file source from Lab5 to Lab6 - Check for Lab6 dist folders instead of Lab5 - Remove cp commands since Lab6 dist folders already exist - Simplifies deployment and avoids cross-lab dependencies - Pre-built files will still be updated with correct API Gateway URLs
Add check for empty TenantStackMapping table scan results to prevent pipeline failures when no tenants exist. This ensures the pipeline completes successfully even if the table is temporarily empty during initial deployment. Fixes issue where first pipeline run would fail with JobFailed error when TenantStackMapping table scan returned 0 items.
Document the root cause and solution for tenant registration KeyError issue. Explains the dependency chain between shared stack, pipeline, pooled stack, and Settings table population.
…g table Add error handling for when pipeline is triggered before DynamoDB tables are created. This can happen during initial deployment when CodeCommit triggers the pipeline before the shared stack completes. The Lambda now gracefully handles this by returning success and allowing the next pipeline run to process the tenants once tables are ready.
Reorder deployment steps to ensure DynamoDB tables are created before the pipeline is deployed. This prevents the pipeline from being triggered before the required tables exist. Previous order: 1. Deploy pipeline (triggers on CodeCommit push) 2. Deploy shared stack (creates DynamoDB tables) New order: 1. Deploy shared stack (creates DynamoDB tables) 2. Wait for DynamoDB tables to be active 3. Deploy pipeline (now tables exist when triggered) This eliminates the ResourceNotFoundException error in the deploy-tenant-stack Lambda function.
Document the root cause and solution for the deployment order issue that caused ResourceNotFoundException and missing API Gateway URL. Explains the corrected deployment sequence and test results showing successful first-time deployment with all resources properly created.
- Fixed test-basic-tier-throttling.sh to use correct stack name (stack-lab6-pooled) - Ensured apiKey is saved to TenantDetails table during tenant creation - This fixes authorizer KeyError when accessing pooled tenant APIs
The authorizer was trying to assume 'authorizer-access-role' but the actual role is 'authorizer-access-role-lab6'. This was causing AccessDenied errors when pooled tenants tried to access APIs.
…ront invalidation - Add timestamped log file output to deploy-with-screen.sh - Display application URLs on successful deployment completion - Improve monitoring with tail -f log file instructions - Move pre-built file detection to client deployment start - Use separate flags for each UI (USED_PREBUILT_ADMIN/LANDING/APP) - Run CloudFront invalidations asynchronously (don't wait) - Remove unnecessary error checking for async operations - Reduce deployment time by ~30 seconds per CloudFront distribution
…cript Complete Deployment Guide: - Step-by-step deployment instructions for fresh AWS accounts - Architecture overview with throttling limits per tier - Detailed troubleshooting section with real-world issues - Testing checklist and verification steps - Performance optimization notes - Best practices from actual deployments Cleanup Script Optimizations: - Add timestamped log files for audit trail - Parallel S3 bucket emptying (50% faster) - Parallel tenant stack deletion - Parallel version deletion in versioned buckets - Duration tracking (shows total cleanup time) - Better progress messages and verification All knowledge from testing in fresh account documented for future deployments
Update client environment files with API Gateway URLs from latest successful deployment after full cleanup and redeploy cycle.
Remove environment.ts and environment.prod.ts files from git tracking as they contain deployment-specific API Gateway URLs that should not be committed. Changes: - Remove all environment files from git tracking (--cached) - Add environment.template.ts files with placeholders - Deployment script already generates actual files dynamically - .gitignore already configured to ignore generated files This follows security best practices: - No deployment-specific URLs in git - Template files show expected structure - Generated files are ignored by git - Each deployment creates fresh environment files
- Add ShortId pattern to CURBucket for global uniqueness - Update Glue Database name to costexplorerdb-lab7 - Update Glue Crawler name to AWSCURCrawler-Multi-tenant-lab7 - Add RoleName to IAM roles with lab7 suffix - Update EventBridge Rule names with lab7 suffix - Update Lambda inline code to reference new crawler name All resources now follow workshop naming convention for independent deployment.
Add minimal tenant stack for cost attribution demo with: - DynamoDB table using PROVISIONED billing (5 RCU/5 WCU) - Lambda functions that request consumed capacity from DynamoDB - Functions log exact consumed_rcu and consumed_wcu values - Three functions: create-product, update-product, get-products This implements Option 1 for cost attribution using provisioned capacity with exact RCU/WCU tracking from DynamoDB responses.
Update tenant_usage_and_cost.py to: - Query CloudWatch Logs for consumed_rcu and consumed_wcu values - Use ispresent() filter to find logged capacity consumption - Calculate cost attribution based on actual DynamoDB usage - Store attribution data in TenantCostAndUsageAttribution-lab7 table - Fix Athena database name to costexplorerdb-lab7 This completes Option 1 implementation for tracking exact RCU/WCU consumption from provisioned DynamoDB tables.
Change cost attribution schedules from 5 minutes to 1 minute for faster testing and demonstration of cost attribution functionality.
Add deployment.sh that: - Deploys main Lab7 stack with SAM - Uploads sample CUR data to S3 - Initializes CUR crawler - Deploys tenant stack (stack-pooled-lab7) - Generates 90 Lambda invocations for testing (30 create + 30 update + 30 get) - Provides clear status messages and next steps Script enables quick deployment and testing of cost attribution.
Add cleanup.sh that: - Empties and deletes S3 buckets with lab7 prefix - Deletes tenant stack (stack-pooled-lab7) - Deletes main CloudFormation stack - Removes DynamoDB tables, Lambda functions, and IAM roles - Cleans up CloudWatch Log Groups and EventBridge Rules - Verifies all resources are deleted - Provides timestamped logs and duration tracking Script ensures complete cleanup of all Lab7 resources.
Add DEPLOYMENT_GUIDE.md with: - Prerequisites and requirements - Step-by-step deployment instructions - Verification steps for cost attribution - Troubleshooting common issues - Cleanup instructions - Architecture overview Guide provides complete documentation for deploying and testing Lab7 cost attribution functionality.
- Add PowerTools logger module with tenant context support - Include aws-lambda-powertools and aws-xray-sdk dependencies - Enables structured JSON logging with X-Ray tracing
- Create create_product, update_product, and get_products functions - Use PowerTools Logger for structured JSON logging - Use PowerTools Tracer for X-Ray distributed tracing - Log consumed DynamoDB capacity (RCU/WCU) for cost attribution - Include tenant_id, operation, and consumed capacity in logs
Reduce Lab2 README from verbose format to streamlined reference. Keep essential deployment commands, Cognito auth flow, tenant onboarding API, and troubleshooting while removing duplication.
Reduce Lab3 README from verbose format to streamlined reference. Keep essential deployment commands, tenant isolation architecture, DynamoDB design, and troubleshooting while removing verbosity.
Reduce Lab4 README from verbose format to streamlined reference. Keep essential deployment commands, tenant provisioning pipeline, CodePipeline architecture, and troubleshooting while cutting bloat.
Reduce Lab5 README from verbose format to streamlined reference. Keep essential deployment commands, centralized logging architecture, CloudWatch cross-account setup, and troubleshooting essentials.
Reduce Lab6 README from verbose format to streamlined reference. Keep essential deployment commands, API throttling architecture, WAF/usage plans config, and troubleshooting while removing bloat.
Reduce Lab7 README from verbose format to streamlined reference. Keep essential deployment commands, cost-per-tenant architecture, CUR/Athena setup, and troubleshooting while removing verbosity.
Rewrite top-level README from 998 to 106 lines. Now serves as a clean workshop index with lab overview table, prerequisites, deploy/cleanup quick start, and project structure overview.
Document that deploy-all.sh uses --disable-rollback by default, explain why this is useful for workshop debugging, and show how to enable standard rollback with --enable-rollback flag.
The orchestration cleanup script was failing to delete orphaned Cognito user pools because it did not remove the associated Cognito domain first. This aligns the cleanup-all.sh behavior with the Lab5 individual cleanup script which already handled this correctly. The fix adds three steps before pool deletion: 1. Delete all users in the pool 2. Delete the Cognito domain 3. Delete the user pool
fix(cleanup): Delete Cognito domain and users before pool deletion
…o Python 3.14
Two related bodies of work committed together because they landed in
the same working tree before the first push. Reorganised from the
automatic pipeline pre-commit message that the deploy-all.sh workflow
creates so history reads clearly on GitHub.
Add scripts/package-for-workshop-studio.sh (1267 lines) plus two
supporting files. The script is the single entry point that builds
the Case A Workshop Studio deliverable from this Case B source tree.
See .kiro/steering/workshop-studio-packaging.md for the detailed
protocol and Placeholder_Token conventions.
What it does, end to end:
1. Parses --profile, --region, --assets-bucket, --assets-prefix,
--dry-run. Runs preflight checks for docker, python3.14, sam,
node. Computes a build_id of the form <UTC-ISO8601>-<git-sha>.
2. For each lab template (10 top-level templates across 7 labs),
runs 'sam build' then 'sam package --s3-bucket ... --s3-prefix
.../<build-id>/lambda/<lab>/'. Emits one flat packaged CFN
template per lab with CodeUri values referencing S3 keys in the
Assets_Bucket.
3. For each Angular app under Lab{N}/client/, sed-replaces any
literal API Gateway URL in environment.ts with
__API_GATEWAY_URL__ (plus sibling Placeholder_Tokens where
applicable). Runs 'ng build --configuration production'.
Post-build sanity check greps the dist main.js for the
placeholder; aborts with a clear error if ng build stripped it.
4. aws s3 sync of every Lambda zip, packaged CFN template, and
Angular dist to s3://ws-assets-us-east-1/<prefix>/serverless-
saas-workshop/<build_id>/. Copies packaged templates back into
workshop-docs/static/templates/ (overwrites — those files are
derivatives per two-case-policy.md Rule 1).
5. Writes scripts/packaging-manifest.json (743 lines) that
enumerates every uploaded artefact with kind, s3_key or
s3_key_prefix, sha256, size_bytes, referenced_by. Schema is
documented in workshop-studio-packaging.md Data Models section.
Property 6 of the spec (packaging-manifest completeness) is
validated against this file.
Add scripts/package-resume-upload.sh (279 lines). A shorter helper
that reads an existing packaging-manifest.json and re-syncs only the
artefacts that failed to upload in a prior run. Useful for flaky
networks; idempotent against S3.
Add scripts/packaging-manifest.json. This is the manifest produced
by the latest packaging run (build_id 2026-05-05T01-50-01Z-eb9ba69)
and is used by both scripts as the source of truth for what got
uploaded.
Replace every `.venv_py313` reference with `.venv_py314` across the
Case B deployment scripts. This aligns the local developer's
virtual environment path with the Lambda runtime (Python 3.14, per
.kiro/steering/core.md stack defaults) so that `pylint` validates
against the same Python ABI the workshop deploys.
Files touched (one or two-line changes):
- Lab3/scripts/deploy-updates.sh
- Lab3/scripts/deployment.sh
- Lab4/scripts/deployment.sh
- Lab5/scripts/deploy-updates.sh
- Lab5/scripts/deployment.sh
- Lab6/scripts/deployment.sh
- Lab7/scripts/deployment.sh
Every script uses the same pattern:
# Use virtual environment Python if available
if [ -f "../../.venv_py314/bin/python" ]; then
PYTHON_CMD="../../.venv_py314/bin/python"
else
PYTHON_CMD="python3"
fi
Falls back to system python3 if the venv is absent, so existing
developers who haven't set up a venv see no behaviour change.
- README.md — 8 lines changed to reference .venv_py314 in the
optional venv section; README gets a larger rewrite in the next
commit (docs(readme): inline extra-info content).
- Lab6/README.md — 2 lines changed, minor tuning.
Requirements satisfied: 2.1, 2.2, 2.3, 2.4, 2.5, 2.7, 2.8, 2.9,
4.3, 4.8, 8.6.
The root README.md referenced four files under extra-info/ that are gitignored (see .gitignore line 39 `extra-info/` — the author keeps those notes as author-only scratch, not published to the repo). Every link to extra-info/*.md in the README resolved to a 404 on GitHub for anyone who cloned the repo. Fix: integrate the useful content from the four extra-info docs directly into the README as inline sections, keeping the README the single auto-contained source for Case B self-guided learners. ## Sections added to README - Expanded 'Prerequisites' section with per-tool install instructions (AWS CLI v2, SAM CLI, Python 3.14, Node.js LTS v20/v22 via nvm, Docker or Finch, AWS CDK). Previously these were in extra-info/ PREREQUISITES.md. - Added a one-liner verification script that iterates over required tools and prints the installed version. - 'What deploy-all.sh does' and 'What cleanup-all.sh does' sections detailing step-by-step behaviour. Previously in extra-info/ DEPLOYMENT_CLEANUP_MANUAL.md. - 'Post-deployment verification' + 'Post-cleanup verification' sections with aws-cli commands learners can run to confirm state. - 'Node.js LTS Setup and Troubleshooting' section with the even/odd Node release explanation, version-switching with nvm, and the common failure modes (peer dep missing, OpenSSL errors, global Angular CLI conflicts). Previously in extra-info/ NODEJS_LTS_SETUP.md. - 'Common Failure Modes' table merged from the DEPLOYMENT_CLEANUP and NODEJS troubleshooting sections. - 'Hard Rules' section listing the must-follow rules (never use bash to invoke scripts, --profile required, don't interrupt cleanup, delete workshop-credentials.txt, don't delete the SAM managed bucket). - 'Resource Naming Convention' section with stack/Lambda/DynamoDB/ Cognito/S3 bucket naming patterns, required resource tags, and custom Cognito user attributes. Previously in extra-info/ RESOURCE_NAMING_CONVENTION.md. ## Sections removed from README - The 'Additional Resources' list that linked to the four broken extra-info/*.md paths. Only the non-broken external link (AWS SaaS Architecture Lens) is kept. - The 'extra-info/' line from the Project Structure tree listing. The extra-info/ directory remains gitignored (author-only scratch). The files inside it are not published to any repo. Learners cloning from GitHub now have one complete README with everything they need.
The Lab 1 AWS::Serverless::Api did not set an explicit Name property. Without it, SAM/CloudFormation assigned an autogenerated display name that varied per stack, which made the Lab 1 'Adding data' learner page unable to point participants to a specific API in the console. Set Name: serverless-saas-lab1-api explicitly so the API appears with a stable, searchable name across every deployment (Case_A and Case_B). Follow-up: re-run ./scripts/package-for-workshop-studio.sh to regenerate the packaged template and re-deploy a test event to validate the new name in console.
…hop-users
The script's default STACK_NAME was 'serverless-saas-lab' (Case_B naming from deploy-all.sh), which caused 'Stack not found' in Case_A (Workshop Studio) where the orchestration stack is named 'serverless-saas-workshop-main'.
Changes:
- Remove the hardcoded default and introduce CASE_A_STACK_NAME and CASE_B_STACK_NAME constants.
- Auto-detect: try Case_A ('serverless-saas-workshop-main') first, then Case_B ('serverless-saas-lab') if not found, or error clearly if neither exists.
- --stack-name <name> still works as an explicit override for both cases.
- Help text updated to document the auto-detect order and both names.
- Configuration banner shows 'auto-detect (Case_A then Case_B)' when no --stack-name is passed.
Net behaviour: from-the-IDE runs in Workshop Studio now succeed without passing --stack-name; existing Case_B calls (deploy-all.sh) continue to work.
…update-function-code The previous scripts used 'sam sync' which requires SAM stack metadata. In Case_A (Workshop Studio), the lab stacks are nested CloudFormation stacks created by the orchestration template, not by 'sam deploy'. sam sync fails with 'Unable to resolve a region' and 'stack not found' errors. Rewrite all three scripts to use 'aws lambda update-function-code' directly: - No dependency on SAM CLI or SAM stack metadata. - No dependency on stack names (uses deterministic Lambda function names). - Works in both Case_A (instance role, no profile) and Case_B (named profile). - --profile is optional (same pattern as create-workshop-users.sh). - Packages the source directory + layers/ into a zip and uploads directly. - Validates Python with pylint before deploying (same as before). Functions updated per lab: - Lab 2: create-user, register-tenant, get-tenant - Lab 3: shared-services-authorizer, business-services-authorizer, create-product - Lab 5: create-tenant-admin-user, provision-tenant
All 5 Landing page components (Labs 2-6) had Lorem ipsum filler text and generic placeholder copy. Replace with workshop-relevant content describing the SaaS architecture, multi-tenancy, and serverless services. This is a source code change that requires re-running package-for-workshop-studio.sh to take effect in Case_A (the Angular dist is rebuilt during packaging).
…dash - package-for-workshop-studio.sh: add pipeline Lambda zip upload step in upload_and_sync() for lab*-pipeline/ directories that are not uploaded by sam package. - transform_cdk_template.py: replace Unicode em-dash with ASCII dash in template Description to avoid rendering issues in CloudFormation. - Lab6/scripts/test-basic-tier-throttling.sh: copy from Lab6/tests/ to match documentation path.
… + CDK pipeline transform
Baseline snapshot before iteration 4 Lab{2..7} exercise revert and Solution/ sync. Scope covers the accumulated iteration 1-7 fixes that are still uncommitted:
- Lab7/template.yaml: CURSeedFunction + CURSeedInvocation Custom Resource for Case_A (copies SampleCUR files from Assets_Bucket to CURBucket, starts Glue Crawler); HasAssetsBucket condition so Case_B behaviour unchanged.
- Lab7/TenantUsageAndCost/tenant_usage_and_cost.py: filter_log_events API replaces Logs Insights (cold-start indexing workaround); TenantCostAndUsageAttribution-lab7 table; orchestration-aware stack discovery.
- Lab7/scripts/seed-sample-data.sh: generates real API traffic against Lab3 pooled endpoints to populate CloudWatch Logs for cost attribution.
- scripts/transform_cdk_template.py: _remove_pipeline_auto_triggers removes EventBridge Rules + IAM Roles from CDK output so Lab5/Lab6 pipelines only fire via StartPipelineExecution.
Sensitive-data scan: zero findings across all 4 files against every scan-list category. Spec: workshop-studio-readiness Phase 5 iteration 4 pre-work.
…olution/Lab7 sync
Two coherent changes in one checkpoint per Task 20.13 of workshop-studio-readiness.
1. Global Lab->Service tag rename across 32 CFN template files. Workshop Studio Cost Allocation uses a fixed allowed tag set; 'Lab' is NOT in the set, 'Service' is. Mechanical rename of 'Key: Lab' -> 'Key: Service' and 'Lab: labN' -> 'Service: labN' in every Lab{1..7}/**/*.yaml + Solution/**/*.yaml source file. .aws-sam/build-* artefacts regenerate on next sam build. Fixes coding-standards.md invariant.
2. Solution/Lab7 sync with post-iteration-7 functional state (3 files): tenant_usage_and_cost.py rewritten to mirror Lab7/ filter_log_events API; template.yaml replaces python3.9/nodejs16.x/unprefixed version with python3.14 + CUR Custom Resource + -lab7 suffix; seed-sample-data.sh new.
Sensitive-data scan: zero findings across all 35 staged files. Spec: workshop-studio-readiness Phase 5 iteration 4 pre-work A (Task 20.13).
…fier sync
Apply -lab5 and -lab6 suffixes to hardcoded identifiers in Solution/Lab{5,6}/ so Solution reflects the post-iteration-7 functional state.
Identifiers normalized: ServerlessSaaS-TenantDetails, ServerlessSaaS-Settings, ServerlessSaaS-TenantStackMapping (DynamoDB tables); serverless-saas-pipeline (CodePipeline name); stack-{0} (tenant stack naming template). Affects 14 Python files + 2 tenant-template.yaml files = 30 auto replacements + 6 targeted tenant-template edits (IAM ARNs + Custom Resource properties).
The #TODO: read table names from env vars markers are intentionally left as-is per phase-5-iteration-4-plan.md \u00a7B (documentation-only markers, not exercise TODOs).
Sensitive-data scan: zero findings. Spec: workshop-studio-readiness Phase 5 iteration 4 pre-work A (Task 20.13 Lab5+Lab6).
…tate (Task 20.14) Revert ~18 TODO markers across Labs 2-7 from functional-code-underneath to exercise stubs. The learner implements the code; Solution/ is the reference for comparison. Lab 2: create_user, register_tenant, get_tenant -> NotImplementedError stubs. Lab 3: create_product + record_metric NotImplementedError; product_service metric call, tenant_authorizer context, shared_service_authorizer admin branch -> pass/empty placeholders. Lab 4: __get_dynamodb_table in ProductService + OrderService NotImplementedError; tenant_authorizer FGAC iam_policy -> None placeholder (kept AssumeRole + context so learner only fills iam_policy). Lab 5: tenant-provisioning CodePipeline start_pipeline_execution removed; user-management user pool provisioning block removed. Lab 6: tenant_authorizer api_key fetch + dict entry removed; tenant-registration tier branching removed; tenant-management put_item apiKey attribute removed. Lab 7: 4 TODOs in calculate_daily_* functions reverted; dead helpers __get_lambda_invocations_by_tenant, __get_dynamodb_usage_by_tenant, __query_cloudwatch_logs removed; __filter_log_events_with_pattern body stubbed to return []. The 'read table names from env vars' markers in tenant-management.py are LEFT as-is (documentation-only markers per plan doc B). Sensitive-data scan: zero findings. Spec: workshop-studio-readiness Phase 5 iteration 4 pre-work B.
…own_function_names, add deploy-updates scripts for Labs 4/6/7, update requirements.txt to [Tracer], update Solution file, add expected results docs
The Case_A (Workshop Studio) packaging pipeline and its generated manifest contain Workshop-Studio-specific references that don't belong in the public Case_B GitHub repo. Untrack them going forward and ignore them so git status stays clean on dev machines where the scripts are still present locally. Paired with a git filter-repo history rewrite that removed the files from all prior commits on main.
- __get_list_of_log_group_names reads Lab4TenantStack and serverless-saas-lab4-* functions - __get_dynamodb_usage_by_tenant reads ReadCapacityUnits/WriteCapacityUnits (EMF field names) - seed-sample-data.sh resolves Lab4 tenant API - Solution updated to match
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
End-to-end improvements to the AWS Serverless SaaS Workshop covering automated deployment orchestration, comprehensive documentation for all 7 labs, operational reference guides, and multiple bug fixes discovered during real-world workshop deliveries.
Changes
Automated Deployment Orchestration
deploy-all.sh (~3,100 lines): Single-command deployment of all 7 labs using a CloudFormation orchestration template with nested stacks. Deploys in true parallel (~15-20 min vs ~70-90 min sequential). Includes automatic retry logic that preserves successfully deployed resources on failure.
cleanup-all.sh (~2,100 lines): Complete teardown script that handles all resources across labs including dynamic tenant stacks, CodeCommit repos, Cognito user pools, DynamoDB tables, and S3 buckets.
scripts/create-workshop-users.sh: Automated Cognito user creation across all labs post-deployment.
scripts/set-log-retention.sh: Sets 60-day retention on all Lambda log groups (avoids CloudFormation race condition with RetentionInDays).
scripts/main-template.yaml: CloudFormation orchestration template for parallel nested stack deployment.
Documentation (62 README files)
Per-lab README for every lab (Lab1-Lab7) covering: architecture diagram, directory structure, prerequisites, deploy/verify/cleanup commands, key concepts, and troubleshooting tables.
Per-component READMEs for server and client directories within each lab.
Solution READMEs mirroring the lab structure for reference.
Operational Reference Guides (extra-info/)
Per-lab and global reference docs including:
DEPLOYMENT_CLEANUP_MANUAL.md — step-by-step manual procedures
PREREQUISITES.md — tool versions and setup
QUICK_REFERENCE.md — cheat sheet for common operations
RESOURCE_NAMING_CONVENTION.md — naming patterns across labs
BASELINE_REFERENCE.md — expected resource state per lab
CENTRALIZED_LOGGING_STRUCTURE.md — log group naming and query patterns
CLEANUP_REFERENCE.md — resource deletion order and dependencies
Bug Fixes and Improvements
CloudFront Origin Hijacking prevention: Cleanup scripts delete CloudFormation stacks before S3 buckets to prevent OAI orphaning.
API Gateway CloudWatch role race condition: Fixed via dedicated APIGatewayCloudWatchRole resource deployed before lab stacks.
CloudWatch Logs Insights indexing: Fixed query patterns for multi-tenant log analysis.
CodeCommit push credential handling: Improved credential helper configuration for workshop environments.
CloudFront security: Updated OAI configuration and distribution settings.
Node.js LTS compatibility: Documented and tested against v20.x and v22.x (odd-numbered versions may fail Angular builds).
Python runtime: Updated to Python 3.14 across all Lambda functions.
Testing
Deployed and validated all 7 labs using deploy-all.sh in multiple AWS accounts.
Full cleanup verified with cleanup-all.sh (zero orphaned resources).
Delivered workshop with real participants — feedback incorporated into documentation and troubleshooting guides.