feat(devops): implement multi-stage aws ecs express infrastructure and segregated oidc ci/cd pipelines

[emanuellcs]

Overview

This Pull Request introduces the infrastructure-as-code (IaC) layer and automated continuous integration and deployment (CI/CD) pipelines for the EngiFlow platform. The cloud architecture is provisioned using modular HashiCorp Terraform targeting AWS ECS Express Mode, Amazon RDS, Amazon S3, and Amazon SES. Additionally, the deployment engine is split into segregated validation and deployment workflows within GitHub Actions, utilizing secure OpenID Connect (OIDC) authentication. This setup ensures that the application can coexist in two distinct states: 100% operational on localhost via Docker Compose, and 100% compliant with cloud-managed specifications on AWS.

Architecture & Design Decisions

• AWS ECS Express Mode Adoption: Due to AWS platform restrictions enacted on April 30, 2026, which closed onboarding for new AWS App Runner customers, the compute tier has been shifted to AWS ECS Express Mode (Fargate). This maintains a fully managed container paradigm with minimal operational overhead while ensuring successful provisioning on newer AWS accounts.
• Cryptographic OIDC Authentication: To eliminate security risks associated with long-lived static AWS IAM User access keys, authentication between GitHub Actions and AWS relies exclusively on OpenID Connect (OIDC). Temporary short-lived tokens are requested dynamically during pipeline execution.
• State Isolation via Bootstrap Stack: The Terraform layout separates the baseline remote state resources (S3 bucket, DynamoDB lock table, and IAM OIDC providers) into an isolated bootstrap/ directory. This ensures that executing the emergency infrastructure teardown destroys only the application stack, leaving the deployment permissions and state tracking mechanics intact.
• Non-Invasive SES SMTP Integration: Adhering to strict codebase isolation constraints, the production mailing system integrates with Amazon SES via its native SMTP interface. This allows the backend to utilize its existing MailKit pipeline, pulling credentials securely from AWS Secrets Manager injected at runtime.
• FinOps Safety Enclosures: To enforce cost containment during testing cycles, compute instances are bounded to a maximum of 1 active task per service. Furthermore, an AWS Budget alert module has been incorporated to trigger administrative email notifications if projected validation infrastructure costs exceed a USD 10.00 threshold.

Key Changes

Infrastructure as Code (Terraform)

• Modular Architecture: Structured configuration blocks inside infra/terraform/modules/ covering isolated Networking (VPC, subnets, and security groups), Database, Storage, Compute, and Email.
• Amazon RDS PostgreSQL 17: Provisioned a managed database instance using the cost-optimized db.t4g.micro Graviton class configured in Single-AZ mode with automated snapshot bypass (skip_final_snapshot = true) for rapid destruction.
• Secure Storage Layer: Configured a private Amazon S3 bucket for attachments, hardened with explicit public access blocks, standard server-side encryption, and a 1-day lifecycle expiration policy to eliminate long-term storage accumulation.
• Amazon ECR Registries: Created dedicated container registries for engiflow-api and engiflow-web, bound to lifecycle rules that retain only the single latest built image layer to reduce hosting fees.

CI/CD Automation (GitHub Actions)

• Pull Request Validation (pr-validation.yml): Configured to execute strictly on pull requests targeting the main branch. It runs code linting, frontend type-checking (tsc --noEmit), backend unit testing, and a read-only terraform plan utilizing -lock=false to avoid concurrency deadlocks on parallel PR reviews.
• Production Deployment (deploy.yml): Triggers only on direct pushes or merges to the main branch. It handles multi-stage local Docker image building, pushes the production layers to Amazon ECR via an elevated write-access OIDC role, and runs an automated, headless terraform apply -auto-approve.
• Infrastructure Teardown (teardown.yml): A manual, high-priority workflow_dispatch pipeline acting as a FinOps kill switch. It instantly runs an automated terraform destroy -auto-approve to tear down all application resources on AWS, returning running cloud costs to zero.

Application Cloud Readiness Patches

• Backend Runtime Adjustments: Updated Program.cs to integrate ASP.NET Core Forwarded Headers middleware for proper HTTPS termination behind the AWS Application Load Balancer (ALB). Added a clean /healthz health probe endpoint for target group status tracking.
• Production Migrations: Configured automatic database schema migrations via EngiFlow__Database__MigrateOnStartup=true when initializing in a production environment, excluding local seed data.
• Isomorphic S3 Client Credentials: Patched S3StorageService.cs to automatically fall back to the AWS SDK Default Credential Chain when explicit static MinIO keys are omitted, ensuring seamless operation in both local and cloud containers.
• Compile-Time Frontend Variable Baking: Updated the frontend build cycle to accept NEXT_PUBLIC_API_BASE_URL as a compilation argument, baking the public API gateway domain directly into the Next.js production build artifacts for client-side WebSocket connections.
• Technical Documentation Updates: Enhanced the markdown documentation files across the root repository, api/, and web/ folders to outline the remote state initialization steps, workflow triggers, and environmental variable mapping layouts.

Verification & Testing

The infrastructure layer and delivery automation have been verified locally using static checking and simulation tools prior to submission:

• IaC Syntax Integrity: Executed terraform validate and terraform fmt -check across both the bootstrap and core application modules with clean passes.
• Pipeline Conformance: Validated the internal logic, triggering events, environment variable scopes, and AWS OIDC role configurations for all three YAML workflows using a structural layout parser.
• Docker Integration: Verified the local multi-stage container configuration schemas via docker compose config to guarantee zero configuration regression for local setups.
• Backend Build Gates: Executed the .NET compilation and automated testing stack under the Release configuration configuration cleanly.
• Frontend Build Gates: Confirmed successful compilation of the Next.js 16 standalone production bundle via npm run build and type-checking scripts.