Skip to content

WIP: Dependency updates, streamlined backup workflow, bugfixes and new chunked E2E test for easier and more resilient testing. #90

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Jmevorach wants to merge 6 commits into from
Closed

WIP: Dependency updates, streamlined backup workflow, bugfixes and new chunked E2E test for easier and more resilient testing. #90

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
d7e1f21
updates dependencies
Jmevorach Jun 1, 2026
7280dbf
updates AWS provider in Terraform
Jmevorach Jun 1, 2026
1e9de3c
Add chunked E2E execution, apply issue #89 version bumps, and refresh…
Jmevorach Jul 2, 2026
942a981
terraform fmt fix
Jmevorach Jul 2, 2026
8cf185d
Restore k8s manifest placeholders removed during E2E deploy.
Jmevorach Jul 2, 2026
60a5074
Fix Mimir chart pin and align BATS defaults with 8.1.x changes.
Jmevorach Jul 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 26 additions & 26 deletions .github/workflows/ci-cd-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,17 +60,17 @@ permissions:

# Environment variables for consistent tool versions across all jobs
env:
PYTHON_VERSION: '3.14.4' # Python version for YAML validation and dependencies
TERRAFORM_VERSION: '1.15.0' # Terraform version for infrastructure validation
KUBECTL_VERSION: 'v1.36.0' # kubectl version for Kubernetes manifest validation
PYTHON_VERSION: '3.14.6' # Python version for YAML validation and dependencies
TERRAFORM_VERSION: '1.15.7' # Terraform version for infrastructure validation
KUBECTL_VERSION: 'v1.36.2' # kubectl version for Kubernetes manifest validation

jobs:
# Primary test job - runs test suites in parallel using matrix strategy
# This job executes the comprehensive test suite across different categories
# to validate code quality, Kubernetes manifests, script syntax, and documentation
test:
name: Run Test Suite
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04
# Matrix strategy for parallel execution of different test suites
# Each test suite runs independently, allowing for faster overall execution
strategy:
Expand All @@ -91,11 +91,11 @@ jobs:
steps:
# Step 1: Checkout repository code for testing
- name: Checkout code
uses: actions/checkout@v6.0.2
uses: actions/checkout@v7.0.0

# Step 2: Set up Python environment for YAML validation and dependencies
- name: Set up Python
uses: actions/setup-python@v6.2.0
uses: actions/setup-python@v6.3.0
with:
python-version: ${{ env.PYTHON_VERSION }}

Expand All @@ -107,7 +107,7 @@ jobs:

# Step 4: Set up Terraform for infrastructure validation
- name: Set up Terraform
uses: hashicorp/setup-terraform@v4.0.0
uses: hashicorp/setup-terraform@v4.0.1
with:
terraform_version: ${{ env.TERRAFORM_VERSION }}

Expand Down Expand Up @@ -185,16 +185,16 @@ jobs:
# and best practices enforcement across different file types
lint-and-validate:
name: Lint and Validate
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04

steps:
# Step 1: Checkout repository code for validation
- name: Checkout code
uses: actions/checkout@v6.0.2
uses: actions/checkout@v7.0.0

# Step 2: Set up Python environment for YAML validation
- name: Set up Python
uses: actions/setup-python@v6.2.0
uses: actions/setup-python@v6.3.0
with:
python-version: ${{ env.PYTHON_VERSION }}

Expand All @@ -206,7 +206,7 @@ jobs:

# Step 4: Set up Terraform for infrastructure validation
- name: Set up Terraform
uses: hashicorp/setup-terraform@v4.0.0
uses: hashicorp/setup-terraform@v4.0.1
with:
terraform_version: ${{ env.TERRAFORM_VERSION }}

Expand Down Expand Up @@ -279,12 +279,12 @@ jobs:
# and misconfigurations across the entire repository. ALL findings will fail the build.
security-scan:
name: Security Scan (Zero-Tolerance)
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04

steps:
# Step 1: Checkout repository code for security scanning
- name: Checkout code
uses: actions/checkout@v6.0.2
uses: actions/checkout@v7.0.0

# Step 2: Run Trivy vulnerability scanner in table format (STRICT MODE)
# This scan produces human-readable output and FAILS on ANY finding
Expand Down Expand Up @@ -366,16 +366,16 @@ jobs:
# and ensuring proper file permissions and structure
code-quality:
name: Code Quality
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04

steps:
# Step 1: Checkout repository code for quality analysis
- name: Checkout code
uses: actions/checkout@v6.0.2
uses: actions/checkout@v7.0.0

# Step 2: Set up Python environment for quality checks
- name: Set up Python
uses: actions/setup-python@v6.2.0
uses: actions/setup-python@v6.3.0
with:
python-version: ${{ env.PYTHON_VERSION }}

Expand Down Expand Up @@ -420,7 +420,7 @@ jobs:
# It runs when warp/ files change, versions.yaml changes, or when explicitly triggered
warp-ci:
name: Warp CI/CD
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04
# Run if manually triggered with 'all' or 'warp', or on push/PR (workflow paths filter handles file detection)
if: |
(github.event_name == 'workflow_dispatch' &&
Expand All @@ -430,11 +430,11 @@ jobs:
steps:
# Step 1: Checkout repository code
- name: Checkout code
uses: actions/checkout@v6.0.2
uses: actions/checkout@v7.0.0

# Step 2: Set up Python environment
- name: Set up Python
uses: actions/setup-python@v6.2.0
uses: actions/setup-python@v6.3.0
with:
python-version: ${{ env.PYTHON_VERSION }}

Expand All @@ -457,7 +457,7 @@ jobs:
pip install -r requirements.txt
pip install -e .
# Install test dependencies (versions pinned to match versions.yaml)
pip install pytest==9.0.3 pytest-cov==7.1.0 flake8==7.3.0 black==26.3.1 mypy==1.20.2
pip install pytest==9.1.1 pytest-cov==7.1.0 flake8==7.3.0 black==26.5.1 mypy==2.1.0

# Step 6: Run pytest with coverage (exclude benchmarks)
- name: Run pytest with coverage
Expand Down Expand Up @@ -523,18 +523,18 @@ jobs:
# It validates the dual-slot rotation logic, EFS atomic writes, and K8s integration
credential-rotation-ci:
name: Credential Rotation CI
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04
if: |
(github.event_name == 'workflow_dispatch' &&
(github.event.inputs.test_suite == 'all' || github.event.inputs.test_suite == 'credential_rotation')) ||
github.event_name != 'workflow_dispatch'

steps:
- name: Checkout code
uses: actions/checkout@v6.0.2
uses: actions/checkout@v7.0.0

- name: Set up Python
uses: actions/setup-python@v6.2.0
uses: actions/setup-python@v6.3.0
with:
python-version: ${{ env.PYTHON_VERSION }}

Expand All @@ -543,7 +543,7 @@ jobs:
cd tools/credential-rotation
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install pytest==9.0.3 pytest-cov==7.1.0 flake8==7.3.0 black==26.3.1 mypy==1.20.2
pip install pytest==9.1.1 pytest-cov==7.1.0 flake8==7.3.0 black==26.5.1 mypy==2.1.0

- name: Run pytest with coverage
run: |
Expand Down Expand Up @@ -602,14 +602,14 @@ jobs:
# overview of the entire CI/CD pipeline execution
summary:
name: Test Summary
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04
needs: [test, lint-and-validate, security-scan, code-quality, warp-ci, credential-rotation-ci]
if: always() # Run summary regardless of job outcomes

steps:
# Step 1: Checkout repository code for summary generation
- name: Checkout code
uses: actions/checkout@v6.0.2
uses: actions/checkout@v7.0.0

# Step 2: Download all test result artifacts from previous jobs
- name: Download all test results
Expand Down
24 changes: 12 additions & 12 deletions .github/workflows/ci-contract-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ permissions:
jobs:
contract-tests:
name: Contract & Version Consistency
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v7.0.0

- name: Install yq
run: |
Expand All @@ -45,11 +45,11 @@ jobs:

terraform-validate:
name: Terraform Validate (offline)
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v7.0.0

- uses: hashicorp/setup-terraform@v4.0.0
- uses: hashicorp/setup-terraform@v4.0.1

- name: Terraform fmt check
run: terraform fmt -check -recursive terraform/
Expand All @@ -59,9 +59,9 @@ jobs:

k8s-manifest-validate:
name: K8s Manifest Schema Validation
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v7.0.0

- name: Install kubeconform
run: |
Expand All @@ -83,9 +83,9 @@ jobs:

tflint:
name: TFLint
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v7.0.0

- uses: terraform-linters/setup-tflint@v4

Expand All @@ -106,12 +106,12 @@ jobs:

full-bats-suite:
name: Full BATS Test Suite
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v7.0.0

- name: Set up Go
uses: actions/setup-go@v6.4.0
uses: actions/setup-go@v6.5.0
with:
go-version: '1.25'
cache-dependency-path: console/go.sum
Expand Down
26 changes: 13 additions & 13 deletions .github/workflows/console-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,13 +58,13 @@ jobs:
# Lint and format check job
lint:
name: Lint and Format Check
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04
steps:
- name: Checkout code
uses: actions/checkout@v6.0.2
uses: actions/checkout@v7.0.0

- name: Set up Go
uses: actions/setup-go@v6.4.0
uses: actions/setup-go@v6.5.0
with:
go-version: ${{ env.GO_VERSION }}
cache-dependency-path: console/go.sum
Expand Down Expand Up @@ -97,13 +97,13 @@ jobs:
# Unit test job
test:
name: Unit Tests
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04
steps:
- name: Checkout code
uses: actions/checkout@v6.0.2
uses: actions/checkout@v7.0.0

- name: Set up Go
uses: actions/setup-go@v6.4.0
uses: actions/setup-go@v6.5.0
with:
go-version: ${{ env.GO_VERSION }}
cache-dependency-path: console/go.sum
Expand Down Expand Up @@ -135,9 +135,9 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-24.04, macos-14, macos-15]
os: [ubuntu-26.04, macos-14, macos-15]
include:
- os: ubuntu-24.04
- os: ubuntu-26.04
goos: linux
goarch: amd64
artifact_name: linux-amd64
Expand All @@ -152,10 +152,10 @@ jobs:

steps:
- name: Checkout code
uses: actions/checkout@v6.0.2
uses: actions/checkout@v7.0.0

- name: Set up Go
uses: actions/setup-go@v6.4.0
uses: actions/setup-go@v6.5.0
with:
go-version: ${{ env.GO_VERSION }}
cache-dependency-path: console/go.sum
Expand Down Expand Up @@ -211,13 +211,13 @@ jobs:
# Security scanning job - ZERO-TOLERANCE policy
security:
name: Security Scan (Zero-Tolerance)
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04
steps:
- name: Checkout code
uses: actions/checkout@v6.0.2
uses: actions/checkout@v7.0.0

- name: Set up Go
uses: actions/setup-go@v6.4.0
uses: actions/setup-go@v6.5.0
with:
go-version: ${{ env.GO_VERSION }}
cache-dependency-path: console/go.sum
Expand Down
14 changes: 7 additions & 7 deletions .github/workflows/manual-releases.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ jobs:
# Job 1: Determine release type and validate inputs
# This job processes the workflow inputs and determines the appropriate version bump
determine-release-type:
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04
outputs:
release-type: ${{ steps.determine-type.outputs.release-type }} # Type of release (major/minor/patch)
bump-version: ${{ steps.determine-type.outputs.bump-version }} # Version bump type for semver
Expand All @@ -88,13 +88,13 @@ jobs:
# since the last tagged release
check-changes:
needs: determine-release-type # Wait for release type determination
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04
outputs:
has-changes: ${{ steps.check-changes.outputs.has-changes }} # Boolean indicating if changes exist
last-tag: ${{ steps.check-changes.outputs.last-tag }} # Last release tag for reference
steps:
# Step 1: Checkout repository with full history for change detection
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v7.0.0
with:
fetch-depth: 0 # Fetch complete git history for change detection
token: ${{ secrets.GITHUB_TOKEN }}
Expand Down Expand Up @@ -129,20 +129,20 @@ jobs:
create-release:
needs: [determine-release-type, check-changes] # Wait for previous jobs
if: needs.check-changes.outputs.has-changes == 'true' # Only run if changes exist
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04
outputs:
new-version: ${{ steps.new-version.outputs.new-version }} # Calculated new version
release-url: ${{ steps.create-release.outputs.html_url }} # GitHub release URL
steps:
# Step 1: Checkout repository with full history for release creation
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v7.0.0
with:
fetch-depth: 0 # Fetch complete git history for release notes
token: ${{ secrets.GITHUB_TOKEN }}

# Step 2: Set up Python environment for semantic versioning
- name: Setup Python
uses: actions/setup-python@v6.2.0
uses: actions/setup-python@v6.3.0
with:
python-version: ${{ env.PYTHON_VERSION }}

Expand Down Expand Up @@ -325,7 +325,7 @@ jobs:
# and generates detailed summaries for both successful and failed releases
notify-completion:
needs: [determine-release-type, check-changes, create-release] # Wait for all previous jobs
runs-on: ubuntu-24.04
runs-on: ubuntu-26.04
if: always() # Run regardless of previous job outcomes
steps:
# Step 1: Generate comprehensive workflow summary
Expand Down
Loading
Loading