Skip to content

Test CDA and database compatibility across runtime versions - #298

Merged
krowvin merged 2 commits into
mainfrom
fix/ci-runtime-and-cda-auth
Sep 8, 2026
Merged

Test CDA and database compatibility across runtime versions#298
krowvin merged 2 commits into
mainfrom
fix/ci-runtime-and-cda-auth

Conversation

@krowvin

@krowvin krowvin commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Reconciles this branch with current main, including Eric's #300 hashed API-key fixtures, test fixes, and Python 3.9/3.13 integration coverage. Removes this PR's legacy plaintext-key feature override and retains the cascade-delete cleanup for timeseries groups.

Addresses Mike's CI feedback by testing CDA and database versions as independent matrix axes: latest development, production release, and test release. With both Python versions, that produces 18 integration jobs. The schema installer matches the selected database tag, backend startup waits for health, and Python caches use the resolved interpreter version. Local CDA defaults to develop-nightly; CONTRIBUTING.md documents all three image overrides and the matrix pins.

Current pins: CDA develop-nightly, 2026.05.12-i, 2026.08.31-testd; database/schema installer latest-dev, 26.02.17, 26.07.16-RC02. Production CDA was verified from its public OpenAPI document on September 8. The test CDA pin follows the latest published test release, and database pins follow CDA's own release/next-release CI lanes; the deployed test/database versions could not be independently confirmed. All image references exist in GHCR.

Validation:

  • Python 3.13: 95 mock tests passed; strict mypy passed for 38 source files.
  • All nine CDA/database Compose combinations validated, including matching installer images and the seeded web user.
  • actionlint passed for both changed workflows; Black passed for the retained timeseries-group test change; repository YAML formatting and whitespace checks passed.
  • Full integration coverage runs in this PR's Actions matrix. Local integration tests were not run because the Docker daemon is unavailable. Tests target disposable containers, never deployed production/test databases.

AI tools used.

@krowvin
krowvin force-pushed the fix/ci-runtime-and-cda-auth branch 2 times, most recently from ac1d854 to da156ba Compare July 7, 2026 15:52
@krowvin
krowvin force-pushed the fix/ci-runtime-and-cda-auth branch from da156ba to df6e646 Compare July 7, 2026 16:01
@sonarqubecloud

sonarqubecloud Bot commented Jul 7, 2026

Copy link
Copy Markdown

@krowvin
krowvin marked this pull request as ready for review July 7, 2026 17:05

@krowvin krowvin left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few thoughts I had while making this PR

Comment thread docker-compose.yml Outdated
traefik:
condition: service_healthy
image: ${CWMS_DATA_API_IMAGE:-ghcr.io/usace/cwms-data-api:latest}
image: ${CWMS_DATA_API_IMAGE:-ghcr.io/usace/cwms-data-api:develop-2026.05.30}

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still am not 100% sure on if we want to lock the versions in.

If we add a new parameter, for example, we would then need to make sure this gets updated to match.

And it maybe confusing to determine that was the cause/need from the logs alone.

cwmsjs is getting around this by being generated as CDA updates

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no and definitely should not be that version. that is behind what is in production.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if the latest image is getting updated frequent enough? @MikeNeilson

Should we target the current production version 2026.05.12-f? If a develop version is targeted for the tests then things might pass that are not on prod yet.

I was trying to target something around this.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

develop-nightly should be getting updated nightly.

Build should be a matrix. Preferably with 3 entries per database and cda, the latest, what's deployed in prod, and what's deployed in test.

This makes sure the client will behave correctly in each environment. We do this for CDA against the the database versions.

NOTE: that does mean the matrix is two variables (CDA + Database) so 3x3, does get a bit large but that's just testing, it's a lot.

@@ -0,0 +1,3 @@
AUTH_RE_ENABLE_NON_HASH_KEY_SUPPORT=true

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should make it so we don't need to hash the keys locally and our old keys keep working

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is easy to add the hashed keys I will set that up.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Want to PR off this or make a separate PR for that?

If you are busy we could merge this in now so the old keys keep working then your PR can use the new ones?

@Enovotny

Enovotny commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

@krowvin check what is still needed with my changes that I just merged in main.

Comment thread docker-compose.yml Outdated
traefik:
condition: service_healthy
image: ${CWMS_DATA_API_IMAGE:-ghcr.io/usace/cwms-data-api:latest}
image: ${CWMS_DATA_API_IMAGE:-ghcr.io/usace/cwms-data-api:develop-2026.05.30}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

arguably the default here should be either develop or what's in prod. But the test matrix should override it for testing. and it should be documented for developers if they want to target something specific for some reason.

@krowvin krowvin changed the title Stabilize CI runtime dependencies Test CDA and database compatibility across runtime versions Sep 8, 2026
@sonarqubecloud

sonarqubecloud Bot commented Sep 8, 2026

Copy link
Copy Markdown

@krowvin
krowvin requested a review from MikeNeilson September 8, 2026 14:03
@krowvin
krowvin merged commit 4569028 into main Sep 8, 2026
26 checks passed
@krowvin
krowvin deleted the fix/ci-runtime-and-cda-auth branch September 8, 2026 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants