Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 36 additions & 3 deletions .github/workflows/CDA-testing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,36 @@ on:

jobs:
integration-tests:
name: integration-tests (Python ${{ matrix.python-version }}, CDA ${{ matrix.cda.name }},
schema ${{ matrix.schema.name }})
runs-on: ubuntu-latest
timeout-minutes: 60

strategy:
fail-fast: false
max-parallel: 6
matrix:
python-version: ['3.9', '3.13']
# Keep the release pins in sync with the environments (see CONTRIBUTING.md).
cda:
- name: latest
image: ghcr.io/usace/cwms-data-api:develop-nightly
- name: production
image: ghcr.io/usace/cwms-data-api:2026.05.12-i
- name: test
image: ghcr.io/usace/cwms-data-api:2026.08.31-testd
schema:
- name: latest
tag: latest-dev
- name: production
tag: '26.02.17'
- name: test
tag: 26.07.16-RC02

env:
CWMS_DATA_API_IMAGE: ${{ matrix.cda.image }}
CWMS_DATABASE_IMAGE: ghcr.io/hydrologicengineeringcenter/cwms-database/cwms/database-ready-ora-23.5:${{ matrix.schema.tag }}
CWMS_SCHEMA_INSTALLER_IMAGE: ghcr.io/hydrologicengineeringcenter/cwms-database/cwms/schema_installer:${{ matrix.schema.tag }}

steps:
- uses: actions/checkout@v7
Expand All @@ -29,9 +53,10 @@ jobs:
- name: Set up backend
run: |
docker compose pull
docker compose up -d
docker compose up -d --wait --wait-timeout 2400

- name: Set Up Python
id: setup-python
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
Expand All @@ -50,8 +75,8 @@ jobs:
id: cache-poetry-venv
with:
path: .venv
key: ${{ runner.os }}-py${{ matrix.python-version }}-poetry-${{ hashFiles('poetry.lock')
}}
key: ${{ runner.os }}-py${{ steps.setup-python.outputs.python-version }}-poetry-${{
hashFiles('poetry.lock') }}

# Install dependencies only if cache is missed
- name: Install dependencies
Expand All @@ -78,3 +103,11 @@ jobs:
file: ./code-coverage-results.md
vars: |-
empty: empty

- name: Show backend logs on failure
if: failure()
run: docker compose logs --no-color --tail 200

- name: Stop test backend
if: always()
run: docker compose down --volumes
4 changes: 3 additions & 1 deletion .github/workflows/testing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ jobs:
- uses: actions/checkout@v7

- name: Set Up Python
id: setup-python
uses: actions/setup-python@v6
with:
python-version: '3.13'
Expand All @@ -25,7 +26,8 @@ jobs:
uses: actions/cache@v6
with:
path: ./.venv
key: venv-${{ hashFiles('poetry.lock') }}
key: ${{ runner.os }}-py${{ steps.setup-python.outputs.python-version }}-venv-${{
hashFiles('poetry.lock') }}

- name: Install Dependencies
run: poetry install
Expand Down
40 changes: 40 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,46 @@ passed, bearer token auth takes precedence.

> **Note:** If you are running other instances of CDA or Oracle on your machine, they may use different ports. Always verify which ports are in use and update your configuration files accordingly to avoid conflicts.

#### Selecting CDA and Database Versions

Local Compose defaults to CDA `develop-nightly` and database/schema-installer
`latest-dev`. Override the image references in a local, uncommitted `.env`
file when reproducing a particular environment, for example:

```dotenv
CWMS_DATA_API_IMAGE=ghcr.io/usace/cwms-data-api:2026.05.12-i
CWMS_DATABASE_IMAGE=ghcr.io/hydrologicengineeringcenter/cwms-database/cwms/database-ready-ora-23.5:26.02.17
CWMS_SCHEMA_INSTALLER_IMAGE=ghcr.io/hydrologicengineeringcenter/cwms-database/cwms/schema_installer:26.02.17
```

Keep the database and schema-installer tags together. Use a separate Compose
project for each database version, so the test data and containers are isolated:

```sh
docker compose -p cwms-python-release pull
docker compose -p cwms-python-release up -d --wait --wait-timeout 2400
```

Use the same project name for subsequent `ps`, `logs`, and `down` commands.
Stop the previous stack before starting another one using the same host ports.

The integration workflow in `.github/workflows/CDA-testing.yml` tests every
combination of three CDA images and three database versions on Python 3.9
and 3.13 (18 jobs):

| Lane | CDA tag | Database and schema-installer tag |
| --- | --- | --- |
| latest | `develop-nightly` | `latest-dev` |
| production | `2026.05.12-i` | `26.02.17` |
| test | `2026.08.31-testd` | `26.07.16-RC02` |

These release pins are maintained in the workflow; update them when the
target environments change. CDA and database are independent matrix axes,
so testing includes mixed versions, not only the three same-lane pairs.
Each job starts a disposable local stack, waits for backend health, and
uses the hashed test keys seeded by `compose_files/sql/users.sql`. CI never
runs these destructive integration tests against the deployed environments.

3. **Run Tests Against CDA**
Once the services are running, execute the tests:
```sh
Expand Down
13 changes: 6 additions & 7 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@ volumes:
auth_data:
services:
db:
image: ghcr.io/hydrologicengineeringcenter/cwms-database/cwms/database-ready-ora-23.5:latest-dev
image: ${CWMS_DATABASE_IMAGE:-ghcr.io/hydrologicengineeringcenter/cwms-database/cwms/database-ready-ora-23.5:latest-dev}
environment:
#- ORACLE_DATABASE=FREEPDB1
- ORACLE_PASSWORD=badSYSpassword
- CWMS_PASSWORD=simplecwmspasswD1
- OFFICE_ID=HQ
- OFFICE_EROC=s0
- OFFICE_EROC=q0
ports: ["1526:1521"]
healthcheck:
test: ["CMD", "tnsping", "FREEPDB1"]
Expand All @@ -18,7 +18,7 @@ services:
retries: 50
start_period: 40m
db_webuser_permissions:
image: ghcr.io/hydrologicengineeringcenter/cwms-database/cwms/schema_installer:latest-dev
image: ${CWMS_SCHEMA_INSTALLER_IMAGE:-ghcr.io/hydrologicengineeringcenter/cwms-database/cwms/schema_installer:latest-dev}
restart: "no"
environment:
- DB_HOST_PORT=db:1521
Expand All @@ -27,7 +27,7 @@ services:
- SYS_PASSWORD=badSYSpassword
# set to HQ/q0 for any national system work
- OFFICE_ID=HQ
- OFFICE_EROC=s0
- OFFICE_EROC=q0
- INSTALLONCE=1
- QUIET=1
command: >
Expand All @@ -52,15 +52,15 @@ services:
condition: service_completed_successfully
traefik:
condition: service_healthy
image: ${CWMS_DATA_API_IMAGE:-ghcr.io/usace/cwms-data-api:latest}
image: ${CWMS_DATA_API_IMAGE:-ghcr.io/usace/cwms-data-api:develop-nightly}
restart: unless-stopped
volumes:
- ./compose_files/pki/certs:/conf/
- ./compose_files/tomcat/logging.properties:/usr/local/tomcat/conf/logging.properties:ro
environment:
- CDA_JDBC_DRIVER=oracle.jdbc.driver.OracleDriver
- CDA_JDBC_URL=jdbc:oracle:thin:@db/FREEPDB1
- CDA_JDBC_USERNAME=s0webtest
- CDA_JDBC_USERNAME=q0webtest
- CDA_JDBC_PASSWORD=simplecwmspasswD1
- CDA_POOL_INIT_SIZE=5
- CDA_POOL_MAX_ACTIVE=10
Expand Down Expand Up @@ -140,4 +140,3 @@ services:
- "traefik.enable=true"
- "traefik.http.routers.traefik.rule=PathPrefix(`/traefik`)"
- "traefik.http.routers.traefik.service=api@internal"

20 changes: 3 additions & 17 deletions tests/cda/timeseries/timeseries_groups_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -155,24 +155,10 @@ def test_update_timeseries_groups():

def test_delete_timeseries_group():

# update with no timeseries in the group first
df = pd.DataFrame(columns=["timeseries-id", "office-id", "alias"])

json_dict = tg.timeseries_group_df_to_json(
data=df,
# delete the group
tg.delete_timeseries_group(
group_id=TEST_GROUP_ID,
group_office_id=TEST_OFFICE,
category_office_id=TEST_OFFICE,
category_id=TEST_CATEGORY_ID,
)
tg.update_timeseries_groups(
group_id=TEST_GROUP_ID,
office_id=TEST_OFFICE,
replace_assigned_ts=True,
data=json_dict,
)

# delete the group
tg.delete_timeseries_group(
group_id=TEST_GROUP_ID, category_id=TEST_CATEGORY_ID, office_id=TEST_OFFICE
cascade_delete=True,
)
Loading