Skip to content

Commit f0d0f67

Browse files
committed
Bump version to 25.12.2 and sync CI/CD infrastructure
- Bump version to 25.12.2 in pyproject.toml and _version.py - Update dependencies: zlmdb>=25.12.2, autobahn[all]>=25.12.2 - Document WHY both zlmdb and autobahn dependencies are needed - Remove tox from dev dependencies (no longer used) - Consolidate download-github-release to /tmp/release-artifacts/<tag> - Add generate-release-notes and docs-integrate-github-release recipes - Add .github/workflows/README.md documenting CI/CD pipelines Ref: #112 Note: This work was completed with AI assistance (Claude Code).
1 parent 467d918 commit f0d0f67

4 files changed

Lines changed: 261 additions & 22 deletions

File tree

.github/workflows/README.md

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# GitHub Actions Workflows
2+
3+
This directory contains the CI/CD workflows for cfxdb.
4+
5+
## Workflows
6+
7+
### main.yml
8+
9+
**Trigger:** Push to `master`, tags `v*`, pull requests
10+
11+
Main CI workflow that runs on every push and PR:
12+
13+
| Job | Purpose |
14+
|-----|---------|
15+
| `identifiers` | Extract release type (development/stable) using wamp-cicd |
16+
| `check` | Code quality checks (ruff format, ruff lint, ty type checker) |
17+
| `test` | Test suite across Python 3.11, 3.12, 3.13, 3.14 |
18+
| `docs` | Build Sphinx documentation |
19+
| `build` | Build sdist + wheel, upload verified artifacts |
20+
21+
### release.yml
22+
23+
**Trigger:** After `main` workflow completes successfully
24+
25+
Release workflow for creating GitHub releases and publishing to PyPI:
26+
27+
| Job | Purpose |
28+
|-----|---------|
29+
| `check-main-workflow` | Verify main workflow completed successfully |
30+
| `identifiers` | Determine release type (development/nightly vs stable) |
31+
| `release-development` | Create pre-release on GitHub for nightly builds |
32+
| `release-production` | Create GitHub release AND publish to PyPI for stable tags |
33+
34+
## Release Types
35+
36+
| Type | Trigger | PyPI | GitHub Release |
37+
|------|---------|------|----------------|
38+
| `development` | Push to master | No | Pre-release |
39+
| `nightly` | Push to master | No | Pre-release |
40+
| `stable` | Tag push (`v*`) | Yes | Full release |
41+
42+
## Shared Infrastructure
43+
44+
These workflows use reusable components from [wamp-cicd](https://github.com/wamp-proto/wamp-cicd):
45+
46+
- **Reusable workflow:** `wamp-proto/wamp-cicd/.github/workflows/identifiers.yml@main`
47+
- **Actions:**
48+
- `wamp-proto/wamp-cicd/actions/upload-artifact-verified@main`
49+
- `wamp-proto/wamp-cicd/actions/download-artifact-verified@main`
50+
- `wamp-proto/wamp-cicd/actions/check-release-fileset@main`
51+
- `wamp-proto/wamp-cicd/actions/validate-audit-file@main`
52+
53+
The `.cicd/` submodule must be present for scripts referenced by these workflows.
54+
55+
## PyPI Publishing
56+
57+
Production releases use OIDC trusted publishing:
58+
- No password/token required
59+
- Automatic attestation generation
60+
- Configured via PyPI project settings
61+
62+
## Audit Requirements
63+
64+
For stable releases, an audit file must exist at `.audit/<branch>.md` documenting:
65+
- AI assistance disclosure
66+
- Policy compliance confirmation
67+
- Related issue reference
68+
69+
## Local Testing
70+
71+
Use `just` recipes to run the same checks locally:
72+
73+
```bash
74+
just check # Run all code quality checks
75+
just test # Run test suite
76+
just docs # Build documentation
77+
just dist # Build distribution packages
78+
just verify-dist # Verify built packages
79+
```

justfile

Lines changed: 165 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -636,22 +636,60 @@ publish-test venv="":
636636
${VENV_PYTHON} -m twine upload --repository testpypi dist/*
637637
echo "--> Published to Test PyPI"
638638
639-
# Download GitHub release artifacts (nightly or tagged release)
640-
download-github-release release_type="nightly":
639+
# Download GitHub release artifacts to /tmp/release-artifacts/<tag> with checksum verification
640+
download-github-release tag="nightly":
641641
#!/usr/bin/env bash
642642
set -e
643-
echo "==> Downloading GitHub release artifacts ({{release_type}})..."
644-
rm -rf ./dist
645-
mkdir -p ./dist
646-
if [ "{{release_type}}" = "nightly" ]; then
647-
gh release download nightly --repo crossbario/cfxdb --dir ./dist --pattern '*.whl' --pattern '*.tar.gz' || \
643+
644+
TAG="{{ tag }}"
645+
DOWNLOAD_DIR="/tmp/release-artifacts/${TAG}"
646+
647+
echo "==> Downloading GitHub release artifacts for ${TAG}..."
648+
echo " Target directory: ${DOWNLOAD_DIR}"
649+
650+
# Check gh CLI is authenticated
651+
if ! gh auth status &>/dev/null; then
652+
echo "❌ Error: gh CLI is not authenticated"
653+
echo " Run: gh auth login"
654+
exit 1
655+
fi
656+
657+
# Create fresh download directory
658+
rm -rf "${DOWNLOAD_DIR}"
659+
mkdir -p "${DOWNLOAD_DIR}"
660+
661+
# Download artifacts
662+
if [ "${TAG}" = "nightly" ]; then
663+
gh release download nightly --repo crossbario/cfxdb --dir "${DOWNLOAD_DIR}" \
664+
--pattern '*.whl' --pattern '*.tar.gz' --pattern 'CHECKSUMS.sha256' 2>/dev/null || \
648665
echo "Note: No nightly release found or no artifacts available"
649666
else
650-
gh release download "{{release_type}}" --repo crossbario/cfxdb --dir ./dist --pattern '*.whl' --pattern '*.tar.gz'
667+
gh release download "${TAG}" --repo crossbario/cfxdb --dir "${DOWNLOAD_DIR}" \
668+
--pattern '*.whl' --pattern '*.tar.gz' --pattern 'CHECKSUMS.sha256'
651669
fi
670+
671+
# Verify checksums if available
672+
if [ -f "${DOWNLOAD_DIR}/CHECKSUMS.sha256" ]; then
673+
echo ""
674+
echo "==> Verifying checksums..."
675+
cd "${DOWNLOAD_DIR}"
676+
if sha256sum -c CHECKSUMS.sha256; then
677+
echo "✅ All checksums verified"
678+
else
679+
echo "❌ Checksum verification failed!"
680+
exit 1
681+
fi
682+
cd - > /dev/null
683+
else
684+
echo ""
685+
echo "⚠️ Warning: No CHECKSUMS.sha256 file found in release"
686+
fi
687+
652688
echo ""
653689
echo "Downloaded artifacts:"
654-
ls -la ./dist/ || echo "No artifacts downloaded"
690+
ls -la "${DOWNLOAD_DIR}/" || echo "No artifacts downloaded"
691+
echo ""
692+
echo "==> Artifacts available at: ${DOWNLOAD_DIR}"
655693
656694
# Download release artifacts from GitHub and publish to PyPI
657695
publish-pypi venv="" tag="": (install-tools venv)
@@ -667,13 +705,16 @@ publish-pypi venv="" tag="": (install-tools venv)
667705
echo "Usage: just publish-pypi cpy311 v24.1.1"
668706
exit 1
669707
fi
708+
709+
DOWNLOAD_DIR="/tmp/release-artifacts/${TAG}"
710+
670711
echo "==> Publishing ${TAG} to PyPI..."
671712
echo ""
672713
echo "Step 1: Download release artifacts from GitHub..."
673714
just download-github-release "${TAG}"
674715
echo ""
675716
echo "Step 2: Verify packages with twine..."
676-
"${VENV_PATH}/bin/twine" check dist/*
717+
"${VENV_PATH}/bin/twine" check "${DOWNLOAD_DIR}"/*.whl "${DOWNLOAD_DIR}"/*.tar.gz
677718
echo ""
678719
echo "Note: This is a pure Python package (py3-none-any wheel)."
679720
echo " auditwheel verification is not applicable (no native extensions)."
@@ -683,7 +724,7 @@ publish-pypi venv="" tag="": (install-tools venv)
683724
echo "WARNING: This will upload to PyPI!"
684725
echo "Press Ctrl+C to cancel, or Enter to continue..."
685726
read
686-
"${VENV_PATH}/bin/twine" upload dist/*
727+
"${VENV_PATH}/bin/twine" upload "${DOWNLOAD_DIR}"/*.whl "${DOWNLOAD_DIR}"/*.tar.gz
687728
echo ""
688729
echo "==> Successfully published ${TAG} to PyPI"
689730
@@ -710,6 +751,119 @@ publish-rtd tag="":
710751
echo ""
711752
712753
754+
# Generate release notes from GitHub release for documentation integration
755+
generate-release-notes tag:
756+
#!/usr/bin/env bash
757+
set -e
758+
759+
TAG="{{ tag }}"
760+
DOWNLOAD_DIR="/tmp/release-artifacts/${TAG}"
761+
762+
echo "==> Generating release notes for ${TAG}..."
763+
764+
# Check gh CLI is authenticated
765+
if ! gh auth status &>/dev/null; then
766+
echo "❌ Error: gh CLI is not authenticated"
767+
echo " Run: gh auth login"
768+
exit 1
769+
fi
770+
771+
# Get release info from GitHub
772+
echo ""
773+
echo "==> Fetching release information from GitHub..."
774+
gh release view "${TAG}" --repo crossbario/cfxdb --json tagName,name,body,createdAt,assets \
775+
> "${DOWNLOAD_DIR}/release-info.json" 2>/dev/null || {
776+
echo "❌ Error: Could not fetch release ${TAG}"
777+
exit 1
778+
}
779+
780+
# Extract release body (notes)
781+
echo ""
782+
echo "==> Release notes for ${TAG}:"
783+
echo "----------------------------------------"
784+
gh release view "${TAG}" --repo crossbario/cfxdb --json body -q '.body'
785+
echo "----------------------------------------"
786+
echo ""
787+
echo "==> Release info saved to: ${DOWNLOAD_DIR}/release-info.json"
788+
789+
# Integrate GitHub release artifacts into documentation (chain-of-custody)
790+
docs-integrate-github-release tag:
791+
#!/usr/bin/env bash
792+
set -e
793+
794+
TAG="{{ tag }}"
795+
DOWNLOAD_DIR="/tmp/release-artifacts/${TAG}"
796+
DOCS_RELEASE_DIR="docs/_releases/${TAG}"
797+
798+
echo "==> Integrating GitHub release ${TAG} into documentation..."
799+
800+
# Ensure artifacts are downloaded
801+
if [ ! -d "${DOWNLOAD_DIR}" ] || [ -z "$(ls -A ${DOWNLOAD_DIR} 2>/dev/null)" ]; then
802+
echo "==> Artifacts not found, downloading..."
803+
just download-github-release "${TAG}"
804+
fi
805+
806+
# Create docs release directory
807+
mkdir -p "${DOCS_RELEASE_DIR}"
808+
809+
# Copy chain-of-custody files
810+
echo ""
811+
echo "==> Copying chain-of-custody files..."
812+
813+
if [ -f "${DOWNLOAD_DIR}/CHECKSUMS.sha256" ]; then
814+
cp "${DOWNLOAD_DIR}/CHECKSUMS.sha256" "${DOCS_RELEASE_DIR}/"
815+
echo " ✓ CHECKSUMS.sha256"
816+
fi
817+
818+
if [ -f "${DOWNLOAD_DIR}/release-info.json" ]; then
819+
cp "${DOWNLOAD_DIR}/release-info.json" "${DOCS_RELEASE_DIR}/"
820+
echo " ✓ release-info.json"
821+
fi
822+
823+
# Generate build-info.txt with download metadata
824+
echo "==> Generating build-info.txt..."
825+
TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
826+
ARTIFACTS=$(ls -la "${DOWNLOAD_DIR}"/*.whl "${DOWNLOAD_DIR}"/*.tar.gz 2>/dev/null | awk '{print " " $NF ": " $5 " bytes"}')
827+
{
828+
echo "Release: ${TAG}"
829+
echo "Downloaded: ${TIMESTAMP}"
830+
echo "Source: GitHub Releases (crossbario/cfxdb)"
831+
echo "Verification: Checksums verified during download"
832+
echo ""
833+
echo "Artifacts:"
834+
echo "${ARTIFACTS}"
835+
} > "${DOCS_RELEASE_DIR}/build-info.txt"
836+
echo " ✓ build-info.txt"
837+
838+
# Generate VALIDATION.txt
839+
echo "==> Generating VALIDATION.txt..."
840+
CHECKSUM_STATUS="NO CHECKSUMS AVAILABLE"
841+
if [ -f "${DOWNLOAD_DIR}/CHECKSUMS.sha256" ]; then
842+
CHECKSUM_STATUS="PASSED"
843+
fi
844+
ARTIFACT_LIST=$(ls "${DOWNLOAD_DIR}"/*.whl "${DOWNLOAD_DIR}"/*.tar.gz 2>/dev/null | while read f; do echo " - $(basename "$f")"; done)
845+
{
846+
echo "Validation Report for ${TAG}"
847+
echo "Generated: ${TIMESTAMP}"
848+
echo ""
849+
echo "Source Verification:"
850+
echo " - Downloaded from: GitHub Releases (crossbario/cfxdb)"
851+
echo " - Release tag: ${TAG}"
852+
echo " - Checksum verification: ${CHECKSUM_STATUS}"
853+
echo ""
854+
echo "Artifact List:"
855+
echo "${ARTIFACT_LIST}"
856+
echo ""
857+
echo "This package is a pure Python wheel (py3-none-any)."
858+
echo "No native code compilation or platform-specific verification required."
859+
} > "${DOCS_RELEASE_DIR}/VALIDATION.txt"
860+
echo " ✓ VALIDATION.txt"
861+
862+
echo ""
863+
echo "==> Documentation integration complete."
864+
echo " Files written to: ${DOCS_RELEASE_DIR}/"
865+
ls -la "${DOCS_RELEASE_DIR}/"
866+
713867
# -----------------------------------------------------------------------------
714868
# -- Release workflow recipes
715869
# -----------------------------------------------------------------------------

pyproject.toml

Lines changed: 16 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
44

55
[project]
66
name = "cfxdb"
7-
version = "25.12.1"
7+
version = "25.12.2"
88
description = "Crossbar.io Database - Core database access classes for Crossbar.io"
99
readme = "README.md"
1010
requires-python = ">=3.11"
@@ -40,18 +40,26 @@ classifiers = [
4040
]
4141

4242
dependencies = [
43-
# WAMP ecosystem packages
44-
"autobahn[all]>=25.12.1",
45-
"zlmdb>=25.12.1",
46-
47-
# Ethereum/Web3 stack
43+
# WAMP ecosystem packages:
44+
# - zlmdb: Core dependency for LMDB-based object-relational database layer.
45+
# cfxdb defines database schemas and table classes that extend zlmdb's
46+
# MapOidFlatBuffers, MapUuidFlatBuffers, etc. base classes.
47+
# - autobahn: Required for WAMP protocol types used in database schemas
48+
# (e.g., autobahn.wamp.types.TransportDetails in realmstore tables,
49+
# autobahn.wamp.serializer.JsonObjectSerializer in exporter, and
50+
# autobahn.util for ID generation and activation codes in tests).
51+
# The [all] extra ensures FlatBuffers serialization support is available.
52+
"zlmdb>=25.12.2",
53+
"autobahn[all]>=25.12.2",
54+
55+
# Ethereum/Web3 stack (for XBR market maker and network schemas)
4856
"eth-abi>=5.1.0",
4957
"eth-account>=0.13.0",
5058
"web3>=7.6.0",
5159

5260
# Additional dependencies
53-
"parsimonious>=0.10.0",
54-
"argon2-cffi>=25.1.0",
61+
"parsimonious>=0.10.0", # PEG parser for schema definitions
62+
"argon2-cffi>=25.1.0", # Password hashing for user authentication
5563
]
5664

5765
[project.optional-dependencies]
@@ -66,8 +74,6 @@ dev = [
6674
"pytest>=3.4.2",
6775
"pytest-runner>=2.11.1",
6876
"coverage>=5.0.0",
69-
"tox>=2.9.1",
70-
"tox-gh-actions>=2.2.0",
7177
"py-multihash>=2.0.1", # For IPFS hash generation in tests
7278

7379
# Code quality

src/cfxdb/_version.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,4 @@
55
#
66
##############################################################################
77

8-
__version__ = "25.12.1"
8+
__version__ = "25.12.2"

0 commit comments

Comments
 (0)