Skip to content

Commit 41a1f43

Browse files
Merge branch 'dev' into LEGLINK-840
2 parents 8dd7d29 + 73d5b0c commit 41a1f43

49 files changed

Lines changed: 15802 additions & 168 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.coderabbit.yaml

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -42,14 +42,22 @@ reviews:
4242
Pull requests that have "TECH_DEBT" in the title should only contain changes related to typos, unused code, linter/IDE suggestions, swagger specification updates,
4343
and logging improvements. These TECH_DEBT PRs must not affect core functionality. All PRs that are not considered technical debt must include information on what
4444
testing was performed in the description of the PR. If it does not, ask the author to provide details on what testing was performed.
45+
If a service REST controller is changed, ensure the `LinkSdk` is updated to reflect the controller changes.
4546
When reviewing code, suggest unit tests using XUnit in the following scenarios:
4647
- If/Else or Switch/Case blocks are introduced or modified — ensure each branch has a corresponding unit test.
4748
- Logic that depends on service or interface configuration — suggest tests to validate different implementations are correctly resolved.
4849
- No network activity (HTTP calls, sockets, etc.) should appear in unit tests. Recommend using mocks (via Moq) for any external communication.
4950
Large unit tests should be avoided; keeping unit tests small and focused on targeted business logic (i.e. string sanitization)
5051
All logging message arguments must be sanitized before they are passed to logger methods to avoid static code scanning security/vulnerability findings.
5152
52-
**App Config Check**:
53-
If new required configuration keys (that don't have default values) are introduced, verify if new configuration keys have been added. If new required configuration keys are introduced, ensure that the
54-
root-level `/app-config.yaml` file has been updated to include these new keys with appropriate descriptions. If `/app-config.yaml` is not updated
55-
when new configuration is added, flag this as a required change and recommend blocking the PR until the configuration is documented in `/app-config.yaml`.
53+
**App Config Check**:
54+
If new required configuration keys (that don't have default values) are introduced, verify if new configuration keys have been added. If new required configuration keys are introduced, ensure that the
55+
root-level `/app-config.yaml` file has been updated to include these new keys with appropriate descriptions. If `/app-config.yaml` is not updated
56+
when new configuration is added, flag this as a required change and recommend blocking the PR until the configuration is documented in `/app-config.yaml`.
57+
The catalog is curated, not exhaustive: a key belongs in it only if it is provisioned per environment, has no safe default, or is a documented
58+
operational knob. A key with a working default in `appsettings.json` or `application.yml` should be `required: false` with that value recorded in
59+
`defaultValue`, because `required: true` obligates a row in all three environment stores. The exhaustive list of every key the code reads is the
60+
generated `docs/config-key-inventory.md`, not this catalog.
61+
Schema conformance and required-key presence are enforced mechanically by
62+
`.github/workflows/appconfig-catalog-check.yml`, so review effort here is better spent on whether an entry's `required` flag and description are
63+
right than on whether the key is present.

.gitattributes

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,18 @@
55
*.sh text eol=lf
66
/topics.txt text eol=lf
77

8+
# Git hooks are shell scripts without a .sh extension. With core.autocrlf=true they would be
9+
# checked out with CRLF, making the shebang "#!/bin/sh\r". Git for Windows' bundled bash
10+
# tolerates that, but a real Linux shell (WSL, CI, a non-Windows developer) reports
11+
# "bad interpreter" and the hook silently stops guarding anything.
12+
/.githooks/** text eol=lf
13+
14+
# The mirror image for batch files. cmd.exe reads a .bat by byte offset rather than parsing
15+
# it whole, so an LF-only file can resume mid-line after a GOTO and run the wrong thing.
16+
# export-appconfigs.bat uses GOTO :usage, which is exactly that case.
17+
*.bat text eol=crlf
18+
*.cmd text eol=crlf
19+
820
###############################################################################
921
# Set default behavior for command prompt diff.
1022
#

.githooks/pre-commit

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
#!/bin/sh
2+
#
3+
# Guards the App Configuration exports and the catalog that describes them.
4+
#
5+
# Config/*.json scanned for credentials
6+
# app-config.yaml validated against its own embedded schema
7+
#
8+
# Enable for your clone with:
9+
# git config core.hooksPath .githooks
10+
#
11+
# Validates the STAGED content, not the working tree, so a problem cannot slip through by
12+
# being fixed in the file after `git add`.
13+
#
14+
# Bypass with `git commit --no-verify`. CI runs the same checks either way.
15+
16+
set -e
17+
18+
staged_exports=$(git diff --cached --name-only --diff-filter=ACM \
19+
| grep -E '^Config/.*\.json$' || true)
20+
staged_catalog=$(git diff --cached --name-only --diff-filter=ACM \
21+
| grep -E '^app-config\.yaml$' || true)
22+
23+
if [ -z "$staged_exports" ] && [ -z "$staged_catalog" ]; then
24+
exit 0
25+
fi
26+
27+
# Probe that the interpreter actually runs. On Windows, `python3` often resolves
28+
# to the Microsoft Store alias, which exists on PATH but exits non-zero -- so
29+
# `command -v` alone picks an interpreter that cannot execute anything.
30+
PYTHON=""
31+
for candidate in python3 python py; do
32+
if command -v "$candidate" >/dev/null 2>&1 &&
33+
"$candidate" -c "import sys" >/dev/null 2>&1; then
34+
PYTHON="$candidate"
35+
break
36+
fi
37+
done
38+
39+
if [ -z "$PYTHON" ]; then
40+
echo "pre-commit: no working python found, skipping App Config checks."
41+
echo "pre-commit: CI will still run them."
42+
exit 0
43+
fi
44+
45+
# Materialize staged blobs so the checks see exactly what is being committed.
46+
tmpdir=$(mktemp -d)
47+
trap 'rm -rf "$tmpdir"' EXIT
48+
49+
stage_blob() {
50+
dest="$tmpdir/$(basename "$1")"
51+
git show ":$1" > "$dest"
52+
echo "$dest"
53+
}
54+
55+
if [ -n "$staged_exports" ]; then
56+
files=""
57+
for path in $staged_exports; do
58+
files="$files $(stage_blob "$path")"
59+
done
60+
61+
# --strict matches appconfig-secret-scan.yml, so the hook cannot pass something CI will
62+
# reject. Errors block either way; --strict is only about warnings - a secret-shaped key
63+
# holding a literal, a duplicate (key, label), a malformed entry. Those are worth stopping
64+
# for here because these files go to a public repository, where a false positive costs a
65+
# --no-verify and a false negative costs a rotated credential and permanent git history.
66+
#
67+
# It does not make the two identical: this scans the staged blob, CI scans all three files
68+
# on disk. A warning in a file you did not stage still passes here and fails there.
69+
# shellcheck disable=SC2086
70+
if ! "$PYTHON" Scripts/AzureAppConfig/validate_aac_secrets.py --strict $files; then
71+
echo ""
72+
echo "pre-commit: BLOCKED -- staged App Config export failed the secret scan."
73+
echo "pre-commit: An ERROR is a credential: move it into Key Vault and reference it,"
74+
echo "pre-commit: then re-stage. A WARNING is something worth a look rather than"
75+
echo "pre-commit: certainly wrong; it blocks here because CI runs --strict too."
76+
echo "pre-commit: If the finding is wrong, commit with --no-verify."
77+
exit 1
78+
fi
79+
fi
80+
81+
if [ -n "$staged_catalog" ]; then
82+
catalog=$(stage_blob "$staged_catalog")
83+
if ! "$PYTHON" Scripts/AzureAppConfig/validate_app_config_schema.py "$catalog"; then
84+
echo ""
85+
echo "pre-commit: BLOCKED -- staged app-config.yaml does not match its own schema."
86+
echo "pre-commit: Fix the entries above, or commit with --no-verify."
87+
exit 1
88+
fi
89+
fi
90+
91+
# The required-key check is deliberately NOT run here. It compares the catalog against all
92+
# three stores, so it would fail on gaps a commit did not introduce and cannot fix. CI reports
93+
# it instead.
94+
95+
exit 0

.github/CODEOWNERS

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,13 @@
33
**/pom.xml @lantanagroup/link-release-management-team
44
**/application*.yml @lantanagroup/link-release-management-team @lantanagroup/devops
55

6+
# The configuration catalog and the per-environment App Configuration exports. These are what
7+
# the App Config import pipeline deploys, so a change here changes deployed configuration.
8+
# Note these are not covered by the patterns above: application*.yml matches neither the stem
9+
# nor the extension of app-config.yaml.
10+
/app-config.yaml @lantanagroup/link-release-management-team @lantanagroup/devops
11+
Config/app-config*.json @lantanagroup/link-release-management-team @lantanagroup/devops
12+
613
docs/* @lantanagroup/devops @lantanagroup/link-arch
714

815
# Similarly, the DevOps team is interested in any changes to Kafka topics
Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
name: "App Config Catalog Check"
2+
3+
# Verifies app-config.yaml describes reality: every key marked required: true has a row in
4+
# every environment store, no store row carries a label no service selects, and the Serilog
5+
# sink index the catalog depends on is still pinned.
6+
#
7+
# LEGLINK-775 will make a pipeline import Config/app-config.*.json into the stores, at which
8+
# point a missing required key becomes a deployment defect rather than a documentation one.
9+
#
10+
# Runs on every PR rather than filtering on paths, so it stays valid as a required status
11+
# check. A path filter would leave PRs that touch nothing here reporting no status at all.
12+
on:
13+
pull_request:
14+
branches:
15+
- dev
16+
- main
17+
- 'release/**'
18+
- 'hotfix/**'
19+
push:
20+
branches:
21+
- dev
22+
- main
23+
- 'release/**'
24+
- 'hotfix/**'
25+
merge_group:
26+
branches:
27+
- dev
28+
29+
# This job only reads the repository: checkout, install PyYAML, then run the catalog checks.
30+
# Nothing is written back, so the token needs no more than read access.
31+
permissions:
32+
contents: read
33+
34+
jobs:
35+
check:
36+
name: Validate catalog and required keys
37+
runs-on: ubuntu-latest
38+
steps:
39+
- name: Check out repository
40+
uses: actions/checkout@v4
41+
42+
- name: Set up Python
43+
uses: actions/setup-python@v5
44+
with:
45+
python-version: '3.11'
46+
47+
# validate_aac_secrets.py is stdlib-only; these two need a YAML parser.
48+
- name: Install dependencies
49+
run: pip install "pyyaml==6.0.*"
50+
51+
- name: Unit tests
52+
run: python -m unittest discover Scripts/AzureAppConfig/tests
53+
54+
- name: Validate catalog schema
55+
run: python Scripts/AzureAppConfig/validate_app_config_schema.py
56+
57+
# Enforcing. The gaps this was advisory for are closed: the three required keys now have
58+
# rows in every environment file and the orphaned "Automation" label is gone.
59+
#
60+
# A failure here means app-config.yaml and Config/app-config.*.json have drifted - most
61+
# often a key marked required: true in the catalog with no row added to each environment
62+
# file. Fix it in the same PR: either add the row, or set required: false and record the
63+
# shipped default in defaultValue.
64+
- name: Check required keys are provisioned
65+
run: python Scripts/AzureAppConfig/check_required_config.py
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
name: "App Config Secret Scan"
2+
3+
# The exports under Config/ come from the Azure App Configuration stores and are
4+
# committed to a public repository. App Configuration will happily hold a literal
5+
# credential, so without this gate an export can carry one into permanent git
6+
# history. Runs on every PR rather than filtering on paths so it stays usable as
7+
# a required status check.
8+
on:
9+
pull_request:
10+
branches:
11+
- dev
12+
- main
13+
- 'release/**'
14+
- 'hotfix/**'
15+
push:
16+
branches:
17+
- dev
18+
- main
19+
- 'release/**'
20+
- 'hotfix/**'
21+
merge_group:
22+
branches:
23+
- dev
24+
25+
# This job only reads the repository: checkout, then run a stdlib-only Python script over
26+
# Config/*.json. Nothing is written back, so the token needs no more than read access.
27+
permissions:
28+
contents: read
29+
30+
jobs:
31+
scan:
32+
name: Scan App Config exports
33+
runs-on: ubuntu-latest
34+
steps:
35+
- name: Check out repository
36+
uses: actions/checkout@v4
37+
38+
- name: Set up Python
39+
uses: actions/setup-python@v5
40+
with:
41+
python-version: '3.11'
42+
43+
- name: Validate App Config exports
44+
run: python Scripts/AzureAppConfig/validate_aac_secrets.py "Config/*.json" --strict

.gitignore

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -414,4 +414,12 @@ FodyWeavers.xsd
414414
/DotNet/Submission/submissions
415415
/DotNet/Census/appsettings.Local.json
416416
/DotNet/Normalization/efpt.config.json
417-
/Java/**/application-dev.yml
417+
/Java/**/application-dev.yml
418+
# Generated by the config-key inventory tooling. Both are derived from source, so a
419+
# committed copy would drift from the code it describes - the exact problem this tooling
420+
# exists to detect. docs/config-key-inventory.md IS committed: it is the human-readable
421+
# deliverable. Regenerate with:
422+
# dotnet run --file Scripts/AzureAppConfig/dump_config_symbols.cs -- DotNet Scripts/AzureAppConfig/config_symbols.json
423+
# python Scripts/AzureAppConfig/extract_config_keys.py
424+
Scripts/AzureAppConfig/config_symbols.json
425+
Config/config-key-inventory.json

0 commit comments

Comments
 (0)