Skip to content

TECH_DEBT: Dops 487 Updating DB Migration Pipeline w/ Automation - #1332

Merged
kissalk merged 8 commits into
devfrom
dops-487
Mar 3, 2026
Merged

TECH_DEBT: Dops 487 Updating DB Migration Pipeline w/ Automation#1332
kissalk merged 8 commits into
devfrom
dops-487

Conversation

@kissalk

@kissalk kissalk commented Dec 18, 2025

Copy link
Copy Markdown
Contributor

This PR will add support to deploy the Database Migration scripts to each environment

🧑‍🔬 Unit Testing

  • Coverage: 0.0%

@coderabbitai

coderabbitai Bot commented Dec 18, 2025

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This pull request modifies GitHub and Azure Pipeline configurations. A minimal formatting adjustment updates the GitHub PR title check workflow. The Azure Pipeline for database schema creation is substantially expanded with new TEST and QA deployment stages, standardized per-service SQL artifact naming conventions, and corresponding per-service SqlAzureDacpacDeployment tasks.

Changes

Cohort / File(s) Change Summary
GitHub Workflow
.github/workflows/pr-title-check.yml
Formatting/newline adjustment to the closing block; no functional change to logic or control flow.
Azure Pipeline Database Schema Creation
Azure_Pipelines/azure-pipelines.db-schema-creation.yaml
Added variable group MSSQL_DBCreation; introduced TEST and QA deployment stages with per-service SqlAzureDacpacDeployment jobs; standardized SQL artifact naming (e.g., accountDb.sql → account-db); refactored job display names and stage names; implemented per-service deployment steps for nine database schemas (account, audit, census, data, normalization, querydispatch, submission, tenant, validation).

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

  • Azure Pipeline changes: Substantial new stages and deployment logic; per-service SQL artifact naming and publishing conventions applied across nine database schemas; risk of artifact-to-database mismatches if job and artifact names are misaligned.
  • Areas requiring extra attention:
    • Verification that each per-service deployment job pulls the correct artifact and applies it to the intended TEST/QA database target.
    • Consistency of SQL script naming conventions across all nine schemas and their corresponding artifact folders.
    • Validation of the new MSSQL_DBCreation variable group usage and scope within pipeline stages.

Possibly related PRs

  • #1324: Modifies the same Azure pipeline to convert dotnet-ef usage to local tool manifests and publish per-service SQL scripts with aligned naming conventions.
  • #478: Adjusts DB schema script generation and artifact publishing targets in the same Azure pipeline file.
  • #769: Alters SQL script generation and deployment job definitions in the same pipeline configuration.

Suggested reviewers

  • amphillipsLGC
  • edward-miller-lcg

Poem

🐰 Per-service schemas hop in line,
TEST and QA stages now align,
Artifacts named with clarity bright,
Database deployments set just right,
The pipeline flows with SQL delight! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The PR description is incomplete. While it mentions the objective of adding database migration script deployment support, it lacks detailed information in critical sections required by the template. Complete the 'Description of Changes' section with specific details about what was modified. Add a 'Testing Performed' section describing how the changes were tested. Replace template placeholders with actual information.
✅ Passed checks (2 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Title check ✅ Passed The title references a real change (DB Migration Pipeline updates) but lacks specificity about the main modifications and uses a generic ticket reference without clear context.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch dops-487

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
Azure_Pipelines/azure-pipelines.db-schema-creation.yaml (1)

354-355: Inconsistent job naming convention.

Job names mix PascalCase (UpdateAccountSchema) with partially lowercase names (UpdateauditSchema, UpdatecensusSchema). For consistency and readability, consider standardizing to PascalCase throughout (e.g., UpdateAuditSchema, UpdateCensusSchema).

Also applies to: 372-373, 390-391

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

  • Jira integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between a2cdc1c and dcd9686.

📒 Files selected for processing (2)
  • .github/workflows/pr-title-check.yml (1 hunks)
  • Azure_Pipelines/azure-pipelines.db-schema-creation.yaml (10 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**

⚙️ CodeRabbit configuration file

**: Pull requests that have "TECH_DEBT" in the title should only contain changes related to typos, unused code, linter/IDE suggestions, swagger specification updates,
and logging improvements. These TECH_DEBT PRs must not affect core functionality. All PRs that are not considered technical debt must include information on what
testing was performed in the description of the PR. If it does not, ask the author to provide details on what testing was performed.
When reviewing code, suggest unit tests using XUnit in the following scenarios:

  • If/Else or Switch/Case blocks are introduced or modified — ensure each branch has a corresponding unit test.
  • Logic that depends on service or interface configuration — suggest tests to validate different implementations are correctly resolved.
  • No network activity (HTTP calls, sockets, etc.) should appear in unit tests. Recommend using mocks (via Moq) for any external communication.
    Large unit tests should be avoided; keeping unit tests small and focused on targeted business logic (i.e. string sanitization)

**: Pull requests that have DOCS in the title should only contain changes related to documentation within the /docs folder or in .md files through-out the code-base. The description
of the PR should specify what documentation was updated. Documentation updates should use EventCatalog.dev structure, where service-specific functionality should be described
in the service's index.mdx (i.e. /services/XXX/index.mdx or /domains/XXX/services/YYY/index.mdx). Configurations that are shared by multiple services should be
reflected in the /docs/docs/config files.

Files:

  • Azure_Pipelines/azure-pipelines.db-schema-creation.yaml
🧠 Learnings (3)
📓 Common learnings
Learnt from: sdmcgeown
Repo: lantanagroup/link-cloud PR: 0
File: :0-0
Timestamp: 2025-07-24T17:49:23.117Z
Learning: PR #1004 in the Link Cloud project introduced comprehensive normalization operation testing functionality including a new test operation dialog, JSON validation, resource type validation, and real-time testing capabilities that require documentation updates across multiple files in the docs folder.
Learnt from: CR
Repo: lantanagroup/link-cloud PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-24T18:43:37.952Z
Learning: PR descriptions must follow the template: 🛠️ Description of Changes, 🧪 Testing Performed, 🧑‍🔬 Unit Testing with checkbox, and 📓 Documentation Updated sections
Learnt from: seanmcilvenna
Repo: lantanagroup/link-cloud PR: 1241
File: Azure_Pipelines/azure-pipelines.db-schema-creation.yaml:251-270
Timestamp: 2025-11-06T19:08:10.347Z
Learning: In the lantanagroup/link-cloud repository, CI/CD pipeline changes (such as modifications to Azure Pipelines YAML files) are not considered functional features to the application and can be included in TECH_DEBT pull requests.
📚 Learning: 2025-11-06T19:08:10.347Z
Learnt from: seanmcilvenna
Repo: lantanagroup/link-cloud PR: 1241
File: Azure_Pipelines/azure-pipelines.db-schema-creation.yaml:251-270
Timestamp: 2025-11-06T19:08:10.347Z
Learning: In the lantanagroup/link-cloud repository, CI/CD pipeline changes (such as modifications to Azure Pipelines YAML files) are not considered functional features to the application and can be included in TECH_DEBT pull requests.

Applied to files:

  • Azure_Pipelines/azure-pipelines.db-schema-creation.yaml
📚 Learning: 2025-11-24T18:43:37.952Z
Learnt from: CR
Repo: lantanagroup/link-cloud PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-24T18:43:37.952Z
Learning: PR descriptions must follow the template: 🛠️ Description of Changes, 🧪 Testing Performed, 🧑‍🔬 Unit Testing with checkbox, and 📓 Documentation Updated sections

Applied to files:

  • .github/workflows/pr-title-check.yml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (13)
  • GitHub Check: Build Documentation
  • GitHub Check: Analyze (java-kotlin)
  • GitHub Check: Analyze (csharp)
  • GitHub Check: Database Script Generation (Build Generate Tenant Schema Script)
  • GitHub Check: Database Script Generation (Build Generate Validation Schema Script)
  • GitHub Check: Database Script Generation (Build Generate QueryDispatch Schema Script)
  • GitHub Check: Database Script Generation (Build Generate Submission Schema Script)
  • GitHub Check: Database Script Generation (Build Generate Data Acquisition Schema Script)
  • GitHub Check: Database Script Generation (Build Generate Normalization Schema Script)
  • GitHub Check: Database Script Generation (Build Generate Account Schema Script)
  • GitHub Check: Database Script Generation (Build Get Changes)
  • GitHub Check: Database Script Generation (Build Generate Audit Schema Script)
  • GitHub Check: Database Script Generation (Build Generate Census Schema Script)
🔇 Additional comments (4)
.github/workflows/pr-title-check.yml (1)

41-41: LGTM - Formatting-only change.

This appears to be a minor whitespace or formatting adjustment with no functional impact on the workflow.

Azure_Pipelines/azure-pipelines.db-schema-creation.yaml (3)

18-19: LGTM - Variable group for credentials.

Good practice to use a variable group (MSSQL_DBCreation) for database credentials rather than hardcoding them.


134-158: Good improvement: Per-service artifact naming.

The change from a generic scripts artifact to per-service artifacts (account-db, audit-db, etc.) improves traceability and allows selective downloads in deployment jobs. This is a cleaner approach. Based on learnings, CI/CD pipeline changes are appropriate for this repository.


680-689: Same hardcoded migration file issue as TEST stage.

This has the same problem as noted for the TEST stage (line 513) - only the initial schema migration is applied, missing any subsequent migrations.

⛔ Skipped due to learnings
Learnt from: smailliwcs
Repo: lantanagroup/link-cloud PR: 581
File: Java/validation/src/main/resources/application-local.yml:16-18
Timestamp: 2024-12-18T20:17:05.877Z
Learning: The project uses Flyway-style naming for migration files but does not actually use Flyway to apply them, which can affect future assumptions about schema versioning.

Comment thread Azure_Pipelines/azure-pipelines.db-schema-creation.yaml
Comment thread Azure_Pipelines/azure-pipelines.db-schema-creation.yaml
Comment thread Azure_Pipelines/azure-pipelines.db-schema-creation.yaml
@kissalk
kissalk force-pushed the dops-487 branch 2 times, most recently from 89d1c96 to 6d5073f Compare December 22, 2025 14:10
@kissalk kissalk changed the title Dops 487 Draft: Dops 487 Dec 22, 2025
@kissalk kissalk changed the title Draft: Dops 487 Draft: Dops 487 Updating DB Migration Pipeline w/ Automation Feb 23, 2026
@kissalk kissalk changed the title Draft: Dops 487 Updating DB Migration Pipeline w/ Automation TECH_DEBT: Dops 487 Updating DB Migration Pipeline w/ Automation Feb 24, 2026
@kissalk
kissalk requested review from seanmcilvenna and removed request for seanmcilvenna February 24, 2026 17:02
@kissalk kissalk changed the title TECH_DEBT: Dops 487 Updating DB Migration Pipeline w/ Automation DRAFT: TECH_DEBT: Dops 487 Updating DB Migration Pipeline w/ Automation Feb 24, 2026
@kissalk
kissalk force-pushed the dops-487 branch 3 times, most recently from 32e9981 to 7d477b4 Compare March 3, 2026 16:00
@kissalk kissalk changed the title DRAFT: TECH_DEBT: Dops 487 Updating DB Migration Pipeline w/ Automation TECH_DEBT: Dops 487 Updating DB Migration Pipeline w/ Automation Mar 3, 2026
@kissalk
kissalk merged commit 41a0487 into dev Mar 3, 2026
16 of 17 checks passed
@kissalk
kissalk deleted the dops-487 branch March 3, 2026 19:05
edward-miller-lcg pushed a commit that referenced this pull request Mar 3, 2026
* Updating for script

* updating artifacts

* updating account db deploy

* Adding Windows Pool for SQL Apply

* adding variable group

* adding TEST environment to Apply

* stashing

* changing server name -> variable
edward-miller-lcg pushed a commit that referenced this pull request Mar 3, 2026
* Updating for script

* updating artifacts

* updating account db deploy

* Adding Windows Pool for SQL Apply

* adding variable group

* adding TEST environment to Apply

* stashing

* changing server name -> variable
edward-miller-lcg added a commit that referenced this pull request Mar 3, 2026
* Update PatientDataService.cs

* Update PatientDataService.cs

* TECH_DEBT: Dops 487 Updating DB Migration Pipeline w/ Automation (#1332)

* Updating for script

* updating artifacts

* updating account db deploy

* Adding Windows Pool for SQL Apply

* adding variable group

* adding TEST environment to Apply

* stashing

* changing server name -> variable

---------

Co-authored-by: Keith Kissal <99497673+kissalk@users.noreply.github.com>
edward-miller-lcg added a commit that referenced this pull request Mar 3, 2026
* Fix Op Outcome retries

* LNK-4857

* add better handling for 404 and 410

* retrigger codeql

* Update PatientDataService.cs

* Update PatientDataServiceTests.cs

* Update PatientDataService.cs

* Update PatientDataServiceTests.cs

* TECH_DEBT: Dops 487 Updating DB Migration Pipeline w/ Automation (#1332)

* Updating for script

* updating artifacts

* updating account db deploy

* Adding Windows Pool for SQL Apply

* adding variable group

* adding TEST environment to Apply

* stashing

* changing server name -> variable

---------

Co-authored-by: Keith Kissal <99497673+kissalk@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant