Skip to content

Split SQL files to reduce memory load for claims#910

Merged
amartinson193 merged 3 commits into
mainfrom
fix-mem-issue-250611
Jun 13, 2025
Merged

Split SQL files to reduce memory load for claims#910
amartinson193 merged 3 commits into
mainfrom
fix-mem-issue-250611

Conversation

@amartinson193

@amartinson193 amartinson193 commented Jun 12, 2025

Copy link
Copy Markdown
Collaborator

Describe your changes

The union + window function required more of the model to be in memory than if they were split apart. This becomes more noticeable with larger files. This update is related to this PR here. This one is applied to claims instead of clinical data since it suffers from the same issue.

How has this been tested?

Ran dbt build

Reviewer focus

None

Checklist before requesting a review

  • I have added at least one Github label to this PR (bug, enhancement, breaking change,...)
  • My code follows style guidelines
  • (New models) YAML files are categorized by sub folder and models listed in alphabetical order
  • (New models) I have added a config to each new model to enable it for claims and/or clinical data
  • (New models) I have added the variable tuva_last_run to the final output
  • (Optional) I have recorded a Loom to explain this PR

Package release checklist

  • I have updated dbt docs
  • I have updated the version number in the dbt_project.yml

(Optional) Gif of how this PR makes you feel

Loom link

@github-project-automation github-project-automation Bot moved this to 👀 Ready for Review in The Tuva Project Jun 12, 2025
@chase-jones
chase-jones self-requested a review June 12, 2025 16:34
@chase-jones chase-jones added enhancement New feature or request v0.14.14 labels Jun 12, 2025
@amartinson193
amartinson193 merged commit 59c2277 into main Jun 13, 2025
9 checks passed
@amartinson193
amartinson193 deleted the fix-mem-issue-250611 branch June 13, 2025 16:30
@github-project-automation github-project-automation Bot moved this from 👀 Ready for Review to 🚀 Ready to Release in The Tuva Project Jun 13, 2025
@sarahmcmorgan sarahmcmorgan moved this from 🚀 Ready to Release to ✅ Done in The Tuva Project Jun 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants