give convert_BAM2Pileup 5G - #354
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR increases the memory allocation for the convert_BAM2Pileup process from 2 GB to 5 GB to address out-of-memory errors that were occurring in 15% of recent runs.
- Increases minimum and maximum memory allocation from 2 GB to 5 GB for convert_BAM2Pileup
- Updates memory fraction from 0.01 to 0.07 to reflect the increased allocation
- Documents the change in the CHANGELOG.md
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| config/resources.json | Updates memory configuration for convert_BAM2Pileup process |
| CHANGELOG.md | Documents the memory allocation increase |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.
| "fraction": 0.01, | ||
| "max": "2 GB" | ||
| "min": "5 GB", | ||
| "fraction": 0.07, |
There was a problem hiding this comment.
[nitpick] The memory fraction of 0.07 appears arbitrary and may not accurately represent the relationship between the 5 GB allocation and total available memory. Consider documenting the calculation basis or using a fraction that reflects actual memory requirements relative to system capacity.
| "min": "2 GB", | ||
| "fraction": 0.01, | ||
| "max": "2 GB" | ||
| "min": "5 GB", |
There was a problem hiding this comment.
suggestion: It may be worth increasing the value in the default profile as well
Description
22 out of 149 (15%) of recent
call-sSNVruns had out of memory errors forsomaticsniper:convert_BAM2Pileup_SAMtool. They all worked on automatic retry. The initial (failed) runs took an average of1 hr 47 min. Looking at thetrace.txtfiles, all but one used less than5 GBin the successful retry and that one was barely over and might have worked with5 GB.I haven't tested it because it doesn't seem worth running it on an F72.
Checklist
I have read the code review guidelines and the code review best practice on GitHub check-list.
I have reviewed the Nextflow pipeline standards.
The name of the branch is meaningful and well formatted following the standards, using [AD_username (or 5 letters of AD if AD is too long)]-[brief_description_of_branch].
I have set up or verified the branch protection rule following the github standards before opening this pull request.
I have added my name to the contributors listings in the
manifestblock in thenextflow.configas part of this pull request, am listed already, or do not wish to be listed. (This acknowledgement is optional.)I have added the changes included in this pull request to the
CHANGELOG.mdunder the next release version or unreleased, and updated the date.I have updated the version number in the
metadata.yamlandmanifestblock of thenextflow.configfile following semver, or the version number has already been updated. (Leave it unchecked if you are unsure about new version number and discuss it with the infrastructure team in this PR.)I have tested the pipeline on at least one A-mini sample.