grass.script: Add available_cpus and resolve_nprocs helpers#7414
Open
petrasovaa wants to merge 1 commit into
Open
grass.script: Add available_cpus and resolve_nprocs helpers#7414petrasovaa wants to merge 1 commit into
petrasovaa wants to merge 1 commit into
Conversation
Affinity-aware CPU count and G_OPT_M_NPROCS resolution (positive as-is, 0 = all cores, negative reserves cores). Migrate ad-hoc cpu_count() sites in pygrass, jupyter, animation, and datacatalog GUI to the new helpers; all become container/cgroup aware. Fix t.rast.univar and t.rast3d.univar: nprocs=0 was silently clamped to 1 instead of using all available cores. New in grass.script public API in GRASS 8.6.
github-actions
Bot
added
GUI
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We are using
G_set_omp_num_threadsthat resolves nprocs <= 0, but in Python, we don't have equivalent, potentially resulting in bugs. This PR adds the function for resolving the nprocs and also function for getting available cpus that works better in containers.available_cpus()— number of CPUs this process may actually use. Prefers affinity-aware sources(
os.process_cpu_counton 3.13+,os.sched_getaffinityon Linux) overos.cpu_count, which reports the host totaland overcounts in containers and cgroup-limited jobs.
resolve_nprocs(nprocs)— mirrorsG_set_omp_num_threads()inlib/gis/omp_threads.c:0→ all available coresavailable + nprocs, clamped to at least 1These functions are applied in the code base, resulting in some bug fixes:
t.rast.univar/t.rast3d.univar:nprocs=0(documented as "use all available cores") was silently clamped to 1, so the tool ran serially. The tool entry points now applygs.resolve_nprocs()so the documentedG_OPT_M_NPROCSsemantics take effect for0and negative values.AI-assisted (Claude Code) — used for migration planning, edge case analysis, and drafting docstrings and tests.