Fix ptxinfo parser for CUDA 11+ ptxas resource-usage output - #138
Open
AndrewVu23 wants to merge 2 commits into
Open
Fix ptxinfo parser for CUDA 11+ ptxas resource-usage output#138AndrewVu23 wants to merge 2 commits into
AndrewVu23 wants to merge 2 commits into
Conversation
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
GPGPU-Sim's current PTX-mode simulation cannot run any kernel compiled with a CUDA 11+ toolkit. Reason is that the parser that reads ptxas's resource-usage output chokes on two fields modern ptxas added, aborting before a single cycle is simulated. This makes every stat come back NA on current toolchains (including the CUDA 12.8 used by CI). This PR teaches the lexer/grammar those two fields with a 2-file, additive change, restoring PTX-mode simulation on CUDA 11/12.
Motivation / root cause
ptxas from CUDA 11+ emits used
N barriersand (on register spills)N bytes cumulative stack sizein its ptxas info summary line. GPGPU-Sim's ptxinfo.l/ptxinfo.y grammar predates both, so kernels fail to parse:Bug:
We can clearly see the fields ptxas emits, which lead us to a new error:
cumulative stack sizeunhandled (spilling kernels): Introduced in modern CUDA ptxas. No grammar rule accepts this phrase, so any kernel that spills registers to local memory hits a syntax error and*** exit detected ***And while I was adding the new field, I found out another bug:
barriersmis-tokenized (every kernel): ptxinfo.l:62 returns theREGStoken instead of theBARRIERSeven though the grammar rule (ptxinfo.y) and handler (ptxinfo_barriers()incuda-sim.cc) for barriers already exist -> not really a compatibility issue with previous ptxas versions. It's just wrong. Therefore,used N barriersre-matches the register rule and overwrites the parsed register count with the barrier count on every kernel. Note: the barrier value itself currently flows into a field nothing consumes downstream, so the benefit of this fix is mostly preventing register-count corruption, not enabling barrier modeling.I believe that this only happens when a kernel is too heavy -> register spill to local memory -> ptxas emits
cumulative stack size, which is not handled yet. In other testsuites (like rodinia-3.1 & rodinia_2.0-ft) the PTX sim runs just fine.Changes
src/cuda-sim/ptxinfo.l: return theBARRIERStoken instead ofREGS; add a lexer rule matching the literal phrase"cumulative stack size"→ newSTACKSIZEtoken.src/cuda-sim/ptxinfo.y: declare%token STACKSIZE; add productionINT_OPERAND BYTES STACKSIZE { ptxinfo_lmem($1,0); }, routing the value into the existingptxinfo_lmem()handler (same per-thread local-memory quantity ptxas used to report as lmem).Evidence (before / after)
Before: every kernel aborts at ptxas-info parsing → get_stats.py returns NA for all stats, all apps
After: same simulator, same apps, real stats (lonestargpu-2.0 sssp, RTX3070 config)

Note: the reason why bfs-wlc failed was because I haven't built the binaries/data for it.
Test plan
Compatibility / backward compatibility
Purely additive. No existing rule is changed or removed.
Limitations / known gaps
lonestargpu-2.0corpus, which reduces to 2 normalized ptxas info line shapes; other kernel shapes (heavy shared-memory/texture use) may emit clauses not exercised here.-
cumulative stack sizeis routed toptxinfo_lmem()on the assumption it's ptxas's renamed successor to the older lmem field (same per-thread local/stack memory). Flagged for reviewer confirmation.lonestargpu-2.0apps have unrelated, pre-existing issues (some binaries don't build; bh hangs; bfs-atomic aborts mid-sim) -> out of scope for this parser fix.lonestargpu-2.0took a lot of time to compile. I had to terminatessspearly since I didn't have much time (I ran it for 2 hours straigth) & WSL kept crashing/disconnecting. My laptop is not built for this.Related
Link to PR that started everything
For reviewers