Skip to content

Fix ptxinfo parser for CUDA 11+ ptxas resource-usage output - #138

Open
AndrewVu23 wants to merge 2 commits into
accel-sim:devfrom
AndrewVu23:fix/ptxinfo-modern-ptxas-format
Open

Fix ptxinfo parser for CUDA 11+ ptxas resource-usage output#138
AndrewVu23 wants to merge 2 commits into
accel-sim:devfrom
AndrewVu23:fix/ptxinfo-modern-ptxas-format

Conversation

@AndrewVu23

@AndrewVu23 AndrewVu23 commented Jul 23, 2026

Copy link
Copy Markdown

Summary

GPGPU-Sim's current PTX-mode simulation cannot run any kernel compiled with a CUDA 11+ toolkit. Reason is that the parser that reads ptxas's resource-usage output chokes on two fields modern ptxas added, aborting before a single cycle is simulated. This makes every stat come back NA on current toolchains (including the CUDA 12.8 used by CI). This PR teaches the lexer/grammar those two fields with a 2-file, additive change, restoring PTX-mode simulation on CUDA 11/12.

Motivation / root cause

ptxas from CUDA 11+ emits used N barriers and (on register spills) N bytes cumulative stack size in its ptxas info summary line. GPGPU-Sim's ptxinfo.l/ptxinfo.y grammar predates both, so kernels fail to parse:

Bug:

GPGPU-Sim: ERROR while parsing output of ptxas (used to capture resource usage information)
    lonestar-bfs-atomic.4.sm_75.ptx (lonestar-bfs-atomic.1.sm_60.ptxas:5) Syntax error:
   ptxas info    : Used 40 registers, used 0 barriers, 56 bytes cumulative stack size, 440 bytes cmem[0]
GPGPU-Sim: *** exit detected ***

We can clearly see the fields ptxas emits, which lead us to a new error:

  • cumulative stack size unhandled (spilling kernels): Introduced in modern CUDA ptxas. No grammar rule accepts this phrase, so any kernel that spills registers to local memory hits a syntax error and *** exit detected ***

And while I was adding the new field, I found out another bug:

  • barriers mis-tokenized (every kernel): ptxinfo.l:62 returns the REGS token instead of the BARRIERS even though the grammar rule (ptxinfo.y) and handler (ptxinfo_barriers() in cuda-sim.cc) for barriers already exist -> not really a compatibility issue with previous ptxas versions. It's just wrong. Therefore, used N barriers re-matches the register rule and overwrites the parsed register count with the barrier count on every kernel. Note: the barrier value itself currently flows into a field nothing consumes downstream, so the benefit of this fix is mostly preventing register-count corruption, not enabling barrier modeling.

I believe that this only happens when a kernel is too heavy -> register spill to local memory -> ptxas emits cumulative stack size, which is not handled yet. In other testsuites (like rodinia-3.1 & rodinia_2.0-ft) the PTX sim runs just fine.

Changes

  • src/cuda-sim/ptxinfo.l: return the BARRIERS token instead of REGS; add a lexer rule matching the literal phrase "cumulative stack size" → new STACKSIZE token.
  • src/cuda-sim/ptxinfo.y: declare %token STACKSIZE; add production INT_OPERAND BYTES STACKSIZE { ptxinfo_lmem($1,0); }, routing the value into the existing ptxinfo_lmem() handler (same per-thread local-memory quantity ptxas used to report as lmem).

Evidence (before / after)

Before: every kernel aborts at ptxas-info parsing → get_stats.py returns NA for all stats, all apps

After: same simulator, same apps, real stats (lonestargpu-2.0 sssp, RTX3070 config)
image
Note: the reason why bfs-wlc failed was because I haven't built the binaries/data for it.

Test plan

  • ./format-code.sh clean
  • compile any spilling kernel and run it in PTX mode — before this change it prints ERROR while parsing output of ptxas / *** exit detected ***; after, it simulates and get_stats.py reports non-NA gpu_tot_sim_insn.

Compatibility / backward compatibility

Purely additive. No existing rule is changed or removed.

Limitations / known gaps

  • Grammar audit scoped to the lonestargpu-2.0 corpus, which reduces to 2 normalized ptxas info line shapes; other kernel shapes (heavy shared-memory/texture use) may emit clauses not exercised here.
    - cumulative stack size is routed to ptxinfo_lmem() on the assumption it's ptxas's renamed successor to the older lmem field (same per-thread local/stack memory). Flagged for reviewer confirmation.
  • Other lonestargpu-2.0 apps have unrelated, pre-existing issues (some binaries don't build; bh hangs; bfs-atomic aborts mid-sim) -> out of scope for this parser fix.
  • Note: lonestargpu-2.0 took a lot of time to compile. I had to terminate sssp early since I didn't have much time (I ran it for 2 hours straigth) & WSL kept crashing/disconnecting. My laptop is not built for this.

Related

Link to PR that started everything


For reviewers

  • The barriers→REGS mis-routing and its register-corruption effect match the diff
  • cumulative stack size → ptxinfo_lmem() is the correct semantic mapping

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant