Skip to content

build-packaging.md

Latest commit

 

History

History
413 lines (341 loc) · 19.2 KB

build-packaging.md

File metadata and controls

413 lines (341 loc) · 19.2 KB
Raw

Build and Packaging Automation

This repo now has two automation entrypoints:

  • doomgeo-build: build/install/package helper for the Neo Geo ROM workflow.
  • doomgeo-plan: plan-only tracker for docs/release-plan.md.

Both are plain Python scripts under tools/ and are packaged as standalone Linux and Windows binaries by GitHub Actions.

There is also a GitHub Pages bundle generated by doomgeo-build pages. It publishes a web-safe generated Neo Geo cartridge plus the null BIOS and loads them with EmulatorJS's FBNeo WebAssembly/asm.js frontend, so the current ROM can be played directly from the repository's Pages site. The same Pages bundle can expose the standalone 68000 assembly ROM at asm.html.

FBNeo is an arcade core and validates known romsets by filename, size, and CRC. For the Pages bundle only, doomgeo-build pages writes FBNeo-compatible launch zips against the magdrop2 driver by adjusting unused padding bytes in the generated ROM chips and null BIOS aliases. magdrop2 is used because its FBNeo driver has 4 MiB C1/C2 graphics ROM windows, matching the generated web sprite bank without cutting assets back to a smaller romset shape. The browser player launches magdrop2.zip and sets EJS_gameName = "magdrop2" because FBNeo identifies arcade games by driver/romset name. The bundle also writes project-named copies (doomgeo-aes.zip and doomgeo-aes-asm.zip) for download links, but those are not the files passed to the emulator. The raw build artifacts under build/rom/ are not rewritten this way; this compatibility package is only for the browser player.

The deployed Pages ROM is built separately from the normal native artifact. CI uses the redistributable Freedoom IWAD and DOOM_CROM_FILE_BYTES=4194304 for that job so the browser package does not embed proprietary Doom assets and still matches the fixed C-ROM chip sizes expected by FBNeo's magdrop2 driver. The regular Linux, Windows, and local ROM artifacts are left on the normal build path.

Local Linux Build

The existing Makefile still expects ngdevkit under .tools/ngdevkit-local/usr by default. If cached ngdevkit .deb files are present under .tools/downloads, install them locally with:

python3 tools/doomgeo_build.py install
python3 tools/doomgeo_build.py doctor
python3 tools/doomgeo_build.py build
python3 tools/doomgeo_build.py build --target asm-rom

To install from the upstream Ubuntu PPA instead, use:

python3 tools/doomgeo_build.py install-tools --method system
python3 tools/doomgeo_build.py build --tools-prefix /usr

The system method follows ngdevkit's documented Ubuntu path: ppa:dciabrin/ngdevkit, ngdevkit, and ngdevkit-gngeo.

For quick local render checks, make smoke-screenshot builds the default cart, launches GnGeo on SMOKE_DISPLAY (:1 by default), moves the window to SMOKE_WORKSPACE (2 by default), and writes .tools/screens/latest/smoke.png. Override SMOKE_BUILD_TARGET, SMOKE_RUN_TARGET, SMOKE_OUTPUT, SMOKE_WAIT_SECS, or SMOKE_START_GAME=1 when capturing variants or when a normal ROM should press B through the intro/menu before capture, such as the sample-map key/door ROM, combat verification ROM, sample encounter ROM, scout-route ROM, close-combat verification ROM, living-monster gallery ROM, weapons/keycard arsenal verification ROM, death/drop verification ROM, or powerup verification ROM:

SMOKE_BUILD_TARGET=combat-test-rom \
SMOKE_RUN_TARGET=combat-test-gngeo \
SMOKE_OUTPUT=.tools/screens/latest/combat-test.png \
make smoke-screenshot

For the generated E1M1 route invariant, run make route-check. It verifies that the converted player start can reach the converted exit from the current build/doom_map_generated.h and reports whether the route depends on generated door cells.

For native Doom visual comparisons, run tools/capture_compare.sh. By default it captures the compiled map start view and writes native, Neo Geo, and side-by-side PNGs under .tools/screens/. The helper uses the same capture lock as smoke screenshots, so native and GnGeo windows cannot be cross-captured by a parallel comparison run. The smoke and comparison helpers place emulator windows on workspace 4 by default and send targeted key events instead of activating the window, so captures do not take focus from the desktop. Direct i3/sway tiling of ngdevkit-gngeo is disabled by default because forcing floating disable resizes the SDL window and reliably crashes GnGeo on this host; set SMOKE_TILE_WINDOWS=1 or COMPARISON_TILE_WINDOWS=1 only when intentionally rechecking that behavior or when running under a wrapper that tolerates resize. Set COMPARE_WAYPOINT to capture a named view:

DOOM_MAP=E1M1 tools/capture_compare.sh
COMPARE_WAYPOINT=e1m1-scout tools/capture_compare.sh
COMPARE_WAYPOINT=e1m2-start tools/capture_compare.sh

Supported waypoint names are start, e1m1-start, e1m2-start, e1m1-encounter, e1m1-scout, and e1m2-keydoor. Start waypoints use the same map spawn on both sides. By default, non-start route waypoints drive both native Doom and the Neo Geo ROM with the same timed input script from that map spawn, with native Doom holding its speed modifier during forward movement. This makes the side-by-side more useful for judging equivalent route views; set COMPARE_NATIVE_MOVE_MODIFIER= to disable the native speed modifier when checking walk-speed captures. When the engines drift to different places, tune the route per side with COMPARE_NATIVE_ROUTE_* and COMPARE_NEO_ROUTE_* variables such as COMPARE_NEO_ROUTE_E1M1_SCOUT_FORWARD1=0.9. Set COMPARE_ROUTE_MODE=focused to use the older focused Neo Geo verification ROMs for e1m1-encounter, e1m1-scout, and e1m2-keydoor. Focused captures wait briefly before grabbing the window and reject all-black frames, so startup/transient captures do not silently become the side-by-side evidence.

For a broader Episode 1 conversion baseline, run make episode-route-report. It converts E1M1 through E1M9 into build/episode-route/ and reports which maps currently have a generated start-to-exit route at the configured grid size. Run make episode-route-check for the strict gate: E1M1-E1M7 and E1M9 must route, and E1M8 must expose the supported boss-death completion path.

To build a standalone ROM for a specific Episode 1 map, run:

make episode-map-rom EPISODE_MAP=E1M3

The output lands in build/episode-roms/E1M3-rom/. Use make episode-map-gngeo EPISODE_MAP=E1M3 to launch that standalone map in GnGeo. make episode-roms loops through E1M1-E1M9 and produces one standalone ROM output per map; this is still map-by-map packaging, not a single multi-map episode cart.

For the broad playable-feature regression pass, run tools/smoke_gameplay.sh. It starts with make route-check, then chains the verified enemy visibility, key-door, weapon shortcut, death/drop, and powerup smoke helpers into one command and refreshes the same screenshots under .tools/screens/latest/.

For movement feel and frame-pacing registers, run tools/stress_movement.sh. It starts the normal ROM, holds forward, turn, and strafe inputs, and captures the resulting poses. Run tools/bench_movement.sh for the same path with GnGeo's --showfps overlay enabled and longer held inputs; outputs land under .tools/screens/latest/movement-bench/ and the emulator log is written to .tools/logs/movement-bench-gngeo.log. The bench finishes with tools/check_movement_screens.py, which rejects missing, blank, static, or obviously wrong movement captures before treating the run as useful evidence. It also runs tools/check_plane_motion_screens.py, which compares the start and held-forward captures in floor side bands and the ceiling band, then writes a marked side-by-side image to .tools/screens/latest/plane-motion-compare/. The bench rejects GnGeo logs containing Invalid write, so palette or VRAM range errors do not pass as normal movement evidence.

For the RIPDOOM/chunked renderer path, make ripdoom-render-check now also reports interactive_pass=1 when a generated closed door/lift cell blocks and the same cell becomes passable after opening. This keeps the host render probe aligned with runtime chunk collision for streamed interactive geometry. The same report includes second_hits=A/B for the start and moved views when the one-sprite far-wall fallback replaces a short lower/upper span with the next wall hit behind it.

The default color threshold targets the bright E1M1 start-room path; darker maps such as E1M2 can pass a lower --min-play-colored value to tools/check_movement_screens.py through MOVEMENT_CHECK_ARGS while still requiring FPS, frame-stat, and pose-delta evidence. By default the bench uses an isolated DOOM_FRAME_STATS=1 build under build/frame-stats/. Its green marker plus NN playfield register reports how many frames in the latest 64-frame window reached wait_vblank_status() after vblank had already started; 00 means no measured late frames in that window. Set SMOKE_MAKE_ARGS to pass isolated build variables through the same path, for example DOOM_DETAIL=speed BUILDDIR=build/speed-movement ROM=build/speed-movement-rom GFX_ROM_DIR=build/speed-movement-assets. When a movement run needs renderer-budget experiments, the same variable can carry DOOM_WALL_UPLOAD_COLUMNS, DOOM_WALL_UPLOAD_OVERRUN_COLUMNS, DOOM_BG_SCROLL_COLUMNS, DOOM_BG_SCROLL_OVERRUN_COLUMNS, DOOM_PLANE_FLOOR_FORWARD_PHASES, DOOM_PLANE_CEILING_FORWARD_PHASES, or DOOM_PLANE_PHASE_SHIFT without editing source. For CPU-side wall intersection tuning, pass DOOM_ADAPTIVE_LINE_REFINEMENT, DOOM_MOVING_LINE_REFINEMENT_CELLS, or DOOM_OVERRUN_LINE_REFINEMENT_CELLS through the same path. When a custom ROM=... directory is used, the smoke helper copies the local neogeo.zip BIOS package there before launching GnGeo.

For chunked RIPDOOM movement debugging, run:

SMOKE_XVFB=1 tools/bench_chunk_debug_movement.sh

This runs the deterministic chunk-movement-check host probe first, then boots chunk-movement-test-rom and stores a debug-register capture under .tools/screens/latest/chunk-debug-movement/. In debug builds the HUD counters mirror global chunk position and active chunk state; in the scripted movement ROM the armor counter mirrors the script tick. This keeps movement proof out of fragile Xvfb timing while still leaving a visible ROM capture for stale-ROM and HUD/register checks.

chunk-movement-check also validates reachable generated lift triggers: a trigger must be reachable while its lift is closed, and after opening that lift the target lift cells must become reachable. This catches coarse-grid conversion breaks where a Doom lift exists in metadata but cannot function in the chunked runtime.

For a combat interaction regression pass, run tools/smoke_combat_interaction.sh. It captures the initial visible imp, the shotgun fire frame, and the resulting death/corpse feedback frame. Override COMBAT_DEATH_WAIT_SECS if an emulator or host is running slowly and the final capture still lands during the shotgun pump animation. The helper also steps the player slightly backward before the final capture so the weapon sprite does not cover the corpse/drop feedback; override COMBAT_DEATH_REVEAL_STEP_SECS if that framing needs tuning.

For a sample-map monster visibility pass, run tools/smoke_e1m1_encounter.sh. It builds make encounter-test-rom, launches make encounter-test-gngeo, and captures both the initial focused encounter and one pistol-fire frame against an authored sample-map monster.

For a first-contact route visibility pass, run tools/smoke_e1m1_scout.sh. It builds make scout-test-rom, launches make scout-test-gngeo, and captures a sample-map route waypoint looking toward visible pickups and monsters plus a pistol-fire frame.

For a focused first-level completion check, run tools/smoke_e1m1_exit.sh. It builds make exit-test-rom, stages the player near the authored sample exit, walks into that trigger, captures the completed frame, and checks the EXIT plus kill/item/secret percentage overlay. Build with DOOM_SIMPLE_MAP=0 when checking the older converted E1M1 exit staging.

For the E1M8 special boss-death exit, run tools/smoke_e1m8_boss_exit.sh. It builds make e1m8-boss-test-rom from the real E1M8 generated map, stages the two original Baron things in front of the player, fires once, and checks that the normal completion overlay appears after both bosses die.

For an invisible-attacker regression pass, run tools/smoke_hidden_attack.sh. It builds make hidden-attack-test-rom, launches make hidden-attack-test-gngeo, and captures both the initial frame and a delayed frame. The delayed frame should still show the same health value, proving hidden or offscreen monsters cannot damage the player without a readable world-sprite slot.

For a combined enemy visibility pass, run tools/smoke_enemy_visibility.sh. It chains the combat interaction, real E1M1 encounter, hidden-attack, and monster-gallery regressions into one command, refreshing the screenshots that prove visible targeting, real converted monster projection, no hidden damage, and multi-sprite enemy coverage. The helper finishes by running tools/check_enemy_visibility_screens.py, a lightweight image-stat sanity check that rejects missing, blank, obviously wrong captures, and frames with weak monster-colored pixel evidence in the expected combat, encounter, scout, and gallery regions.

For a corpse/drop rendering pass, run tools/smoke_death_drop.sh. It builds make death-test-rom, launches make death-test-gngeo, and captures the staged shareware corpse sprites plus the dropped shotgun pickup above the status bar. The helper finishes with tools/check_death_drop_screens.py, which rejects frames without the expected corpse-colored pixels, dropped-weapon evidence, and status-bar evidence.

SMOKE_BUILD_TARGET=melee-test-rom \
SMOKE_RUN_TARGET=melee-test-gngeo \
SMOKE_OUTPUT=.tools/screens/latest/melee-test.png \
make smoke-screenshot
SMOKE_BUILD_TARGET=key-door-test-rom \
SMOKE_RUN_TARGET=key-door-test-gngeo \
SMOKE_OUTPUT=.tools/screens/latest/key-door-test.png \
make smoke-screenshot

For the full key-door interaction path, use tools/smoke_key_door.sh. It captures the initial sample-map key-door scene, the missing-key message, the post-pickup HUD state, the opened-door frame, and one extra frame after walking through the opened doorway. The helper finishes with tools/check_key_door_screens.py, which rejects missing captures, missing KEY/keycard message evidence, a door that still looks closed in the opened frame, or a through-door frame that does not show the player past the door.

SMOKE_BUILD_TARGET=arsenal-test-rom \
SMOKE_RUN_TARGET=arsenal-test-gngeo \
SMOKE_OUTPUT=.tools/screens/latest/arsenal-test.png \
make smoke-screenshot
SMOKE_BUILD_TARGET=monster-gallery-rom \
SMOKE_RUN_TARGET=monster-gallery-gngeo \
SMOKE_OUTPUT=.tools/screens/latest/monster-gallery.png \
make smoke-screenshot
tools/smoke_powerup.sh

The powerup helper builds and launches the isolated powerup ROM, captures the visible pickup/imp setup, then runs tools/check_powerup_screens.py to reject frames without powerup-colored pickups, the visible imp, and status-bar evidence.

tools/smoke_capture.sh and tools/capture_compare.sh serialize emulator launches with an owner-tracked lock under .tools/locks/ so parallel screenshot refreshes do not capture the wrong native or GnGeo window. If a previous aborted run left a directory-only or dead-PID lock, the helpers clear it automatically before starting the next capture. Set SMOKE_WORKSPACE= or COMPARISON_WORKSPACE= to override the default workspace 4 placement.

Fast weapon shortcut input can be smoke-checked with:

tools/smoke_weapon_shortcuts.sh

The helper builds and launches the arsenal verification ROM, captures a baseline frame, sends the configured GnGeo C+Down shortcut, then holds C before pressing Right to prove both shortcut key orders. It captures the frames under .tools/screens/latest/, runs tools/check_weapon_shortcut_screens.py to verify that the visible weapon silhouette changes between plasma, rocket, and chaingun states, and leaves GnGeo open on SMOKE_WORKSPACE for visual inspection.

Map conversion honors Doom THING skill flags through DOOM_SKILL_MASK. The default is 4, which keeps the hard/Ultra-Violence population used by the current E1M1 prototype. Use DOOM_SKILL_MASK=1 for easy placement or DOOM_SKILL_MASK=2 for medium placement:

make DOOM_MAP=E1M1 DOOM_SKILL_MASK=2

Sprite spec checks can be run without a full ROM build when adding optional registered, commercial, or Doom II monster art:

python3 tools/check_sprite_specs.py --iwad path/to/doom.wad \
  --spec '3005:HEADA1,3006:SKULA1,9028:HEADL0'

The helper uses the same WAD patch loader as tools/gen_gfx.py, but only bakes one scale level for the requested frames.

Local Windows Build

ngdevkit's current Windows path is native MSYS2 UCRT64. Run the helper from an MSYS2 UCRT64 shell and let the standalone installer command configure the upstream package repository and install the needed packages:

doomgeo-build.exe install --method msys2
doomgeo-build.exe doctor --tools-prefix /ucrt64
doomgeo-build.exe build --tools-prefix /ucrt64

If the helper is run from normal Windows with WSL available, ROM builds delegate to WSL as a fallback. Native Windows builds are intentionally standardized on MSYS2 UCRT64 because that is the upstream ngdevkit Windows package target.

The Windows standalone binary still supports:

doomgeo-build.exe doctor
doomgeo-build.exe uninstall --dry-run
doomgeo-plan.exe list

GitHub Actions validates the native Windows/MSYS2 path by installing the same UCRT64 packages, building the ROM, and uploading a Windows-built ROM artifact.

Uninstall

The build helper removes only repo-local generated tool/cache state:

python3 tools/doomgeo_build.py uninstall
python3 tools/doomgeo_build.py uninstall --all

uninstall removes .tools/ngdevkit-local. --all also removes cached assets and package downloads under .tools. It does not remove committed source files or system packages installed through apt/MSYS2.

GitHub Actions

.github/workflows/build.yml has these responsibilities:

  • Build and upload the Neo Geo ROM on ubuntu-24.04.
  • Build and upload a separate Freedoom-based web-safe ROM for GitHub Pages.
  • Build and upload a separate 68000 ASM Neo Geo ROM on ubuntu-24.04.
  • Build and upload the Neo Geo ROM on Windows through MSYS2 UCRT64.
  • Package doomgeo-build and doomgeo-plan as standalone binaries on Linux and Windows.
  • Publish a GitHub Pages playable build on branch pushes.

The Ubuntu ROM job uses the upstream ngdevkit PPA and passes TOOLS_PREFIX=/usr so the CI machine does not need repo-local .tools state.

The Pages job consumes the Freedoom web ROM artifact and writes:

dist/pages/index.html
dist/pages/asm.html
dist/pages/rom/web-<hash>/magdrop2.zip
dist/pages/rom/web-<hash>/doomgeo-aes.zip
dist/pages/rom/web-<hash>/neogeo.zip
dist/pages/rom/asm/web-<hash>/magdrop2.zip
dist/pages/rom/asm/web-<hash>/doomgeo-aes-asm.zip

The web players use the hosted EmulatorJS loader, set EJS_core = "fbneo", and point EJS_gameUrl/EJS_biosUrl at those generated files. The <hash> path component is generated from the source artifacts so browser/CDN caches cannot reuse a stale ROM zip with older CRCs.