Note: While working on #1235 I was reminded of several longstanding issues. I asked Claude to turn them into reports.
The up-to-date check reads the mtimes of the top-level Stan program and the user
header. Files reached by #include are not checked at any depth — including
one level down, which is worth stating because "nested includes" suggests the
direct case works. It does not:
# include dir holds params.stan declaring `alpha`
mod <- cmdstan_model(stan_file, include_paths = inc, force_recompile = TRUE)
# edit inc/params.stan to declare `beta` instead
mod$compile()
#> no recompilation
mod$variables()$parameters
#> beta <- but the executable still has alpha
The object then validates data and initial values against a program it is not
running, which is the failure class of #1228. force_recompile = TRUE is the
escape hatch and is now documented as such, but the check should eventually
notice on its own.
Preferred design. During a successful compile, when stanc is running anyway
and the cost is invisible, record which files the program actually pulled in.
The later up-to-date check just stats those recorded paths. Editing a recorded
include is caught directly; adding a directive to the top-level file or to an
already-recorded include changes a file that is already checked. Fast-path cost
stays in the same order as today.
Known gap in that design. Shadowing is caught by neither this nor the
include-path check added in #1235: with include_paths = c(A, B) and
foo.stan resolving from B, creating A/foo.stan changes which file the
program uses while the path vector is unchanged and no recorded file's mtime
moves. Deletion is the mirror image, and is recoverable by treating a recorded
file that no longer exists as dirty. Recording and statting the include
directories as well would close both, since create/delete bumps a directory's
mtime — but compile artifacts (the executable, the generated .hpp) are written
into dirname(stan_file), which is the include path CmdStanR infers when none is
given, so that trade-off needs thought.
Alternatives considered. Hashing the generated C++ is complete — it moves
with source edits, include edits, include paths, stanc version and options — but
makes every no-op $compile() spawn stanc, which is the one path that is
currently just a few stat() calls. Statting whole include trees runs into the
same artifact-directory problem above.
Getting the scan wrong degrades the heuristic rather than corrupting a build, so
it should fail toward rebuilding whenever a directive cannot be resolved.
Note: While working on #1235 I was reminded of several longstanding issues. I asked Claude to turn them into reports.
The up-to-date check reads the mtimes of the top-level Stan program and the user
header. Files reached by
#includeare not checked at any depth — includingone level down, which is worth stating because "nested includes" suggests the
direct case works. It does not:
The object then validates data and initial values against a program it is not
running, which is the failure class of #1228.
force_recompile = TRUEis theescape hatch and is now documented as such, but the check should eventually
notice on its own.
Preferred design. During a successful compile, when stanc is running anyway
and the cost is invisible, record which files the program actually pulled in.
The later up-to-date check just stats those recorded paths. Editing a recorded
include is caught directly; adding a directive to the top-level file or to an
already-recorded include changes a file that is already checked. Fast-path cost
stays in the same order as today.
Known gap in that design. Shadowing is caught by neither this nor the
include-path check added in #1235: with
include_paths = c(A, B)andfoo.stanresolving fromB, creatingA/foo.stanchanges which file theprogram uses while the path vector is unchanged and no recorded file's mtime
moves. Deletion is the mirror image, and is recoverable by treating a recorded
file that no longer exists as dirty. Recording and statting the include
directories as well would close both, since create/delete bumps a directory's
mtime — but compile artifacts (the executable, the generated
.hpp) are writteninto
dirname(stan_file), which is the include path CmdStanR infers when none isgiven, so that trade-off needs thought.
Alternatives considered. Hashing the generated C++ is complete — it moves
with source edits, include edits, include paths, stanc version and options — but
makes every no-op
$compile()spawn stanc, which is the one path that iscurrently just a few
stat()calls. Statting whole include trees runs into thesame artifact-directory problem above.
Getting the scan wrong degrades the heuristic rather than corrupting a build, so
it should fail toward rebuilding whenever a directive cannot be resolved.