Skip to content

add big model fallback#38303

Open
elkoled wants to merge 4 commits into
masterfrom
fallback
Open

add big model fallback#38303
elkoled wants to merge 4 commits into
masterfrom
fallback

Conversation

@elkoled

@elkoled elkoled commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

big model fallback logic

Startup: load small model and big model in separate processes, ready to engage small model instantly
Big model loaded: next user disengage switches to big model to prevent abrupt shifts in model outputs
Big model failed: soft disengage, switches to small model, next user engage drives on small model

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Process replay diff report

Replays driving segments through this PR and compares the behavior to master.
Please review any changes carefully to ensure they are expected.

✅ 0 changed, 66 passed, 0 errors

@commaci-public

commaci-public commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

@elkoled elkoled marked this pull request as ready for review July 10, 2026 21:55
@elkoled elkoled requested a review from adeebshihadeh July 10, 2026 22:13

def main() -> None:
try:
subprocess.run(["udevadm", "settle"], check=True)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this mess with other things like the modem that's also on USB? what does this do?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also can this hang?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason not to do this in modeld? i think it's a bit cleaner

Comment on lines +152 to +153
if model_name == "smallModelV2":
config_realtime_process([0, 1, 2, 3], 54)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

both don't fit on a single core? this changes the runtime of the small model....


PythonProcess("modeld", "openpilot.selfdrive.modeld.modeld", only_onroad),
PythonProcess("bigmodeld", "openpilot.selfdrive.modeld.bigmodeld", bigmodeld),
PythonProcess("modelrouterd", "openpilot.selfdrive.modeld.modelrouterd", usbgpu),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let’s always route? keeps both codepaths closer, avoids extra logic in modeld

@@ -129,5 +129,7 @@ inline static std::unordered_map<std::string, ParamKeyAttributes> keys = {
{"UptimeOnroad", {PERSISTENT, FLOAT, "0.0"}},
{"UsbGpuPresent", {CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION, BOOL}},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we use the chestnutPresent message here instead?



def main(demo=False):
def main(demo=False, model_name="modelV2"):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

splitting on model_name to publish without routing when no usbgpu is confusing to read
it'd be nicer to always route and to have the publishing logic in a single place, it doesn't depend on the usbgpu being present

params.put_bool("UsbGpuPresent", _present)
params.put_bool("UsbGpuCompiled", _compiled)
if model_name == "modelV2":
_present = usbgpu_present()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slightly confusing that small modeld does the detection for big modeld
could we move the detection logic elsewhere and always pass big/small to modeld?

@Armandpl

Copy link
Copy Markdown
Contributor

having to disengage then reengage to switch from small to big is going to be super confusing.
how slow is loading the 2GB model? can we just wait for it to load? do we think we can improve loading time soon? on a workstation we get ~700MB/s into the chestnut, that’d be 3s.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants