Enable re-init after OgaShutdown#2292
Open
skottmckay wants to merge 4 commits into
Open
Conversation
…-OgaShutdown. Allows user to control lifetime of the OrtEnv instance that is pinned in OgaGlobals. See the plan document for details. Adds some additional library load validation/diagnostic info (copied from ORT) so we stay in a clean state if an EP can't be loaded.
…itAfterOgaShutdown
Contributor
There was a problem hiding this comment.
Pull request overview
Enables using ONNX Runtime GenAI again after calling OgaShutdown() by refactoring GenAI’s process-global state to be destroyable/re-creatable (OrtGlobals), rebuilding device interfaces per env cycle, and adding a dedicated re-init test executable plus improved EP library load diagnostics.
Changes:
- Refactors
OrtGlobalscreation/teardown to support shutdown → re-init cycles, including per-cycle device-interface ownership/caching and CUDA add-on unload. - Adds
reinit_testsstandalone binary and shared EP discovery/registration helpers to validate repeated teardown/reinit and device-interface recreation. - Adds pre-flight EP library load validation and improves DLL dependency diagnostics to avoid half-registered EP states.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| test/reinit_tests.cpp | New standalone tests exercising repeated OgaShutdown() + re-init and device-interface recreation. |
| test/ep_registration.h | Test-only EP discovery/registration helper for plugin EP DLLs (WinML + --ep_dir). |
| test/CMakeLists.txt | Builds reinit_tests separately from unit_tests and registers a new CTest entry. |
| src/webgpu/interface.h | Switches WebGPU interface accessor to a factory returning std::unique_ptr. |
| src/webgpu/interface.cpp | Implements WebGPU factory to create a fresh interface per env cycle. |
| src/search.cpp | Uses GetDeviceInterface(DeviceType::CPU) instead of the old CPU singleton accessor. |
| src/ryzenai/session_options.cpp | Routes RyzenAI setup through the device interface instance returned by GetDeviceInterface. |
| src/ryzenai/interface.h | Replaces RyzenAI singleton getter with a per-env factory API. |
| src/ryzenai/interface.cpp | Makes RyzenAI interface env-scoped and tolerant of “already registered” registration status. |
| src/qnn/interface.h | Replaces QNN singleton getter with a factory returning std::unique_ptr. |
| src/qnn/interface.cpp | Implements QNN factory and adds null checks for optional provider options shaping. |
| src/ort_genai_c.h | Updates OgaShutdown contract docs for lifetime rules and re-initialization semantics. |
| src/ort_genai_c.cpp | Adds EP library pre-flight load validation before registering with ORT. |
| src/openvino/interface.h | Replaces OpenVINO singleton getter with a factory returning std::unique_ptr. |
| src/openvino/interface.cpp | Implements OpenVINO factory; routes allocations/search creation through CPU device interface. |
| src/models/model.cpp | Minor formatting change near provider option shaping. |
| src/generators.h | Extends OrtGlobals with device-interface ownership/cache and CUDA library handle lifetime. |
| src/generators.cpp | Makes globals lazily re-creatable after shutdown; moves teardown to ~OrtGlobals; adds device-interface cache. |
| src/dml/interface.cpp | Uses CPU device interface instead of CPU singleton accessor for search creation. |
| src/dll_load_error.h | Adds CanLoadLibrary declaration (used for EP pre-flight) and improves documentation. |
| src/dll_load_error.cpp | Fixes Windows import-directory parsing (MappedAsImage=TRUE) and implements CanLoadLibrary on all platforms. |
| src/cpu/interface.h | Replaces CPU singleton getter with a factory returning std::unique_ptr. |
| src/cpu/interface.cpp | Implements CPU factory; makes InitOrt idempotent for re-init cycles. |
| docs/EnableReinitAfterOgaShutdown.md | New design/teardown-order plan document for shutdown + re-init. |
| CMakeLists.txt | Introduces onnxruntime-genai-obj OBJECT library split; adjusts link/usage requirements; adds MSVC warning suppression for newer toolsets. |
skottmckay
commented
Jul 13, 2026
Contributor
Author
There was a problem hiding this comment.
@akholodnamdcom would you be able to validate the RyzenAI changes?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Enable re-init after OgaShutdown. State returns to 'just loaded' post-OgaShutdown.
Allows user to control lifetime of the OrtEnv instance that is pinned in OgaGlobals.
See the plan document for details.
Adds some additional library load validation/diagnostic info (copied from ORT) so we stay in a clean state if an EP can't be loaded.