Skip to content

Enable re-init after OgaShutdown#2292

Open
skottmckay wants to merge 4 commits into
mainfrom
skottmckay/EnableReinitAfterOgaShutdown
Open

Enable re-init after OgaShutdown#2292
skottmckay wants to merge 4 commits into
mainfrom
skottmckay/EnableReinitAfterOgaShutdown

Conversation

@skottmckay

Copy link
Copy Markdown
Contributor

Enable re-init after OgaShutdown. State returns to 'just loaded' post-OgaShutdown.
Allows user to control lifetime of the OrtEnv instance that is pinned in OgaGlobals.

See the plan document for details.

Adds some additional library load validation/diagnostic info (copied from ORT) so we stay in a clean state if an EP can't be loaded.

…-OgaShutdown.

Allows user to control lifetime of the OrtEnv instance that is pinned in OgaGlobals.

See the plan document for details.

Adds some additional library load validation/diagnostic info (copied from ORT) so we stay in a clean state if an EP can't be loaded.
@skottmckay skottmckay requested a review from a team as a code owner July 13, 2026 00:36
Copilot AI review requested due to automatic review settings July 13, 2026 00:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Enables using ONNX Runtime GenAI again after calling OgaShutdown() by refactoring GenAI’s process-global state to be destroyable/re-creatable (OrtGlobals), rebuilding device interfaces per env cycle, and adding a dedicated re-init test executable plus improved EP library load diagnostics.

Changes:

  • Refactors OrtGlobals creation/teardown to support shutdown → re-init cycles, including per-cycle device-interface ownership/caching and CUDA add-on unload.
  • Adds reinit_tests standalone binary and shared EP discovery/registration helpers to validate repeated teardown/reinit and device-interface recreation.
  • Adds pre-flight EP library load validation and improves DLL dependency diagnostics to avoid half-registered EP states.

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
test/reinit_tests.cpp New standalone tests exercising repeated OgaShutdown() + re-init and device-interface recreation.
test/ep_registration.h Test-only EP discovery/registration helper for plugin EP DLLs (WinML + --ep_dir).
test/CMakeLists.txt Builds reinit_tests separately from unit_tests and registers a new CTest entry.
src/webgpu/interface.h Switches WebGPU interface accessor to a factory returning std::unique_ptr.
src/webgpu/interface.cpp Implements WebGPU factory to create a fresh interface per env cycle.
src/search.cpp Uses GetDeviceInterface(DeviceType::CPU) instead of the old CPU singleton accessor.
src/ryzenai/session_options.cpp Routes RyzenAI setup through the device interface instance returned by GetDeviceInterface.
src/ryzenai/interface.h Replaces RyzenAI singleton getter with a per-env factory API.
src/ryzenai/interface.cpp Makes RyzenAI interface env-scoped and tolerant of “already registered” registration status.
src/qnn/interface.h Replaces QNN singleton getter with a factory returning std::unique_ptr.
src/qnn/interface.cpp Implements QNN factory and adds null checks for optional provider options shaping.
src/ort_genai_c.h Updates OgaShutdown contract docs for lifetime rules and re-initialization semantics.
src/ort_genai_c.cpp Adds EP library pre-flight load validation before registering with ORT.
src/openvino/interface.h Replaces OpenVINO singleton getter with a factory returning std::unique_ptr.
src/openvino/interface.cpp Implements OpenVINO factory; routes allocations/search creation through CPU device interface.
src/models/model.cpp Minor formatting change near provider option shaping.
src/generators.h Extends OrtGlobals with device-interface ownership/cache and CUDA library handle lifetime.
src/generators.cpp Makes globals lazily re-creatable after shutdown; moves teardown to ~OrtGlobals; adds device-interface cache.
src/dml/interface.cpp Uses CPU device interface instead of CPU singleton accessor for search creation.
src/dll_load_error.h Adds CanLoadLibrary declaration (used for EP pre-flight) and improves documentation.
src/dll_load_error.cpp Fixes Windows import-directory parsing (MappedAsImage=TRUE) and implements CanLoadLibrary on all platforms.
src/cpu/interface.h Replaces CPU singleton getter with a factory returning std::unique_ptr.
src/cpu/interface.cpp Implements CPU factory; makes InitOrt idempotent for re-init cycles.
docs/EnableReinitAfterOgaShutdown.md New design/teardown-order plan document for shutdown + re-init.
CMakeLists.txt Introduces onnxruntime-genai-obj OBJECT library split; adjusts link/usage requirements; adds MSVC warning suppression for newer toolsets.

Comment thread src/dll_load_error.h
Comment thread src/generators.cpp
Comment thread src/ryzenai/interface.cpp
Comment thread src/cpu/interface.h
Comment thread src/webgpu/interface.h
Comment thread src/openvino/interface.h
Comment thread src/ryzenai/interface.h
Comment thread src/ryzenai/interface.cpp

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akholodnamdcom would you be able to validate the RyzenAI changes?

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated 5 comments.

Comment thread src/qnn/interface.h
Comment thread src/generators.cpp
Comment thread test/ep_registration.h
Comment thread test/ep_registration.h
Comment thread test/reinit_tests.cpp Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated no new comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants