Add Apple Foundation Models provider support #1265
vincent-herlemont
started this conversation in
Ideas
Replies: 1 comment
|
Looks good. We should add this as a companion crate though, I have a feeling that this will require adding a lot of new crates. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Apple recently released the Foundation Models framework (macOS/iOS 26+), enabling on-device LLM inference with no network dependency. This would be a valuable addition to Rig's provider ecosystem, especially for privacy-focused and offline applications on Apple platforms.
Unlike existing HTTP-based providers, Foundation Models requires native platform integration. The
libailibrary provides C bindings that could bridge the Swift framework to Rust via FFI.Key points:
#[cfg(target_os = "macos")]feature gatingOpen questions:
SystemLanguageModel.isAvailable)?References:
Happy to contribute an initial implementation if there's interest.
All reactions