Skip to content

Commit ed54750

Browse files
helenkwokclaude
andcommitted
feat(0.2.0-rc.0): iOS LiteRT-LM hardening — file:// URLs, cancel latency, xcframework publish
- ios/ExpoLitertLmModule.swift: accept POSIX paths and file:// URLs in loadModel; retain security-scoped resource access for the lifetime of the loaded model - example/App.tsx: pass raw DocumentPicker URI through (no caller-side strip); record cancelLatencyMs even on early cancel before first token - package.json: 0.1.0 -> 0.2.0-rc.0; npm files now includes ios/BinaryPods/Frameworks/CLiteRTLM.xcframework + GemmaModelConstraintProvider.xcframework + rewrap-manifest.json - ios/BinaryPods/Frameworks/.npmignore: keep xcframeworks in tarball even though they are gitignored (consumers must not rebuild from source) - README/CHANGELOG/example docs: SE 3rd gen + Gemma 3 1B INT4 iOS framing, no Gemma 4 E2B iOS claim; npm badges + View-on-npm link added Verification (all green): - npm run build, npx tsc -p example/tsconfig.json --noEmit - make verify (Layer A/B/C CocoaPods checks) - npm publish --tag next --dry-run: 48 files, 14.5 MB packed, xcframeworks included - xcodebuild CODE_SIGNING_ALLOWED=NO -> BUILD SUCCEEDED Android guardrail: zero diff under android/, src/, lib/. Pre-release under --tag next; latest stays at 0.1.2 until competition fully ends. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent e63cc8c commit ed54750

8 files changed

Lines changed: 125 additions & 31 deletions

File tree

CHANGELOG.md

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,24 @@
11
# Changelog
22

3-
## 0.2.0-dev.1Phase 14 (in progress, 2026-05-15)
3+
## 0.2.0 — 2026-05-16
44

5+
- **Added:** iOS LiteRT-LM text generation via vendored LiteRTLM-Swift / `CLiteRTLM.xcframework`. Validated on iPhone SE 3rd gen with `gemma3-1b-it-int4.litertlm` (Gemma 3 1B INT4). Gemma 4 E2B remains Android-validated only; it was not tested on SE 3rd gen because of memory limits.
56
- **Added:** `sampleMemoryAsync()` — returns the current process `phys_footprint` in MB for Stage B integration measurement (Phase 14 D-07). iOS only; surfaced via colocated `ios/MemoryProbe.swift` (task_info / TASK_VM_INFO). Stage B example app under `example/` exercises the full Expo + RN + native bridge end-to-end against the rewrapped LiteRTLM-Swift xcframework on SE 3rd gen.
67
- **Changed:** `scripts/sync-litertlm-swift.sh` rewritten — now fetches rewrapped LiteRTLM-Swift from `helenkwok/LiteRTLM-Swift`'s GitHub Release by tag, verifies SHA-256 against `rewrap-manifest.json`, places artifacts under `ios/BinaryPods/Frameworks/`. Manual sync per Phase 14 D-34. Run `make sync TAG=v<upstream>+rewrap.<n>`.
78
- **Added:** `ios/BinaryPods/Frameworks/rewrap-manifest.json` — trust anchor (source-controlled JSON, populated by sync script). xcframework binaries remain gitignored.
89
- **Added:** `ExpoLitertLm.podspec` now reads `vendored_frameworks` and `s.version` from `ios/BinaryPods/Frameworks/rewrap-manifest.json` via Ruby `JSON.parse` at install time. Phase 14 D-31 single source of truth.
910
- **Added:** Layer A/B/C verification gates (`scripts/verify-consumption.sh`, `make verify`). Layer A = pod install from fresh consumer; Layer B = grep gate for raw xcframework refs; Layer C = manifest-driven podspec consistency.
1011
- **Added:** `.github/workflows/verify-consumption.yml` — CI mirrors fork's three-layer gate on macos-latest.
12+
- **Added:** Expo config plugin (`app.plugin.js`) that injects the split binary pods required for iOS consumers. Verified under `useFrameworks: "static"`; `useFrameworks: "dynamic"` was rejected by CocoaPods because Expo SDK 55 includes static transitive binaries.
13+
- **Fixed:** iOS `loadModelAsync` accepts either a POSIX path or a `file://` URL, including values returned by `expo-document-picker`.
14+
- **Fixed:** Stage B example now records `cancelLatencyMs` when cancellation ends before a final token event arrives.
1115

1216
- **BREAKING:** iOS path migrated from `MediaPipeTasksGenAI` default-dep to vendored `LiteRTLM-Swift` (`CLiteRTLM.xcframework` + `GemmaModelConstraintProvider.xcframework`). Consumers must run `pod install` after upgrade.
1317
- **BREAKING:** `ExpoLitertLm.podspec` no longer carries `static_framework = true` — see CocoaPods issue [#11948](https://github.com/CocoaPods/CocoaPods/issues/11948) + offlineaid Phase 14 CONTEXT D-21. Static linkage of a dylib-containing xcframework crashes the host app at launch.
1418
- **BREAKING:** MediaPipe `.task` support moves to opt-in `MediaPipeFallback` subspec; default install does not include `MediaPipeTasksGenAI` on iOS. Consumers who need it: `pod 'ExpoLitertLm', :subspecs => ['Core', 'MediaPipeFallback']` in their Podfile.
15-
- Added: `scripts/sync-litertlm-swift.sh` for manual fork sync. `helenkwok/LiteRTLM-Swift` is the upstream fork pinned by SHA; vendoring happens at sync-time, not at `pod install`-time.
16-
- Added: `scripts/rewrap-xcframework.sh` in the fork resolves upstream issue [#6](https://github.com/mylovelycodes/LiteRTLM-Swift/issues/6) offline (Info.plist `CFBundleShortVersionString`, loose-dylib promotion, dSYM emission). TestFlight validation deferred to v1.2 per offlineaid CONTEXT D-25.
19+
- **Added:** `scripts/sync-litertlm-swift.sh` for manual fork sync. `helenkwok/LiteRTLM-Swift` is the upstream fork pinned by SHA; vendoring happens at sync-time, not at `pod install`-time.
20+
- **Added:** `scripts/rewrap-xcframework.sh` in the fork resolves upstream issue [#6](https://github.com/mylovelycodes/LiteRTLM-Swift/issues/6) offline (Info.plist `CFBundleShortVersionString`, loose-dylib promotion, dSYM emission). TestFlight validation deferred to v1.2 per offlineaid CONTEXT D-25.
21+
- **Synced:** Rewrapped LiteRTLM-Swift `v0.10.2+rewrap.5` (2 xcframeworks, first sha256 `ba2d0c9a6b2a...`).
1722

1823
## 0.1.2 — 2026-05-12
1924

@@ -26,5 +31,3 @@
2631
## 0.1.0
2732

2833
- feat: initial expo-litert-lm — Expo Modules bindings for LiteRT-LM.
29-
2026-05-16 — synced rewrapped LiteRTLM-Swift v0.10.2+rewrap.1 (2 xcframeworks, first sha256: 10565a593a70...)
30-
2026-05-16 — synced rewrapped LiteRTLM-Swift v0.10.2+rewrap.5 (2 xcframeworks, first sha256: ba2d0c9a6b2a...)

README.md

Lines changed: 39 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,22 @@
11
# expo-litert-lm
22

3-
> Expo Modules bindings for [Google AI Edge LiteRT-LM](https://github.com/google-ai-edge/litert-lm) — on-device Gemma 4 / Gemma 3n inference for Expo apps.
3+
[![npm version](https://img.shields.io/npm/v/expo-litert-lm.svg)](https://www.npmjs.com/package/expo-litert-lm)
4+
[![npm downloads](https://img.shields.io/npm/dm/expo-litert-lm.svg)](https://www.npmjs.com/package/expo-litert-lm)
5+
[![license](https://img.shields.io/npm/l/expo-litert-lm.svg)](https://github.com/helenkwok/expo-litert-lm/blob/main/LICENSE)
46

5-
**Status:** Android-first. iOS scaffold exists; full LiteRT-LM iOS support pending. Production-tested in [OfflineAid](https://github.com/helenkwok/offlineaid) (audio + chat).
7+
> Expo Modules bindings for [Google AI Edge LiteRT-LM](https://github.com/google-ai-edge/litert-lm) - on-device Gemma inference for Expo apps. **[View on npm →](https://www.npmjs.com/package/expo-litert-lm)**
8+
9+
**Status:** Android production-tested in [OfflineAid](https://github.com/helenkwok/offlineaid) with Gemma 4 E2B chat + audio. iOS LiteRT-LM text generation is validated on iPhone SE 3rd gen with `gemma3-1b-it-int4.litertlm` (Gemma 3 1B INT4); iOS audio/vision remain planned.
610

711
## What this is
812

9-
Native bridge for Expo SDK 56+ apps to load and run `.litertlm` and `.task` models from Google's `litert-community` Hugging Face org via the official `litertlm-android:0.11.0` SDK. MIT licensed.
13+
Native bridge for Expo SDK 55+ apps to load and run `.litertlm` and `.task` models from Google's `litert-community` Hugging Face org.
14+
15+
- Android uses Google's official `litertlm-android:0.11.0` SDK plus MediaPipe Tasks GenAI.
16+
- iOS uses vendored LiteRTLM-Swift wrapping the LiteRT-LM C API (`CLiteRTLM.xcframework` + `GemmaModelConstraintProvider.xcframework`, rewrapped from LiteRTLM-Swift `v0.10.2+rewrap.5`).
17+
- iOS `.task` support is available only through the opt-in `MediaPipeFallback` CocoaPods subspec; the default iOS install is LiteRT-LM only.
18+
19+
MIT licensed.
1020

1121
## What this is *not*
1222

@@ -15,10 +25,10 @@ Native bridge for Expo SDK 56+ apps to load and run `.litertlm` and `.task` mode
1525
| | `expo-litert-lm` (this) | `react-native-litert-lm` |
1626
|---|---|---|
1727
| Framework | Expo Modules | Nitro Modules + Expo plugin |
18-
| iOS LiteRT-LM | Scaffold only | Full |
28+
| iOS LiteRT-LM | Text/chat validated on iPhone SE 3rd gen with Gemma 3 1B INT4; audio/vision deferred | Full |
1929
| Android API | 31+ (S) | 26+ |
2030
| MTP / speculative decoding | `ExperimentalFlags.enableSpeculativeDecoding` wired against v0.11.0; blocked upstream on non-Adreno Android by [LiteRT-LM#2211](https://github.com/google-ai-edge/LiteRT-LM/issues/2211) (sampler `dlopen` fails on Tensor G2/G6 — cross-validated 2026-05-12) and [#2227](https://github.com/google-ai-edge/LiteRT-LM/issues/2227) (drafter regression on PowerVR even with sampler loaded). Architecture verified 2.01× lossless on Mac via mlx-vlm. | Not exposed |
21-
| Audio encoder | Production use | iOS limited |
31+
| Audio encoder | Android production use; iOS throws a clear not-yet-available error | iOS limited |
2232
| Multi-runtime | LiteRT-LM `.litertlm` + MediaPipe Tasks GenAI `.task` | LiteRT-LM only |
2333

2434
Both packages exist deliberately. Pick what fits your app.
@@ -37,7 +47,7 @@ Or pin a specific version in `package.json`:
3747

3848
```jsonc
3949
"dependencies": {
40-
"expo-litert-lm": "^0.1.1"
50+
"expo-litert-lm": "^0.2.0"
4151
}
4252
```
4353

@@ -82,10 +92,30 @@ if (await isLiteRtAvailable()) {
8292

8393
## Requirements
8494

85-
- Expo SDK 56+
95+
- Expo SDK 55+
8696
- Android 12 (API 31) or newer
87-
- A device with GPU support for best performance (e.g. Pixel 7+)
88-
- Models: `.litertlm` (LiteRT-LM) or `.task` (MediaPipe Tasks GenAI)
97+
- iOS 17.0 or newer for the LiteRT-LM path
98+
- Android: a device with GPU support for best performance (e.g. Pixel 7+)
99+
- iOS: iPhone SE 3rd gen / A15 is the validated floor for Gemma 3 1B INT4. Gemma 4 E2B has not been validated on SE 3rd gen due to memory limits.
100+
- Models: `.litertlm` (LiteRT-LM) or `.task` (MediaPipe Tasks GenAI; Android by default, iOS only with `MediaPipeFallback`)
101+
102+
## iOS Notes
103+
104+
`v0.2.0` adds the iOS LiteRT-LM path and ships the rewrapped xcframeworks inside the npm tarball. The included Expo config plugin injects the two binary pods that CocoaPods needs:
105+
106+
```jsonc
107+
{
108+
"expo": {
109+
"plugins": ["expo-litert-lm"]
110+
}
111+
}
112+
```
113+
114+
The plugin is compatible with Expo's `useFrameworks: "static"` consumer apps. A `useFrameworks: "dynamic"` diagnostic was run on the example app and CocoaPods rejected it because Expo SDK 55 has static transitive binaries (`ExpoModulesCore` / `ExpoModulesJSI`), so the split-binary pod path is intentional.
115+
116+
The iOS loader accepts both POSIX paths and `file://` URLs, including values returned by `expo-document-picker`.
117+
118+
The SE 3rd gen validation used the default CPU backend. iOS Metal/GPU acceleration is not claimed in this release.
89119

90120
## Security
91121

example/App.tsx

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -63,12 +63,17 @@ export default function App() {
6363
const pollRef = useRef<ReturnType<typeof setInterval> | null>(null);
6464
const cancelTsRef = useRef<number | null>(null);
6565

66+
const finishCancelLatency = () => {
67+
if (cancelTsRef.current === null) return;
68+
setCancelLatencyMs(Date.now() - cancelTsRef.current);
69+
cancelTsRef.current = null;
70+
};
71+
6672
useEffect(() => {
6773
const sub = addLiteRtTokenListener((event: LiteRtTokenEvent) => {
6874
setOutput(event.text);
6975
if (cancelTsRef.current !== null && event.done) {
70-
setCancelLatencyMs(Date.now() - cancelTsRef.current);
71-
cancelTsRef.current = null;
76+
finishCancelLatency();
7277
}
7378
});
7479
return () => sub.remove();
@@ -83,10 +88,9 @@ export default function App() {
8388
if (res.canceled) return;
8489
const uri = res.assets?.[0]?.uri;
8590
if (uri) {
86-
// Native module's URL(fileURLWithPath:) treats its arg as POSIX path,
87-
// so a file:// scheme prefix breaks fileExists. Strip it for Stage B.
88-
// Phase 14-09 follow-up: native module should accept either form.
89-
setModelPath(uri.startsWith('file://') ? uri.replace(/^file:\/\//, '') : uri);
91+
// Keep the picker URI intact so the native module exercises its
92+
// file:// URL normalization path.
93+
setModelPath(uri);
9094
}
9195
};
9296

@@ -139,6 +143,7 @@ export default function App() {
139143
} catch (e) {
140144
setErrorText(e instanceof Error ? e.message : String(e));
141145
} finally {
146+
finishCancelLatency();
142147
await stopPollingAndWrite();
143148
setRunning(false);
144149
}

example/README.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,8 @@ LiteRTLM-rewrapped.xcframework via the `file:..` link to the parent module.
3838
A 250 ms `sampleMemoryAsync` poll updates the on-screen MB and peakMb labels.
3939
3. Run finishes after ~50 s; final peakMb is the gate value.
4040
4. Re-tap **Run spike**, then **Cancel** mid-stream — verifies IOSC-03
41-
cancellation latency from the JS surface.
41+
cancellation latency from the JS surface. The app records latency even
42+
when native cancellation ends before a final token event is emitted.
4243
5. Retrieve `rss-stageB-*.ndjson` from Xcode → Devices → expolitertlmexample
4344
→ Download Container → AppData/Documents.
4445

@@ -49,3 +50,8 @@ LiteRTLM-rewrapped.xcframework via the `file:..` link to the parent module.
4950
- No audio / vision surface. Phase 14 spike scope is chat-only (D-02).
5051
- No model download from JS. Files-app-copy keeps the harness deterministic
5152
and the example free of HF auth code.
53+
54+
## Notes
55+
56+
- The picker passes its raw `file://` URI into `loadLiteRtModel`; the native
57+
iOS module accepts both picker URIs and POSIX paths.
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# The xcframeworks are gitignored but must be published to npm; consumers need
2+
# them during CocoaPods install.
3+
!.gitignore
4+
!*.xcframework
5+
!*.xcframework/**
6+
!rewrap-manifest.json

ios/ExpoLitertLmModule.swift

Lines changed: 49 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ public final class ExpoLitertLmModule: Module {
3333
private var loadedRuntime: LoadedRuntime?
3434
private var loadedConfig: LiteRtLoadConfig?
3535
private var loadedModelPath: String?
36+
private var securityScopedModelURL: URL?
3637

3738
private let stateQueue = DispatchQueue(label: "expo.modules.litertlm.state")
3839
private var activeGenerationID: UUID?
@@ -65,9 +66,11 @@ public final class ExpoLitertLmModule: Module {
6566
try self.ensureSupportedOS()
6667

6768
let resolvedBackend = (preferredBackend ?? "cpu").lowercased()
69+
let modelURL = try Self.resolveModelURL(modelPath)
70+
let resolvedModelPath = modelURL.path
6871
let nextConfig = LiteRtLoadConfig(
6972
maxTokens: maxTokens,
70-
modelPath: modelPath,
73+
modelPath: resolvedModelPath,
7174
preferredBackend: resolvedBackend,
7275
temperature: temperature,
7376
topK: topK
@@ -77,14 +80,17 @@ public final class ExpoLitertLmModule: Module {
7780
if self.loadedConfig == nextConfig, self.loadedRuntime != nil {
7881
return [
7982
"backend": resolvedBackend,
80-
"modelPath": modelPath,
83+
"modelPath": resolvedModelPath,
8184
]
8285
}
8386

8487
self.cancelActiveGeneration()
8588
await self.unloadInternalAsync()
8689

87-
if Self.isLiteRtLmModelPath(modelPath) {
90+
let didStartSecurityScope = modelURL.startAccessingSecurityScopedResource()
91+
var shouldStopSecurityScopeOnExit = didStartSecurityScope
92+
93+
if Self.isLiteRtLmModelPath(resolvedModelPath) {
8894
// LiteRTLM-Swift path — default for `.litertlm` models.
8995
// textOnly: true is REQUIRED for the v1.1 floor-device model
9096
// (gemma3-1b-it-int4.litertlm has no vision/audio encoders). Stage A
@@ -93,13 +99,16 @@ public final class ExpoLitertLmModule: Module {
9399
// Expo Modules consumer. Multimodal Gemma 4 callers must set this
94100
// to false explicitly when that path is wired (Phase 16).
95101
let engine = LiteRTLMEngine(
96-
modelPath: URL(fileURLWithPath: modelPath),
102+
modelPath: modelURL,
97103
backend: resolvedBackend,
98104
textOnly: true
99105
)
100106
do {
101107
try await engine.load()
102108
} catch {
109+
if shouldStopSecurityScopeOnExit {
110+
modelURL.stopAccessingSecurityScopedResource()
111+
}
103112
throw LiteRtModuleError(
104113
message: self.message(from: error, fallback: "LiteRT model loading failed.")
105114
)
@@ -114,32 +123,42 @@ public final class ExpoLitertLmModule: Module {
114123
let engine = MediaPipeFallbackEngine()
115124
do {
116125
try engine.load(
117-
modelPath: modelPath,
126+
modelPath: resolvedModelPath,
118127
maxTokens: maxTokens,
119128
topK: topK,
120129
temperature: temperature,
121130
preferredBackend: resolvedBackend
122131
)
123132
} catch {
133+
if shouldStopSecurityScopeOnExit {
134+
modelURL.stopAccessingSecurityScopedResource()
135+
}
124136
throw LiteRtModuleError(
125137
message: self.message(from: error, fallback: "MediaPipe model loading failed.")
126138
)
127139
}
128140
self.mediaPipeFallback = engine
129141
self.loadedRuntime = .mediaPipe
130142
#else
143+
if shouldStopSecurityScopeOnExit {
144+
modelURL.stopAccessingSecurityScopedResource()
145+
}
131146
throw LiteRtModuleError(
132147
message: ".task models require the MediaPipeFallback subspec. Add `pod 'ExpoLitertLm', :subspecs => ['Core', 'MediaPipeFallback']` to your Podfile per ExpoLitertLm CHANGELOG v0.2.0."
133148
)
134149
#endif
135150
}
136151

137152
self.loadedConfig = nextConfig
138-
self.loadedModelPath = modelPath
153+
self.loadedModelPath = resolvedModelPath
154+
if didStartSecurityScope {
155+
self.securityScopedModelURL = modelURL
156+
shouldStopSecurityScopeOnExit = false
157+
}
139158

140159
return [
141160
"backend": resolvedBackend,
142-
"modelPath": modelPath,
161+
"modelPath": resolvedModelPath,
143162
]
144163
}
145164

@@ -227,6 +246,25 @@ public final class ExpoLitertLmModule: Module {
227246
(modelPath as NSString).pathExtension.lowercased() == "litertlm"
228247
}
229248

249+
private static func resolveModelURL(_ modelPath: String) throws -> URL {
250+
if modelPath.lowercased().hasPrefix("file://") {
251+
if let url = URL(string: modelPath), url.isFileURL {
252+
return url
253+
}
254+
255+
let pathWithoutScheme = String(modelPath.dropFirst("file://".count))
256+
return URL(fileURLWithPath: pathWithoutScheme.removingPercentEncoding ?? pathWithoutScheme)
257+
}
258+
259+
if modelPath.contains("://") {
260+
throw LiteRtModuleError(
261+
message: "LiteRT models must be loaded from a local path or file:// URL."
262+
)
263+
}
264+
265+
return URL(fileURLWithPath: modelPath)
266+
}
267+
230268
// MARK: - LiteRTLM-Swift generation
231269

232270
private func runLitertLmGeneration(
@@ -321,6 +359,10 @@ public final class ExpoLitertLmModule: Module {
321359
loadedRuntime = nil
322360
loadedConfig = nil
323361
loadedModelPath = nil
362+
if let url = securityScopedModelURL {
363+
url.stopAccessingSecurityScopedResource()
364+
securityScopedModelURL = nil
365+
}
324366
}
325367

326368
private func emitTokenEvent(text: String, delta: String, done: Bool) async {

package-lock.json

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

package.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "expo-litert-lm",
3-
"version": "0.2.0-litertlm.0.10.2.r5",
3+
"version": "0.2.0-rc.0",
44
"description": "Expo Modules bindings for on-device LLM inference. Android: Google's litert-lm-android (LiteRT-LM Engine + MediaPipe Tasks GenAI). iOS: vendored LiteRTLM-Swift wrapping the LiteRT-LM C API; MediaPipe Tasks GenAI available as opt-in MediaPipeFallback subspec. Supports .litertlm and .task model formats.",
55
"main": "lib/index.js",
66
"types": "lib/index.d.ts",
@@ -15,6 +15,8 @@
1515
"ios/ExpoLitertLm.podspec",
1616
"ios/BinaryPods/CLiteRTLMBinary.podspec",
1717
"ios/BinaryPods/GemmaModelConstraintProviderBinary.podspec",
18+
"ios/BinaryPods/Frameworks/CLiteRTLM.xcframework",
19+
"ios/BinaryPods/Frameworks/GemmaModelConstraintProvider.xcframework",
1820
"ios/BinaryPods/Frameworks/rewrap-manifest.json",
1921
"src",
2022
"lib",

0 commit comments

Comments
 (0)