[iOS] Add cancel/stop generation API for LlmInference

[DenisovAV]

Description

The iOS SDK (MediaPipeTasksGenAI CocoaPods) does not expose a cancel/stop generation API for LlmInference, while Android and Web already support it.

Current State

Platform	Cancel API	Status
Android	cancelGenerateResponseAsync()	✅ Available
Web	cancelProcessing()	✅ Available since @mediapipe/tasks-genai@0.10.26 (#6128)
iOS	—	❌ Missing

Use Case

Users want to cancel an ongoing LLM response mid-stream — this is a standard UX pattern in all chat AI applications. Without cancellation on iOS, the only workaround is closing the entire session and creating a new one, which is slow and wastes resources.

Requested API

Add a cancelProcessing() or equivalent method to LlmInference on iOS (Swift), matching the Android and Web APIs.

Swift example:

let llmInference = try LlmInference(options: options)
// Start async generation
llmInference.generateResponseAsync(inputText: prompt) { partial, error in ... }
// Cancel mid-stream
llmInference.cancelProcessing()  // <-- this doesn't exist on iOS

Context

We maintain flutter_gemma, a Flutter plugin for on-device LLM inference. Cancel generation is one of the most requested features by our users (see DenisovAV/flutter_gemma#34, DenisovAV/flutter_gemma#194).

Related

• Stop response generation before completion #6128 — Web cancelProcessing() (fixed in 0.10.26)
• Allow cancelling ongoing async generation in LlmInference #5327 — Earlier stop generation request (closed without resolution)

[HarishPermal]

Hi @DenisovAV,

Thanks for reporting, will look into this and get back to you with updates. 😊

[m13v]

ran into the same need for cancel/stop on iOS with on-device inference. when the user starts speaking a new command mid-generation, you need to kill the current inference immediately. we ended up using a cancellation token pattern - the inference loop checks a flag on each token emission and bails if it's set. not ideal since it still finishes the current forward pass, but at least you don't wait for the full generation. would be great to have this at the framework level instead of everyone reimplementing it.

[m13v]

for reference, here's how we handle the cancel-on-new-input pattern in our voice-first AI agent: https://github.com/m13v/fazm/blob/main/Desktop/Sources/Providers/ChatProvider.swift - the LLM provider has cancellation support so when a new voice command comes in mid-generation, we abort the current stream and start fresh.