feat(assemblyai): new STT plugin for Universal-Streaming (v3)

Adds a new `@livekit/agents-plugin-assemblyai` STT plugin that connects to
AssemblyAI's Universal-Streaming v3 WebSocket API. Ported line-by-line from
the Python `livekit-plugins-assemblyai` plugin; every non-trivial block has
a `// Ref: python .../stt.py - N-M lines` comment pointing back to the
source so future syncs are straightforward.

Architecture follows the existing Deepgram STT plugin: retry loop around a
single WebSocket lifetime, concurrent sendTask/listenTask/configTask under
`#runWS`, graceful close via `{"type":"Terminate"}`.

Key features:
- All AssemblyAI v3 options exposed: speechModel (universal-streaming-*,
  u3-rt-pro), sampleRate, bufferSizeMs, min/maxTurnSilence, formatTurns,
  keytermsPrompt, prompt (u3-rt-pro only), vadThreshold, endOfTurnConfidenceThreshold,
  speakerLabels, maxSpeakers, domain, custom baseUrl (for EU region).
- Live config updates via `UpdateConfiguration` messages over the active
  socket — no reconnect needed, unlike Deepgram's updateOptions.
- `forceEndpoint()` to finalize the current turn on demand.
- `sessionId` / `expiresAt` getters for debugging; `sessionId` is also
  propagated as `requestId` on RECOGNITION_USAGE events.
- Turn-event mapping: `Turn` message → INTERIM_TRANSCRIPT (cumulative),
  PREFLIGHT_TRANSCRIPT (chunk-based from `utterance`), FINAL_TRANSCRIPT +
  END_OF_SPEECH on `end_of_turn` (with `turn_is_formatted` check when
  `formatTurns` is enabled).
- Word timestamps converted from ms → seconds for `createTimedString`.
- `u3-pro` → `u3-rt-pro` rename handled with deprecation warning (matches
  Python).

Known framework gap: `stt.SpeechData` in `@livekit/agents` doesn't yet have
a `speakerId` field, so per-word speaker labels from AssemblyAI's
diarization are not surfaced on emitted events. The `speakerLabels` option
is exposed and sent to the server; a follow-up PR can wire through speaker
ids once the base type is extended. See the TSDoc on `speakerLabels` for
the exact forwarding point.

Smoke-tested end-to-end against the AssemblyAI API using the shared
`@livekit/agents-plugins-test` STT harness; transcription matches the
reference transcript within the 20% Levenshtein threshold.

Also adds `ASSEMBLYAI_API_KEY` to `turbo.json` globalEnv (required for
lint to pass).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: chasef07

.changeset/assemblyai-stt-plugin.md

@@ -0,0 +1,5 @@
+---
+"@livekit/agents-plugin-assemblyai": patch
+---
+
+feat(assemblyai): new STT plugin for AssemblyAI Universal-Streaming (v3) API. Ported from the Python livekit-plugins-assemblyai plugin.

plugins/assemblyai/README.md

@@ -0,0 +1,18 @@
+<!--
+SPDX-FileCopyrightText: 2026 LiveKit, Inc.
+
+SPDX-License-Identifier: Apache-2.0
+-->
+# AssemblyAI plugin for LiveKit Agents
+
+The Agents Framework is designed for building realtime, programmable
+participants that run on servers. Use it to create conversational, multi-modal
+voice agents that can see, hear, and understand.
+
+This package contains the AssemblyAI plugin, which allows for speech recognition
+via AssemblyAI's Universal-Streaming (v3) API. Refer to the
+[documentation](https://docs.livekit.io/agents/overview/) for information on how
+to use it, or browse the [API
+reference](https://docs.livekit.io/agents-js/modules/plugins_agents_plugin_assemblyai.html).
+See the [repository](https://github.com/livekit/agents-js) for more information
+about the framework as a whole.

plugins/assemblyai/api-extractor.json

@@ -0,0 +1,20 @@
+/**
+ * Config file for API Extractor.  For more info, please visit: https://api-extractor.com
+ */
+{
+  "$schema": "https://developer.microsoft.com/json-schemas/api-extractor/v7/api-extractor.schema.json",
+
+  /**
+   * Optionally specifies another JSON config file that this file extends from.  This provides a way for
+   * standard settings to be shared across multiple projects.
+   *
+   * If the path starts with "./" or "../", the path is resolved relative to the folder of the file that contains
+   * the "extends" field.  Otherwise, the first path segment is interpreted as an NPM package name, and will be
+   * resolved using NodeJS require().
+   *
+   * SUPPORTED TOKENS: none
+   * DEFAULT VALUE: ""
+   */
+  "extends": "../../api-extractor-shared.json",
+  "mainEntryPointFilePath": "./dist/index.d.ts"
+}

plugins/assemblyai/package.json

@@ -0,0 +1,53 @@
+{
+  "name": "@livekit/agents-plugin-assemblyai",
+  "version": "1.2.4",
+  "description": "AssemblyAI plugin for LiveKit Agents for Node.js",
+  "main": "dist/index.js",
+  "require": "dist/index.cjs",
+  "types": "dist/index.d.ts",
+  "exports": {
+    "import": {
+      "types": "./dist/index.d.ts",
+      "default": "./dist/index.js"
+    },
+    "require": {
+      "types": "./dist/index.d.cts",
+      "default": "./dist/index.cjs"
+    }
+  },
+  "author": "LiveKit",
+  "type": "module",
+  "repository": "git@github.com:livekit/agents-js.git",
+  "license": "Apache-2.0",
+  "files": [
+    "dist",
+    "src",
+    "README.md"
+  ],
+  "scripts": {
+    "build": "tsup --onSuccess \"pnpm build:types\"",
+    "build:types": "tsc --declaration --emitDeclarationOnly && node ../../scripts/copyDeclarationOutput.js",
+    "clean": "rm -rf dist",
+    "clean:build": "pnpm clean && pnpm build",
+    "lint": "eslint -f unix \"src/**/*.{ts,js}\"",
+    "api:check": "api-extractor run --typescript-compiler-folder ../../node_modules/typescript",
+    "api:update": "api-extractor run --local --typescript-compiler-folder ../../node_modules/typescript --verbose"
+  },
+  "devDependencies": {
+    "@livekit/agents": "workspace:*",
+    "@livekit/agents-plugin-silero": "workspace:*",
+    "@livekit/agents-plugins-test": "workspace:*",
+    "@livekit/rtc-node": "catalog:",
+    "@microsoft/api-extractor": "^7.35.0",
+    "@types/ws": "^8.5.10",
+    "tsup": "^8.3.5",
+    "typescript": "^5.0.0"
+  },
+  "dependencies": {
+    "ws": "^8.16.0"
+  },
+  "peerDependencies": {
+    "@livekit/agents": "workspace:*",
+    "@livekit/rtc-node": "catalog:"
+  }
+}

plugins/assemblyai/src/index.ts

@@ -0,0 +1,19 @@
+// SPDX-FileCopyrightText: 2026 LiveKit, Inc.
+//
+// SPDX-License-Identifier: Apache-2.0
+import { Plugin } from '@livekit/agents';
+
+export * from './models.js';
+export * from './stt.js';
+
+class AssemblyAIPlugin extends Plugin {
+  constructor() {
+    super({
+      title: 'assemblyai',
+      version: __PACKAGE_VERSION__,
+      package: __PACKAGE_NAME__,
+    });
+  }
+}
+
+Plugin.registerPlugin(new AssemblyAIPlugin());

plugins/assemblyai/src/models.ts

@@ -0,0 +1,16 @@
+// SPDX-FileCopyrightText: 2026 LiveKit, Inc.
+//
+// SPDX-License-Identifier: Apache-2.0
+
+// Ref: python livekit-plugins/livekit-plugins-assemblyai/livekit/plugins/assemblyai/stt.py - 48-54 lines
+export type STTModels =
+  | 'universal-streaming-english'
+  | 'universal-streaming-multilingual'
+  | 'u3-rt-pro'
+  // Deprecated alias — AssemblyAI maps this to `u3-rt-pro` server-side, but the
+  // Python plugin emits a warning and rewrites it. Kept here so TS users don't
+  // break if they already pass it.
+  | 'u3-pro';
+
+// Ref: python livekit-plugins/livekit-plugins-assemblyai/livekit/plugins/assemblyai/stt.py - 47 lines
+export type STTEncoding = 'pcm_s16le' | 'pcm_mulaw';

plugins/assemblyai/src/stt.test.ts

@@ -0,0 +1,19 @@
+// SPDX-FileCopyrightText: 2026 LiveKit, Inc.
+//
+// SPDX-License-Identifier: Apache-2.0
+import { VAD } from '@livekit/agents-plugin-silero';
+import { stt } from '@livekit/agents-plugins-test';
+import { describe, it } from 'vitest';
+import { STT } from './stt.js';
+
+const hasAssemblyAIApiKey = Boolean(process.env.ASSEMBLYAI_API_KEY);
+
+if (hasAssemblyAIApiKey) {
+  describe('AssemblyAI', async () => {
+    await stt(new STT(), await VAD.load(), { nonStreaming: false });
+  });
+} else {
+  describe('AssemblyAI', () => {
+    it.skip('requires ASSEMBLYAI_API_KEY', () => {});
+  });
+}