|
| 1 | +# AI SDK - ByteDance Provider |
| 2 | + |
| 3 | +The **ByteDance provider** for the [AI SDK](https://ai-sdk.dev/docs) contains video model support for ByteDance's Seedance family of video generation models through the [BytePlus ModelArk](https://docs.byteplus.com/en/docs/ModelArk/Video_Generation_API) platform. |
| 4 | + |
| 5 | +> **Deploying to Vercel?** With Vercel's AI Gateway you can access ByteDance (and hundreds of models from other providers) — no additional packages, API keys, or extra cost. [Get started with AI Gateway](https://vercel.com/ai-gateway). |
| 6 | +
|
| 7 | +## Setup |
| 8 | + |
| 9 | +The ByteDance provider is available in the `@ai-sdk/bytedance` module. You can install it with: |
| 10 | + |
| 11 | +```bash |
| 12 | +npm i @ai-sdk/bytedance |
| 13 | +``` |
| 14 | + |
| 15 | +## Skill for Coding Agents |
| 16 | + |
| 17 | +If you use coding agents such as Claude Code or Cursor, we highly recommend adding the AI SDK skill to your repository: |
| 18 | + |
| 19 | +```shell |
| 20 | +npx skills add vercel/ai |
| 21 | +``` |
| 22 | + |
| 23 | +## Provider Instance |
| 24 | + |
| 25 | +You can import the default provider instance `byteDance` from `@ai-sdk/bytedance`: |
| 26 | + |
| 27 | +```ts |
| 28 | +import { byteDance } from '@ai-sdk/bytedance'; |
| 29 | +``` |
| 30 | + |
| 31 | +## Video Models |
| 32 | + |
| 33 | +This provider supports text-to-video, image-to-video, audio-video sync, first-and-last frame control, and multi-reference image generation. |
| 34 | + |
| 35 | +### Text-to-Video |
| 36 | + |
| 37 | +Generate video from a text prompt. |
| 38 | + |
| 39 | +Available models: `seedance-1-5-pro-251215`, `seedance-1-0-pro-250528`, `seedance-1-0-pro-fast-251015`, `seedance-1-0-lite-t2v-250428` |
| 40 | + |
| 41 | +```ts |
| 42 | +import { byteDance } from '@ai-sdk/bytedance'; |
| 43 | +import { experimental_generateVideo as generateVideo } from 'ai'; |
| 44 | + |
| 45 | +const { video } = await generateVideo({ |
| 46 | + model: byteDance.video('seedance-1-0-pro-250528'), |
| 47 | + prompt: 'A cat playing with a ball of yarn in a sunlit room', |
| 48 | + aspectRatio: '16:9', |
| 49 | + duration: 5, |
| 50 | + providerOptions: { |
| 51 | + bytedance: { |
| 52 | + watermark: false, |
| 53 | + }, |
| 54 | + }, |
| 55 | +}); |
| 56 | + |
| 57 | +console.log(video.url); |
| 58 | +``` |
| 59 | + |
| 60 | +### Image-to-Video |
| 61 | + |
| 62 | +Generate video from a first-frame image with an optional text prompt. |
| 63 | + |
| 64 | +Available models: `seedance-1-5-pro-251215`, `seedance-1-0-pro-250528`, `seedance-1-0-pro-fast-251015`, `seedance-1-0-lite-i2v-250428` |
| 65 | + |
| 66 | +```ts |
| 67 | +import { byteDance } from '@ai-sdk/bytedance'; |
| 68 | +import { experimental_generateVideo as generateVideo } from 'ai'; |
| 69 | + |
| 70 | +const { video } = await generateVideo({ |
| 71 | + model: byteDance.video('seedance-1-5-pro-251215'), |
| 72 | + prompt: { |
| 73 | + image: 'https://example.com/first-frame.png', |
| 74 | + text: 'The cat slowly turns its head and blinks', |
| 75 | + }, |
| 76 | + duration: 5, |
| 77 | + providerOptions: { |
| 78 | + bytedance: { |
| 79 | + watermark: false, |
| 80 | + }, |
| 81 | + }, |
| 82 | +}); |
| 83 | +``` |
| 84 | + |
| 85 | +### Image-to-Video with Audio |
| 86 | + |
| 87 | +Seedance 1.5 Pro supports generating synchronized audio alongside the video. |
| 88 | + |
| 89 | +```ts |
| 90 | +import { byteDance } from '@ai-sdk/bytedance'; |
| 91 | +import { experimental_generateVideo as generateVideo } from 'ai'; |
| 92 | + |
| 93 | +const { video } = await generateVideo({ |
| 94 | + model: byteDance.video('seedance-1-5-pro-251215'), |
| 95 | + prompt: { |
| 96 | + image: 'https://example.com/pianist.png', |
| 97 | + text: 'A young man sits at a piano, playing calmly. Gentle piano music plays in sync.', |
| 98 | + }, |
| 99 | + duration: 5, |
| 100 | + providerOptions: { |
| 101 | + bytedance: { |
| 102 | + generateAudio: true, |
| 103 | + watermark: false, |
| 104 | + }, |
| 105 | + }, |
| 106 | +}); |
| 107 | +``` |
| 108 | + |
| 109 | +### First-and-Last Frame Video |
| 110 | + |
| 111 | +Generate smooth transitions between a starting and ending keyframe image. |
| 112 | + |
| 113 | +```ts |
| 114 | +import { byteDance } from '@ai-sdk/bytedance'; |
| 115 | +import { experimental_generateVideo as generateVideo } from 'ai'; |
| 116 | + |
| 117 | +const { video } = await generateVideo({ |
| 118 | + model: byteDance.video('seedance-1-5-pro-251215'), |
| 119 | + prompt: { |
| 120 | + image: 'https://example.com/first-frame.jpg', |
| 121 | + text: 'Create a 360-degree orbiting camera shot based on this photo', |
| 122 | + }, |
| 123 | + duration: 5, |
| 124 | + providerOptions: { |
| 125 | + bytedance: { |
| 126 | + lastFrameImage: 'https://example.com/last-frame.jpg', |
| 127 | + generateAudio: true, |
| 128 | + watermark: false, |
| 129 | + }, |
| 130 | + }, |
| 131 | +}); |
| 132 | +``` |
| 133 | + |
| 134 | +### Multi-Reference Image-to-Video |
| 135 | + |
| 136 | +Provide multiple reference images (1-4) that the model uses to faithfully reproduce object shapes, colors, and textures. Use `[Image 1]`, `[Image 2]`, etc. in your prompt to reference specific images. |
| 137 | + |
| 138 | +Available models: `seedance-1-0-lite-i2v-250428` |
| 139 | + |
| 140 | +```ts |
| 141 | +import { byteDance } from '@ai-sdk/bytedance'; |
| 142 | +import { experimental_generateVideo as generateVideo } from 'ai'; |
| 143 | + |
| 144 | +const { video } = await generateVideo({ |
| 145 | + model: byteDance.video('seedance-1-0-lite-i2v-250428'), |
| 146 | + prompt: |
| 147 | + 'A boy from [Image 1] and a corgi from [Image 2], sitting on the lawn from [Image 3]', |
| 148 | + aspectRatio: '16:9', |
| 149 | + duration: 5, |
| 150 | + providerOptions: { |
| 151 | + bytedance: { |
| 152 | + referenceImages: [ |
| 153 | + 'https://example.com/boy.png', |
| 154 | + 'https://example.com/corgi.png', |
| 155 | + 'https://example.com/lawn.png', |
| 156 | + ], |
| 157 | + watermark: false, |
| 158 | + }, |
| 159 | + }, |
| 160 | +}); |
| 161 | +``` |
| 162 | + |
| 163 | +## Provider Options |
| 164 | + |
| 165 | +Use `providerOptions.bytedance` to configure video generation: |
| 166 | + |
| 167 | +| Option | Description | |
| 168 | +| ----------------- | -------------------------------------------------------------------------------------- | |
| 169 | +| `watermark` | Whether to add a watermark to the generated video | |
| 170 | +| `generateAudio` | Generate synchronized audio (Seedance 1.5 Pro only) | |
| 171 | +| `cameraFixed` | Whether to fix the camera during generation | |
| 172 | +| `returnLastFrame` | Return the last frame of the generated video (useful for chaining) | |
| 173 | +| `serviceTier` | `'default'` for online inference, `'flex'` for offline at 50% price | |
| 174 | +| `draft` | Enable draft mode for low-cost 480p preview generation (Seedance 1.5 Pro only) | |
| 175 | +| `lastFrameImage` | URL of last frame image for first-and-last frame generation | |
| 176 | +| `referenceImages` | Array of reference image URLs (1-4) for multi-reference generation (1.0 Lite I2V only) | |
| 177 | +| `pollIntervalMs` | Poll interval for async generation (default: 3000ms) | |
| 178 | +| `pollTimeoutMs` | Max wait time for generation (default: 300000ms) | |
| 179 | + |
| 180 | +Supported aspect ratios: `16:9`, `4:3`, `1:1`, `3:4`, `9:16`, `21:9`, `adaptive` (image-to-video only). |
| 181 | + |
| 182 | +## Authentication |
| 183 | + |
| 184 | +Set the `ARK_API_KEY` environment variable: |
| 185 | + |
| 186 | +``` |
| 187 | +ARK_API_KEY=your-api-key |
| 188 | +``` |
| 189 | + |
| 190 | +Or pass it directly: |
| 191 | + |
| 192 | +```ts |
| 193 | +import { createByteDance } from '@ai-sdk/bytedance'; |
| 194 | + |
| 195 | +const byteDance = createByteDance({ |
| 196 | + apiKey: 'your-api-key', |
| 197 | +}); |
| 198 | +``` |
| 199 | + |
| 200 | +You can [obtain an API key](https://console.byteplus.com/ark/apiKey) from the BytePlus console. |
| 201 | + |
| 202 | +## Documentation |
| 203 | + |
| 204 | +Please check out the **[ByteDance provider](https://ai-sdk.dev/providers/ai-sdk-providers/bytedance)** for more information. |
![vercel-ai-sdk[bot]](https://avatars.githubusercontent.com/in/1765080?v=4&size=40){kind=avatar}
0 commit comments