Skip to content

Commit d20c26e

Browse files
Rahul-2903-juspaymurdore
authored andcommitted
docs(video-generation): Update docs for video generation feature
1 parent ebec022 commit d20c26e

20 files changed

Lines changed: 1423 additions & 14 deletions

README.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,9 +37,25 @@ Extracted from production systems at Juspay and battle-tested at enterprise scal
3737

3838
| Feature | Version | Description | Guide |
3939
| ---------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
40+
| **Video Generation with Veo** | v8.32.0 | Video generation using Veo 3.1 (`veo-3.1`). Realistic video generation with many parameter options | [Video Generation Guide](docs/features/video-generation.md) |
4041
| **Image Generation with Gemini** | v8.31.0 | Native image generation using Gemini 2.0 Flash Experimental (`imagen-3.0-generate-002`). High-quality image synthesis directly from Google AI. | [Image Generation Guide](docs/image-generation-streaming.md) |
4142
| **HTTP/Streamable HTTP Transport** | v8.29.0 | Connect to remote MCP servers via HTTP with authentication headers, automatic retry with exponential backoff, and configurable rate limiting. | [HTTP Transport Guide](docs/mcp-http-transport.md) |
4243

44+
- **Video Generation** – Transform images into 8-second videos with synchronized audio using Google Veo 3.1 via Vertex AI. Supports 720p/1080p resolutions, portrait/landscape aspect ratios. → [Video Generation Guide](docs/features/video-generation.md)
45+
- **Image Generation** – Generate images from text prompts using Gemini models via Vertex AI or Google AI Studio. Supports streaming mode with automatic file saving. → [Image Generation Guide](docs/IMAGE-GENERATION-STREAMING.md)
46+
- **HTTP/Streamable HTTP Transport for MCP** – Connect to remote MCP servers via HTTP with authentication headers, retry logic, and rate limiting. → [HTTP Transport Guide](docs/MCP-HTTP-TRANSPORT.md)
47+
- 🧠 **Gemini 3 Preview Support** - Full support for gemini-3-flash-preview and gemini-3-pro-preview with extended thinking capabilities
48+
- **Structured Output with Zod Schemas** – Type-safe JSON generation with automatic validation using `schema` + `output.format: "json"` in `generate()`. → [Structured Output Guide](docs/features/structured-output.md)
49+
- **CSV File Support** – Attach CSV files to prompts for AI-powered data analysis with auto-detection. → [CSV Guide](docs/features/multimodal-chat.md#csv-file-support)
50+
- **PDF File Support** – Process PDF documents with native visual analysis for Vertex AI, Anthropic, Bedrock, AI Studio. → [PDF Guide](docs/features/pdf-support.md)
51+
- **LiteLLM Integration** – Access 100+ AI models from all major providers through unified interface. → [Setup Guide](docs/LITELLM-INTEGRATION.md)
52+
- **SageMaker Integration** – Deploy and use custom trained models on AWS infrastructure. → [Setup Guide](docs/SAGEMAKER-INTEGRATION.md)
53+
- **OpenRouter Integration** – Access 300+ models from OpenAI, Anthropic, Google, Meta, and more through a single unified API. → [Setup Guide](docs/getting-started/providers/openrouter.md)
54+
- **Human-in-the-loop workflows** – Pause generation for user approval/input before tool execution. → [HITL Guide](docs/features/hitl.md)
55+
- **Guardrails middleware** – Block PII, profanity, and unsafe content with built-in filtering. → [Guardrails Guide](docs/features/guardrails.md)
56+
- **Context summarization** – Automatic conversation compression for long-running sessions. → [Summarization Guide](docs/CONTEXT-SUMMARIZATION.md)
57+
- **Redis conversation export** – Export full session history as JSON for analytics and debugging. → [History Guide](docs/features/conversation-history.md)
58+
4359
```typescript
4460
// Image Generation with Gemini (v8.31.0)
4561
const image = await neurolink.generateImage({

docs/cli-guide.md

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,16 @@ npx @juspay/neurolink gen "Analyze this problem" --provider google-ai --model ge
124124
- `--context <json>` - JSON context object for custom data (e.g., '{"userId":"123","project":"api-design"}')
125125
- `--disableTools` - Disable MCP tool integration (tools enabled by default)
126126

127+
**Video Generation Options (Veo 3.1):**
128+
129+
- `--outputMode <mode>` - Output mode: 'text' (default) or 'video'
130+
- `--image <path>` - Path to input image file for image-based video generation (required for video mode, e.g., ./input.jpg)
131+
- `--videoOutput <path>` - Path to save generated video file (e.g., ./output.mp4)
132+
- `--videoResolution <res>` - Video resolution: '720p' or '1080p' (default: 720p)
133+
- `--videoLength <seconds>` - Video duration: 4, 6, or 8 seconds (default: 6)
134+
- `--videoAspectRatio <ratio>` - Aspect ratio: '9:16' (portrait) or '16:9' (landscape, default: 16:9)
135+
- `--videoAudio <bool>` - Include synchronized audio (default: true)
136+
127137
**Output Example:**
128138

129139
```
@@ -1130,15 +1140,15 @@ npx @juspay/neurolink status
11301140
11311141
## When to Use CLI vs SDK
11321142
1133-
### Use the CLI when:
1143+
### Use the CLI when
11341144
11351145
- 🔧 **Prototyping**: Quick testing of prompts and providers
11361146
- 📜 **Scripting**: Shell scripts and automation workflows
11371147
- 🔍 **Debugging**: Checking provider status and testing connectivity
11381148
- 📊 **Batch Processing**: Processing multiple prompts from files
11391149
- 🎯 **One-off Tasks**: Generating content without writing code
11401150
1141-
### Use the SDK when:
1151+
### Use the SDK when
11421152
11431153
- 🏗️ **Application Development**: Building web apps, APIs, or services
11441154
- 🔄 **Real-time Integration**: Chat interfaces, streaming responses

docs/cli-reference.md

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,18 @@ npx @juspay/neurolink gen "Write code" --provider openai
6666
| ----------------- | ------- | ------- | ------------------------------------------------------- |
6767
| `--disable-tools` | boolean | `false` | Disable MCP tool integration (tools enabled by default) |
6868

69+
### Video Generation (Veo 3.1)
70+
71+
| Flag | Type | Default | Description |
72+
| ---------------------- | ------- | ------- | ------------------------------------------------------------------------- |
73+
| `--outputMode` | string | `text` | Output mode: 'text' or 'video' |
74+
| `--image` | string | none | Path to an input image to base the generated video on (e.g., ./input.png) |
75+
| `--videoOutput`, `-vo` | string | none | Path to save generated video (e.g., ./output.mp4) |
76+
| `--videoResolution` | string | `720p` | Video resolution: '720p' or '1080p' |
77+
| `--videoLength` | number | `6` | Video duration in seconds: 4, 6, or 8 |
78+
| `--videoAspectRatio` | string | `16:9` | Aspect ratio: '9:16' (portrait) or '16:9' (landscape) |
79+
| `--videoAudio` | boolean | `true` | Include synchronized audio |
80+
6981
## Usage Examples
7082

7183
### Basic Text Generation
@@ -264,6 +276,39 @@ Artificial Intelligence (AI) refers to...
264276
4. **Debug Efficiently**: Use `--debug` only when troubleshooting to avoid verbose output
265277
5. **Context Size**: Keep `--context` objects small to minimize token usage
266278

279+
## Video Generation Examples
280+
281+
Generate videos from images using Veo 3.1 via Vertex AI:
282+
283+
```bash
284+
# Basic video generation
285+
npx @juspay/neurolink generate "Product showcase with smooth camera movement" \
286+
--image ./product.jpg \
287+
--outputMode video \
288+
--videoOutput ./output.mp4
289+
290+
# Full options
291+
npx @juspay/neurolink generate "Cinematic reveal with dramatic lighting" \
292+
--image ./hero-image.png \
293+
--provider vertex \
294+
--model veo-3.1 \
295+
--outputMode video \
296+
--videoResolution 1080p \
297+
--videoLength 8 \
298+
--videoAspectRatio 16:9 \
299+
--videoOutput ./cinematic.mp4
300+
301+
# Portrait video for social media
302+
npx @juspay/neurolink generate "Vertical scroll animation" \
303+
--image ./mobile-screenshot.jpg \
304+
--outputMode video \
305+
--videoResolution 720p \
306+
--videoAspectRatio 9:16 \
307+
--videoOutput ./story.mp4
308+
```
309+
310+
> **Note:** Video generation requires Vertex AI credentials. See [Video Generation Guide](./features/video-generation.md).
311+
267312
## Environment Variables
268313

269314
See the [Environment Variables](./getting-started/environment-variables.md) documentation for complete configuration options.

docs/cli/commands.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,18 @@ Key flags:
5858
- `--debug` – verbose logging and full JSON payloads.
5959
- `--quiet` – suppress spinners.
6060

61+
**Video Generation (Veo 3.1):**
62+
63+
- `--outputMode` – output mode: `text` (default) or `video`.
64+
- `--image` – path to input image file (required for video generation, e.g., ./input.jpg).
65+
- `--videoOutput`, `-vo` – path to save generated video file.
66+
- `--videoResolution``720p` or `1080p` (default `720p`).
67+
- `--videoLength` – duration: `4`, `6`, or `8` seconds (default `6`).
68+
- `--videoAspectRatio``9:16` (portrait) or `16:9` (landscape, default `16:9`).
69+
- `--videoAudio` – include synchronized audio (default `true`).
70+
71+
**Note:** Video generation requires Vertex AI provider (`vertex`) and Veo 3.1 model (`veo-3.1`). The provider auto-switches to Vertex when `--outputMode video` is specified. Supported image formats: PNG, JPEG, WebP (max 20MB).
72+
6173
`gen` is a short alias with the same options.
6274

6375
### `stream <input>` {#stream}

docs/configuration.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -194,6 +194,26 @@ export MEM0_VECTOR_STORE="qdrant"
194194

195195
Consult [`docs/mem0-integration.md`](mem0-integration.md) for vector store specifics and SDK examples.
196196

197+
### **Video Generation (Veo 3.1)**
198+
199+
Configure Vertex AI credentials to enable video generation with Veo 3.1. Choose **one** of the following credential options:
200+
201+
```bash
202+
# Option 1: Service account file (recommended for production)
203+
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"
204+
export GOOGLE_VERTEX_PROJECT="your-project-id"
205+
export GOOGLE_VERTEX_LOCATION="us-central1"
206+
207+
# Option 2: Environment variables (alternative to service account file)
208+
# GOOGLE_CLOUD_* variables are equivalent to GOOGLE_VERTEX_* for project/location configuration
209+
export GOOGLE_CLOUD_PROJECT="your-project-id"
210+
export GOOGLE_CLOUD_LOCATION="us-central1"
211+
```
212+
213+
**Note:** These are alternative approaches—you only need one. `GOOGLE_VERTEX_PROJECT` and `GOOGLE_CLOUD_PROJECT` map to the same project configuration; similarly, `GOOGLE_VERTEX_LOCATION` and `GOOGLE_CLOUD_LOCATION` specify the same region.
214+
215+
See [Video Generation Guide](features/video-generation.md) for complete setup and usage.
216+
197217
### **Regional Routing**
198218

199219
Set region-specific variables to control latency and compliance.

docs/dynamic-models.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ The dynamic model system enables:
1111
- **Smart model resolution** with fuzzy matching and aliases
1212
- **Capability-based search** to find models with specific features
1313
- **Cost optimization** by automatically selecting cheapest models for tasks
14+
- **Video generation** support via Veo 3.1 model on Vertex AI
1415

1516
## 🏗️ Architecture
1617

docs/error-handling.md

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,17 @@ This document covers error handling strategies in NeuroLink.
2222
- Memory allocation issues
2323
- Timeout errors
2424

25+
### Video Generation Errors
26+
27+
Video generation via Veo 3.1 on Vertex AI may encounter specific error conditions:
28+
29+
- **VIDEO_GENERATION_FAILED** - Video generation process failed
30+
- **PROVIDER_NOT_CONFIGURED** - Vertex AI credentials not configured
31+
- **VIDEO_POLL_TIMEOUT** - Video generation timed out (exceeds 3 minutes)
32+
- **VIDEO_INVALID_INPUT** - Invalid image format or parameters
33+
- **VIDEO_QUOTA_EXCEEDED** - Vertex AI quota or rate limit exceeded
34+
- **VIDEO_REGION_UNAVAILABLE** - Veo 3.1 not available in specified region
35+
2536
## Error Recovery
2637

2738
### Automatic Retry
@@ -36,6 +47,77 @@ Configure fallback providers to handle primary provider failures.
3647

3748
System continues to operate with reduced functionality when errors occur.
3849

50+
### Video Generation Error Handling
51+
52+
**Example: Handling video generation errors**
53+
54+
```typescript
55+
import { NeuroLink } from "@juspay/neurolink";
56+
import { readFile, writeFile } from "fs/promises";
57+
58+
const neurolink = new NeuroLink();
59+
60+
try {
61+
const result = await neurolink.generate({
62+
input: {
63+
text: "Product showcase video",
64+
images: [await readFile("./product.jpg")],
65+
},
66+
provider: "vertex",
67+
model: "veo-3.1",
68+
output: {
69+
mode: "video",
70+
video: {
71+
resolution: "1080p",
72+
length: 8,
73+
aspectRatio: "16:9",
74+
},
75+
},
76+
timeout: 180, // 3 minutes for video generation
77+
});
78+
79+
if (result.video) {
80+
await writeFile("output.mp4", result.video.data);
81+
}
82+
} catch (error) {
83+
// Use your logger for production: logger.error('Video generation failed', { code: error.code, error })
84+
if (error.code === "PROVIDER_NOT_CONFIGURED") {
85+
console.error(
86+
"Vertex AI credentials not configured. Set GOOGLE_APPLICATION_CREDENTIALS.",
87+
);
88+
} else if (error.code === "VIDEO_POLL_TIMEOUT") {
89+
console.error(
90+
"Video generation timed out. Try again or reduce video length.",
91+
);
92+
} else if (error.code === "VIDEO_INVALID_INPUT") {
93+
console.error(
94+
"Invalid image format. Ensure PNG, JPEG, or WebP under 20MB.",
95+
);
96+
} else if (error.code === "VIDEO_QUOTA_EXCEEDED") {
97+
console.error("Vertex AI quota exceeded. Check your billing and quotas.");
98+
} else {
99+
console.error("Video generation failed:", error.message);
100+
}
101+
}
102+
```
103+
104+
**CLI Error Handling:**
105+
106+
```bash
107+
# Video generation with error handling
108+
npx @juspay/neurolink generate "Product video" \
109+
--image ./product.jpg \
110+
--outputMode video \
111+
--videoOutput ./output.mp4 \
112+
--timeout 180
113+
114+
# Check exit code for automation
115+
if [ $? -ne 0 ]; then
116+
echo "Video generation failed"
117+
exit 1
118+
fi
119+
```
120+
39121
## Monitoring and Logging
40122

41123
### Error Logging

0 commit comments

Comments
 (0)