Skip to content

Commit f9884c7

Browse files
committed
docs: Update documentation for Venice Large 128k context window
- Add Venice Large model examples and context window guidance to README.md - Update client utilities documentation with model capability notes - Enhance async streaming guide with large context usage examples - Bump version to 1.1.2 - Update CHANGELOG with comprehensive release notes Reflects API improvements: - Venice Large context increased from 32k to 128k tokens - Server-side thinking message processing improvements
1 parent 2e7e6e4 commit f9884c7

5 files changed

Lines changed: 47 additions & 2 deletions

File tree

CHANGELOG.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,24 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [1.1.2] - 2025-06-19
9+
10+
### Changed
11+
12+
- **Documentation Updates**: Updated documentation to reflect Venice.ai API improvements
13+
- Added information about Venice Large model's increased context window (32k → 128k tokens)
14+
- Enhanced `README.md` with Venice Large examples and context window guidance
15+
- Updated `docs/client_utilities.rst` with model capability notes and token management best practices
16+
- Enhanced `docs/async_chat_streaming_guide.rst` with large context window usage guidance
17+
- Added practical examples showing how to leverage the 128k context window with `max_completion_tokens`
18+
19+
### Notes
20+
21+
- **API Compatibility**: No SDK code changes required - existing functionality automatically benefits from API improvements
22+
- Venice Large's increased context size can be utilized through existing `max_completion_tokens` parameter
23+
- Non-streaming chat completions now receive cleaner responses due to server-side "thinking" message processing improvements
24+
- Streaming behavior remains unchanged and continues to pass through all API-sent events
25+
826
## [1.1.1] - 2025-06-13
927

1028
### Fixed

README.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,6 +199,10 @@ It's important to `close()` (for `VeniceClient`) or `await async_client.close()`
199199

200200
The response objects (`response`, `chunk`) are `TypedDict`s. You can explore their structure for more details (see `src/venice_ai/types/chat.py` or the Sphinx-generated API documentation).
201201

202+
**Model Context Windows:**
203+
204+
Different models support different context window sizes. For example, the "Venice Large" model supports up to 128k tokens, allowing for extensive conversations or document processing. Use the `max_completion_tokens` parameter to control response length within the model's context limits.
205+
202206
**Parameters:**
203207

204208
:param logprobs: Whether to return log probabilities of the output tokens. If `True`, the `logprobs` field will be populated in the `choices` of the response. Defaults to `False`.
@@ -222,6 +226,24 @@ with VeniceClient() as client:
222226
print(response.choices[0].message.content)
223227
```
224228

229+
**Example with Venice Large (128k context window):**
230+
231+
```python
232+
from venice_ai import VeniceClient
233+
234+
with VeniceClient() as client:
235+
# Venice Large supports up to 128k tokens, ideal for long documents or conversations
236+
response = client.chat.completions.create(
237+
model="venice-large",
238+
messages=[
239+
{"role": "system", "content": "You are a helpful assistant that can analyze long documents."},
240+
{"role": "user", "content": "Please analyze this extensive document..."} # Can include very long content
241+
],
242+
max_completion_tokens=4000 # Can use higher values with Venice Large's 128k context
243+
)
244+
print(response.choices[0].message.content)
245+
```
246+
225247
**Example with `logprobs` and `top_logprobs`:**
226248

227249
```python
@@ -435,6 +457,8 @@ with VeniceClient() as client:
435457
print(f"Text models: {[m.id for m in text_models.data]}")
436458
```
437459

460+
**Note:** Different models have varying capabilities and context window sizes. For example, "Venice Large" supports up to 128k tokens, making it ideal for processing extensive documents or maintaining long conversations. Refer to the official Venice AI documentation for detailed specifications of each model.
461+
438462
### API Key Management
439463

440464
```python

docs/async_chat_streaming_guide.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,7 @@ Further Exploration
119119
-------------------
120120

121121
- Explore other parameters of the `create` method to customize model behavior (e.g., `temperature`, `max_completion_tokens`).
122+
- When using models with larger context windows like "Venice Large" (128k tokens), you can adjust `max_completion_tokens` accordingly to leverage the full context for longer responses or more detailed analysis.
122123
- Integrate this streaming logic into a web application or a command-line interface for a more interactive experience.
123124
- Refer to the :ref:`API Reference <api_reference>` for detailed information on all available methods and parameters.
124125

docs/client_utilities.rst

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,8 @@ Model Utilities
6767
# Run the async example
6868
asyncio.run(example_model_utilities())
6969
70+
**Note:** For detailed specifications of individual models, such as context window size (e.g., "Venice Large" supports up to 128k tokens), refer to the official Venice AI model documentation. Different models have varying capabilities that may affect your application's design.
71+
7072
Chat Message Utilities
7173
~~~~~~~~~~~~~~~~~~~~~~
7274

@@ -130,5 +132,5 @@ Best Practices
130132

131133
- **Error Handling**: Always check for errors when using validation functions.
132134
- **Capabilities Checking**: Use the model utility functions to ensure you're working with models that support your required features.
133-
- **Token Management**: Use ``estimate_token_count`` to proactively manage token usage within limits.
135+
- **Token Management**: Use ``estimate_token_count`` to proactively manage token usage, especially when working with models with large context windows like "Venice Large" (128k tokens), to stay within API and model limits.
134136
- **Async Consistency**: Most model utility functions are async and should be awaited properly.

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"
44

55
[tool.poetry]
66
name = "venice-ai"
7-
version = "1.1.1"
7+
version = "1.1.2"
88
description = "Python client library for interacting with the Venice.ai API, offering comprehensive access to its features."
99
readme = "README.md"
1010
license = "MIT"

0 commit comments

Comments
 (0)