You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: Update documentation for Venice Large 128k context window
- Add Venice Large model examples and context window guidance to README.md
- Update client utilities documentation with model capability notes
- Enhance async streaming guide with large context usage examples
- Bump version to 1.1.2
- Update CHANGELOG with comprehensive release notes
Reflects API improvements:
- Venice Large context increased from 32k to 128k tokens
- Server-side thinking message processing improvements
Copy file name to clipboardExpand all lines: README.md
+24Lines changed: 24 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -199,6 +199,10 @@ It's important to `close()` (for `VeniceClient`) or `await async_client.close()`
199
199
200
200
The response objects (`response`, `chunk`) are `TypedDict`s. You can explore their structure for more details (see `src/venice_ai/types/chat.py` or the Sphinx-generated API documentation).
201
201
202
+
**Model Context Windows:**
203
+
204
+
Different models support different context window sizes. For example, the "Venice Large" model supports up to 128k tokens, allowing for extensive conversations or document processing. Use the `max_completion_tokens` parameter to control response length within the model's context limits.
205
+
202
206
**Parameters:**
203
207
204
208
:param logprobs: Whether to return log probabilities of the output tokens. If `True`, the `logprobs` field will be populated in the `choices` of the response. Defaults to `False`.
@@ -222,6 +226,24 @@ with VeniceClient() as client:
222
226
print(response.choices[0].message.content)
223
227
```
224
228
229
+
**Example with Venice Large (128k context window):**
230
+
231
+
```python
232
+
from venice_ai import VeniceClient
233
+
234
+
with VeniceClient() as client:
235
+
# Venice Large supports up to 128k tokens, ideal for long documents or conversations
236
+
response = client.chat.completions.create(
237
+
model="venice-large",
238
+
messages=[
239
+
{"role": "system", "content": "You are a helpful assistant that can analyze long documents."},
240
+
{"role": "user", "content": "Please analyze this extensive document..."} # Can include very long content
241
+
],
242
+
max_completion_tokens=4000# Can use higher values with Venice Large's 128k context
243
+
)
244
+
print(response.choices[0].message.content)
245
+
```
246
+
225
247
**Example with `logprobs` and `top_logprobs`:**
226
248
227
249
```python
@@ -435,6 +457,8 @@ with VeniceClient() as client:
435
457
print(f"Text models: {[m.id for m in text_models.data]}")
436
458
```
437
459
460
+
**Note:** Different models have varying capabilities and context window sizes. For example, "Venice Large" supports up to 128k tokens, making it ideal for processing extensive documents or maintaining long conversations. Refer to the official Venice AI documentation for detailed specifications of each model.
Copy file name to clipboardExpand all lines: docs/async_chat_streaming_guide.rst
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -119,6 +119,7 @@ Further Exploration
119
119
-------------------
120
120
121
121
- Explore other parameters of the `create` method to customize model behavior (e.g., `temperature`, `max_completion_tokens`).
122
+
- When using models with larger context windows like "Venice Large" (128k tokens), you can adjust `max_completion_tokens` accordingly to leverage the full context for longer responses or more detailed analysis.
122
123
- Integrate this streaming logic into a web application or a command-line interface for a more interactive experience.
123
124
- Refer to the :ref:`API Reference <api_reference>` for detailed information on all available methods and parameters.
Copy file name to clipboardExpand all lines: docs/client_utilities.rst
+3-1Lines changed: 3 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -67,6 +67,8 @@ Model Utilities
67
67
# Run the async example
68
68
asyncio.run(example_model_utilities())
69
69
70
+
**Note:** For detailed specifications of individual models, such as context window size (e.g., "Venice Large" supports up to 128k tokens), refer to the official Venice AI model documentation. Different models have varying capabilities that may affect your application's design.
71
+
70
72
Chat Message Utilities
71
73
~~~~~~~~~~~~~~~~~~~~~~
72
74
@@ -130,5 +132,5 @@ Best Practices
130
132
131
133
- **Error Handling**: Always check for errors when using validation functions.
132
134
- **Capabilities Checking**: Use the model utility functions to ensure you're working with models that support your required features.
133
-
- **Token Management**: Use ``estimate_token_count`` to proactively manage token usagewithin limits.
135
+
- **Token Management**: Use ``estimate_token_count`` to proactively manage token usage, especially when working with models with large context windows like "Venice Large" (128k tokens), to stay within API and model limits.
134
136
- **Async Consistency**: Most model utility functions are async and should be awaited properly.
0 commit comments