This is some low-hanging fruit that will improve the library's usability for interacting with vLLM models.
If we were to add a Reasoning field to the ChatCompletionChunkChoiceDelta struct definition, as shown below, then we would be able to access the returned reasoning, if any.
// A chat completion delta generated by streamed model responses.
type ChatCompletionChunkChoiceDelta struct {
// The contents of the chunk message.
Content string `json:"content" api:"nullable"`
// The reasoning produced by the model.
Reasoning string `json:"reasoning" api:"nullable"`
// Deprecated and replaced by `tool_calls`. The name and arguments of a function
// that should be called, as generated by the model.
//
// Deprecated: deprecated
FunctionCall ChatCompletionChunkChoiceDeltaFunctionCall `json:"function_call"`
// The refusal message generated by the model.
Refusal string `json:"refusal" api:"nullable"`
// The role of the author of this message.
//
// Any of "developer", "system", "user", "assistant", "tool".
Role string `json:"role"`
ToolCalls []ChatCompletionChunkChoiceDeltaToolCall `json:"tool_calls"`
// JSON contains metadata for fields, check presence with [respjson.Field.Valid].
JSON struct {
Content respjson.Field
FunctionCall respjson.Field
Refusal respjson.Field
Role respjson.Field
ToolCalls respjson.Field
ExtraFields map[string]respjson.Field
raw string
} `json:"-"`
}
I would find this useful since it would allow me to return thinking content to the caller so they can see the reasoning process and know that the model has not hung unexpectedly, and also monitor that it is answering the prompt as intended.
This is some low-hanging fruit that will improve the library's usability for interacting with vLLM models.
If we were to add a
Reasoningfield to theChatCompletionChunkChoiceDeltastruct definition, as shown below, then we would be able to access the returned reasoning, if any.I would find this useful since it would allow me to return thinking content to the caller so they can see the reasoning process and know that the model has not hung unexpectedly, and also monitor that it is answering the prompt as intended.