Skip to content

[IMPROVEMENT PERFORMANCE] System.IO.MemoryStream replaced with Microsoft.IO.RecyclableMemoryStream#1197

Draft
moljac wants to merge 2 commits into
openai:mainfrom
Forks4Work-Microsoft-Xamarin:dev/moljac/20260604-memorystream-vs-recyclablememorystream
Draft

[IMPROVEMENT PERFORMANCE] System.IO.MemoryStream replaced with Microsoft.IO.RecyclableMemoryStream#1197
moljac wants to merge 2 commits into
openai:mainfrom
Forks4Work-Microsoft-Xamarin:dev/moljac/20260604-memorystream-vs-recyclablememorystream

Conversation

@moljac

@moljac moljac commented Jun 4, 2026 •
edited
Loading

Copy link
Copy Markdown

System.IO.MemoryStream was replaced with Microsoft.IO.RecyclableMemoryStream

Changes:

  1. added package reference Microsoft.IO.RecyclableMemoryStream
  2. System.IO.MemoryStream instance occurences replaced with Microsoft.IO.RecyclableMemoryStream

Microsoft.IO.RecyclableMemoryStream replaces standard MemoryStream instances with pooled byte buffers, eliminating Large Object Heap (LOH) allocations and significantly reducing Garbage Collection (GC) pauses in high-throughput applications

Utilities like Kiota do use Microsoft.IO.RecyclableMemoryStream as dependency

Copilot AI review requested due to automatic review settings June 4, 2026 16:02
Copilot AI reviewed Jun 4, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR replaces ad-hoc MemoryStream allocations with Microsoft.IO.RecyclableMemoryStream across the library and tests, introducing a shared MemoryStreamManager and adding the required package reference.

Changes:

  • Add Microsoft.IO.RecyclableMemoryStream as a dependency and centralize its version.
  • Update production code paths that buffer JSON/response content to use RecyclableMemoryStream.
  • Update tests/examples to use RecyclableMemoryStreamManager.GetStream() instead of new MemoryStream().

Reviewed changes

Copilot reviewed 21 out of 22 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tests/UserAgentTests.cs Switches empty request content stream to pooled recyclable stream
tests/Realtime/RealtimeTests.cs Uses pooled stream for concurrent audio-send test
tests/Images/ImagesMockTests.cs Replaces multiple MemoryStream allocations with recyclable streams
tests/Files/FilesMockTests.cs Uses recyclable streams for request-body capture and cancellation tests
tests/Chat/ChatTests.cs Uses recyclable streams for audio output buffering in streaming tests
tests/Batch/BatchTests.cs Uses recyclable streams for batch input file serialization
tests/Audio/TranslationMockTests.cs Uses recyclable stream for cancellation test input
tests/Audio/TranscriptionMockTests.cs Uses recyclable stream for cancellation test input
examples/Chat/Example08_ChatSerialization.cs Switches serialization buffer to recyclable stream and changes Utf8JsonWriter construction
OpenAI/src/Utility/RecyclableMemoryStreamManager.cs Introduces shared recyclable stream manager wrapper
OpenAI/src/OpenAI.csproj Adds package reference for recyclable streams
OpenAI/src/Generated/Internal/Utf8JsonBinaryContent.cs Uses recyclable stream for JSON binary content buffering
OpenAI/src/Custom/Realtime/Internal/WebsocketPipelineResponse.cs Uses recyclable stream for websocket response content buffering
OpenAI/src/Custom/Moderations/ModerationClient.cs Uses recyclable stream for moderation input JSON serialization
OpenAI/src/Custom/FineTuning/Internal/Pagination/FineTuningEventsPageToken.cs Uses recyclable stream for page token serialization
OpenAI/src/Custom/FineTuning/Internal/Pagination/FineTuningEventCollectionPageToken.cs Uses recyclable stream for page token serialization
OpenAI/src/Custom/FineTuning/Internal/Pagination/FineTuningCheckpointCollectionPageToken.cs Uses recyclable stream for page token serialization
OpenAI/src/Custom/FineTuning/Internal/FineTuningJobToken.cs Uses recyclable stream for token serialization
OpenAI/src/Custom/Embeddings/EmbeddingClient.cs Uses recyclable stream for embedding request JSON serialization
OpenAI/src/Custom/Batch/Internal/CreateBatchOperationToken.cs Uses recyclable stream for token serialization
OpenAI/src/Custom/Assistants/MessageCreationAttachment.cs Uses recyclable stream when serializing tool definitions
Directory.Packages.props Adds centrally managed version for recyclable stream package

Comment thread OpenAI/src/Generated/Internal/Utf8JsonBinaryContent.cs
internal partial class Utf8JsonBinaryContent : BinaryContent
{
private readonly MemoryStream _stream;
private readonly Microsoft.IO.RecyclableMemoryStream _stream;

@moljac moljac Jun 4, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well

https://github.com/microsoft/Microsoft.IO.RecyclableMemoryStream/blob/master/src/RecyclableMemoryStream.cs#L90

    public sealed class RecyclableMemoryStream : MemoryStream, IBufferWriter<byte>

Tests were passing (until I hit limit)

Comment thread OpenAI/src/Generated/Internal/Utf8JsonBinaryContent.cs Outdated
Comment on lines +21 to +23
_stream = MemoryStreamManager.Manager.GetStream();
_content = Create(_stream);
JsonWriter = new Utf8JsonWriter(_stream);
JsonWriter = new Utf8JsonWriter(_stream as IBufferWriter<byte>);

@moljac moljac Jun 4, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well

https://github.com/microsoft/Microsoft.IO.RecyclableMemoryStream/blob/master/src/RecyclableMemoryStream.cs#L90

    public sealed class RecyclableMemoryStream : MemoryStream, IBufferWriter<byte>

Tests were passing (until I hit limit)

Comment thread examples/Chat/Example08_ChatSerialization.cs Outdated
Comment on lines +32 to +33
using Microsoft.IO.RecyclableMemoryStream stream = Manager.GetStream();
using Utf8JsonWriter writer = new(stream as IBufferWriter<byte>);

@moljac moljac Jun 4, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well

https://github.com/microsoft/Microsoft.IO.RecyclableMemoryStream/blob/master/src/RecyclableMemoryStream.cs#L90

    public sealed class RecyclableMemoryStream : MemoryStream, IBufferWriter<byte>

Tests were passing (until I hit limit)

Comment thread OpenAI/src/Custom/Moderations/ModerationClient.cs Outdated
Comment on lines +242 to +243
using Microsoft.IO.RecyclableMemoryStream stream = OpenAI.MemoryStreamManager.Manager.GetStream();
using Utf8JsonWriter writer = new(stream as IBufferWriter<byte>);

@moljac moljac Jun 4, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well

https://github.com/microsoft/Microsoft.IO.RecyclableMemoryStream/blob/master/src/RecyclableMemoryStream.cs#L90

    public sealed class RecyclableMemoryStream : MemoryStream, IBufferWriter<byte>

Tests were passing (until I hit limit)

Comment thread OpenAI/src/Utility/RecyclableMemoryStreamManager.cs
Comment on lines +5 to +8
internal class MemoryStreamManager
{
public static readonly RecyclableMemoryStreamManager Manager = new ();
} No newline at end of file

@moljac moljac Jun 4, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well

https://github.com/microsoft/Microsoft.IO.RecyclableMemoryStream/blob/master/src/RecyclableMemoryStream.cs#L90

    public sealed class RecyclableMemoryStream : MemoryStream, IBufferWriter<byte>

Tests were passing (until I hit limit)

Comment thread OpenAI/src/Custom/Assistants/MessageCreationAttachment.cs
Comment on lines +40 to +41
using Microsoft.IO.RecyclableMemoryStream ms = OpenAI.MemoryStreamManager.Manager.GetStream();
using (var tempWriter = new Utf8JsonWriter(ms as System.IO.Stream))

@moljac moljac Jun 4, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well

https://github.com/microsoft/Microsoft.IO.RecyclableMemoryStream/blob/master/src/RecyclableMemoryStream.cs#L90

    public sealed class RecyclableMemoryStream : MemoryStream, IBufferWriter<byte>

Tests were passing (until I hit limit)

@jsquire jsquire Jun 6, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're not interested in the potential null outcome and an NRE would be confusing, let's please use a direct cast.

@moljac moljac Jun 20, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I will fix that.

@moljac

moljac commented Jun 4, 2026

Copy link
Copy Markdown
Author

I am hitting API_KEY limits with tests.

But some results

ChatTracingAndMetricsMultiple: Temporarily disabling for reliability.
NUnit Adapter 6.1.0.0: Test execution complete
  OpenAI.Tests test net10.0 failed with 2 error(s) (89,3s)
    ./tests/FineTuning/FineTuningClientTests.cs(133): error TESTERROR: 
      AllParameters (42ms): Error Message:   Assert.That(ft.Hyperparameters.LearningRateMultiplier, Is.EqualTo(3))
        Expected: 3
        But was:  30.0f
      
      Stack Trace:
         at OpenAI.Tests.FineTuning.FineTuningClientTests.AllParameters() in ./tests/FineTuning/FineTuningClientTests.cs:lin
      e 133
         at NUnit.Framework.Internal.Commands.TestMethodCommand.Execute(TestExecutionContext context)
         at NUnit.Framework.Internal.Commands.DelegatingTestCommand.RunTestMethodInThreadAbortSafeZone(TestExecutionContext context, Action action)
      
      1)    at OpenAI.Tests.FineTuning.FineTuningClientTests.AllParameters() in ./tests/FineTuning/FineTuningClientTests.cs:
      line 133
         at OpenAI.Tests.FineTuning.FineTuningClientTests.AllParameters()
    ./tests/FineTuning/FineTuningClientTests.cs(133): error TESTERROR: 
      AllParameters (21ms): Error Message:   Assert.That(ft.Hyperparameters.LearningRateMultiplier, Is.EqualTo(3))
        Expected: 3
        But was:  30.0f
      
      Stack Trace:
         at OpenAI.Tests.FineTuning.FineTuningClientTests.AllParameters() in ./tests/FineTuning/FineTuningClientTests.cs:lin
      e 133
         at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.BlockUntilCompleted()
         at NUnit.Framework.Internal.Commands.TestMethodCommand.Execute(TestExecutionContext context)
         at NUnit.Framework.Internal.Commands.DelegatingTestCommand.RunTestMethodInThreadAbortSafeZone(TestExecutionContext context, Action action)
      
      1)    at OpenAI.Tests.FineTuning.FineTuningClientTests.AllParameters() in ./tests/FineTuning/FineTuningClientTests.cs:
      line 133

[EVENT ID: event_Dn6w8yanQnNz5qVfpREeb]
Error: You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.

@moljac moljac requested a review from Copilot June 5, 2026 10:19
Copilot AI reviewed Jun 5, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 22 changed files in this pull request and generated 5 comments.

Comment thread OpenAI/src/Generated/Internal/Utf8JsonBinaryContent.cs
Comment on lines +16 to 24
private readonly Microsoft.IO.RecyclableMemoryStream _stream;
private readonly BinaryContent _content;

public Utf8JsonBinaryContent()
{
_stream = new MemoryStream();
_stream = MemoryStreamManager.Manager.GetStream();
_content = Create(_stream);
JsonWriter = new Utf8JsonWriter(_stream);
JsonWriter = new Utf8JsonWriter(_stream as System.IO.Stream);
}
Comment thread OpenAI/src/Custom/Realtime/Internal/WebsocketPipelineResponse.cs
Comment on lines 27 to 35
public override Stream ContentStream
{
get => _contentStream ??= new MemoryStream();
get => _contentStream ??= MemoryStreamManager.Manager.GetStream();
set => throw new NotImplementedException();
}
private MemoryStream _contentStream;
private Microsoft.IO.RecyclableMemoryStream _contentStream;

public override BinaryData Content => _content ??= new(_contentStream.ToArray());
private BinaryData _content;

@jsquire jsquire Jun 6, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe that Copilot is incorrect here, unfortunately. It looks like the WebsocketPipelineResponse is consumed only by AsyncWebsocketMessageEnumerator which does not currently expose a way to dispose, as it wraps the response in ClientResult<T> when handing back to callers via the enumerator.

ClientResult is not disposable and requires that callers hold responsibility for disposal of messages. While there is a disposal path in the pipeline for HTTP responses, this RealTime implementation appears to lack them.

Comment thread OpenAI/src/Utility/RecyclableMemoryStreamManager.cs
Comment on lines +5 to +8
internal class MemoryStreamManager
{
public static readonly RecyclableMemoryStreamManager Manager = new ();
} No newline at end of file
Comment thread examples/Chat/Example08_ChatSerialization.cs
Comment on lines +32 to +33
using Microsoft.IO.RecyclableMemoryStream stream = Manager.GetStream();
using Utf8JsonWriter writer = new(stream as System.IO.Stream);
Comment thread tests/UserAgentTests.cs
[Test]
public void UserAgentWithApplicationIdWorks() => UserAgentStringWorks(useApplicationId: true);

private static readonly Microsoft.IO.RecyclableMemoryStreamManager Manager = new ();
jsquire
jsquire requested changes Jun 6, 2026
View reviewed changes

@jsquire jsquire left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @moljac. Thank you for your contribution and interest in improving the OpenAI developer experience.

While we're open to the proposed approach, we are also very intentional about introducing additional dependencies. Though this is a Microsoft-owned library, it is also one that has not received a release in 2 years, and which has very low activity from maintainers. It also introduces a non-trivial amount of complexity and risk of leaks unless we're ensuring that disposal is honored.

I'd like to understand the impact of this change in real-world scenarios to understand both the performance and memory characteristics. If we see an impact significant enough to warrant the additional dependency and complexity, we'll evaluate next steps for moving forward.

Would you be open to providing BenchmarkDotNet results for the some of our sample scenarios in each area that you're updating?

Comment thread examples/Chat/Example08_ChatSerialization.cs
}
#endregion

private static readonly Microsoft.IO.RecyclableMemoryStreamManager Manager = new ();

@jsquire jsquire Jun 6, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll definitely want to preserve use of the standard MemoryStream in examples to reduce complexity.

Comment thread OpenAI/src/Custom/Assistants/MessageCreationAttachment.cs
{
using var ms = new System.IO.MemoryStream();
using (var tempWriter = new Utf8JsonWriter(ms))
using Microsoft.IO.RecyclableMemoryStream ms = OpenAI.MemoryStreamManager.Manager.GetStream();

@jsquire jsquire Jun 6, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
using Microsoft.IO.RecyclableMemoryStream ms = OpenAI.MemoryStreamManager.Manager.GetStream();
using var ms = OpenAI.MemoryStreamManager.Manager.GetStream();

nit: Please preserve the existing form.

Comment thread OpenAI/src/Custom/Assistants/MessageCreationAttachment.cs
Comment on lines +40 to +41
using Microsoft.IO.RecyclableMemoryStream ms = OpenAI.MemoryStreamManager.Manager.GetStream();
using (var tempWriter = new Utf8JsonWriter(ms as System.IO.Stream))

@jsquire jsquire Jun 6, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're not interested in the potential null outcome and an NRE would be confusing, let's please use a direct cast.

Comment thread OpenAI/src/Custom/Batch/Internal/CreateBatchOperationToken.cs
{
using MemoryStream stream = new();
using Utf8JsonWriter writer = new(stream);
using Microsoft.IO.RecyclableMemoryStream stream = OpenAI.MemoryStreamManager.Manager.GetStream();

@jsquire jsquire Jun 6, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
using Microsoft.IO.RecyclableMemoryStream stream = OpenAI.MemoryStreamManager.Manager.GetStream();
using var stream = OpenAI.MemoryStreamManager.Manager.GetStream();

Comment thread OpenAI/src/Custom/Batch/Internal/CreateBatchOperationToken.cs
using MemoryStream stream = new();
using Utf8JsonWriter writer = new(stream);
using Microsoft.IO.RecyclableMemoryStream stream = OpenAI.MemoryStreamManager.Manager.GetStream();
using Utf8JsonWriter writer = new(stream as System.IO.Stream);

@jsquire jsquire Jun 6, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
using Utf8JsonWriter writer = new(stream as System.IO.Stream);
using Utf8JsonWriter writer = new((MemoryStream)stream);

Comment thread OpenAI/src/Custom/Realtime/Internal/WebsocketPipelineResponse.cs
Comment on lines 27 to 35
public override Stream ContentStream
{
get => _contentStream ??= new MemoryStream();
get => _contentStream ??= MemoryStreamManager.Manager.GetStream();
set => throw new NotImplementedException();
}
private MemoryStream _contentStream;
private Microsoft.IO.RecyclableMemoryStream _contentStream;

public override BinaryData Content => _content ??= new(_contentStream.ToArray());
private BinaryData _content;

@jsquire jsquire Jun 6, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe that Copilot is incorrect here, unfortunately. It looks like the WebsocketPipelineResponse is consumed only by AsyncWebsocketMessageEnumerator which does not currently expose a way to dispose, as it wraps the response in ClientResult<T> when handing back to callers via the enumerator.

ClientResult is not disposable and requires that callers hold responsibility for disposal of messages. While there is a disposal path in the pipeline for HTTP responses, this RealTime implementation appears to lack them.

Comment thread OpenAI/src/Generated/Internal/Utf8JsonBinaryContent.cs
internal partial class Utf8JsonBinaryContent : BinaryContent
{
private readonly MemoryStream _stream;
private readonly Microsoft.IO.RecyclableMemoryStream _stream;

@jsquire jsquire Jun 6, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is generated code that cannot be modified by hand. As the generator is an externally owned construct, this is not a path that you'll be able to update.

Comment thread OpenAI/src/Utility/RecyclableMemoryStreamManager.cs

namespace OpenAI;

internal class MemoryStreamManager

@jsquire jsquire Jun 6, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
internal class MemoryStreamManager
internal static class MemoryStreamManager

@moljac

moljac commented Jun 20, 2026

Copy link
Copy Markdown
Author

@jsquire

Sorry for delay. I was dragged away with my daily tasks.

I will add comments.

@moljac

moljac commented Jun 20, 2026

Copy link
Copy Markdown
Author

Hi @jsquire

Thank you for your contribution and interest in improving the OpenAI developer experience.

Sure.

While we're open to the proposed approach, we are also very intentional about introducing additional dependencies. Though this is a Microsoft-owned library, it is also one that has not received a release in 2 years, and which has very low activity from maintainers. It also introduces a non-trivial amount of complexity and risk of leaks unless we're ensuring that disposal is honored.

I do understand your concern. I noticed that some of state of the art tools made by Microsoft use this library.

To name some:

  1. kiota - OpenAPI based HTTP Client code generator

    https://github.com/microsoft/kiota

    ... and this is how I decided to create this PR. I created kiota wrapper for OpenAI
    and saw opportunity to improve existing lib - in this case your library.

  2. dotnet/maui does use it partially

more info:

https://www.nuget.org/packages/Microsoft.IO.RecyclableMemoryStream#usedby-body-tab

I'd like to understand the impact of this change in real-world scenarios to understand both the performance and memory characteristics. If we see an impact significant enough to warrant the additional dependency and complexity, we'll evaluate next steps for moving forward.

M.IO.RMS reduces GC pressure by less Heap allocations. I was able to make few MAUI apps more performant and I have set of serialization/deserialization wrappers for several formats JSON, XML, CSV together with benchmarks and this library does help in quite a lot cases.

Would you be open to providing BenchmarkDotNet results for the some of our sample scenarios in each area that you're updating?

Sure. I will take a look.

Until then I will make PR WIP.

@moljac moljac marked this pull request as draft June 20, 2026 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments
@jsquire jsquire jsquire requested changes
@joseharriaga joseharriaga Awaiting requested review from joseharriaga joseharriaga is a code owner
@christothes christothes Awaiting requested review from christothes christothes is a code owner
@ShivangiReja ShivangiReja Awaiting requested review from ShivangiReja ShivangiReja is a code owner

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@moljac @jsquire