SNOW-2900795: Make MAX_BUFFER_SIZE in SnowflakeFileTranferAgent Configurable

[XIIKJIIX]

Overview

SNOW-2900795: Github Issue #2443

Pre-review self checklist

• PR branch is updated with all the changes from master branch
• The code is correctly formatted (run mvn -P check-style validate)
• New public API is not unnecessary exposed (run mvn verify and inspect target/japicmp/japicmp.html)
• The pull request name is prefixed with SNOW-XXXX:
• Code is in compliance with internal logging requirements

External contributors - please answer these questions before submitting a pull request. Thanks!

1. What GitHub issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

   Issue: SNOW-2900795: Make MAX_BUFFER_SIZE in SnowflakeFileTranferAgent Configurable #2443

2. Fill out the following pre-review checklist:

   • I am adding a new automated test(s) to verify correctness of my new code
   • I am adding new logging messages
   • I am modifying authorization mechanisms
   • I am adding new credentials
   • I am modifying OCSP code
   • I am adding a new dependency or upgrading an existing one
   • I am adding new public/protected component not marked with @SnowflakeJdbcInternalApi (note that public/protected methods/fields in classes marked with this annotation are already internal)

3. Please describe how your code solves the related issue.

This change improves memory management during data uploads by allowing users to configure the buffer threshold, enhancing performance and stability.

• Added a new property fileBackedBufferThreshold to UploadStreamConfig to control the threshold in bytes at which the internal buffer switches from heap memory to a temporary file on disk during compression and digest computation.
• Updated the builder pattern to include methods for setting and getting the buffer threshold, with validation for positive values.
• Modified SnowflakeConnectionImpl and SFBaseFileTransferAgent to utilize the new buffer threshold in upload operations.
• Introduced unit tests for UploadStreamConfig and SnowflakeFileTransferConfig to validate default and custom buffer threshold behavior, including edge cases for invalid values.
• Updated relevant documentation and comments to reflect the new functionality.

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[XIIKJIIX]

I have read the CLA Document and I hereby sign the CLA