Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] BlobOpenWriteOptions.BufferSize ignored in BlockBlobClient.OpenWriteAsync #47782

Open
rlrossiter opened this issue Jan 10, 2025 · 1 comment
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Attention Workflow: This issue is responsible by Azure service team. Storage Storage Service (Queues, Blobs, Files)

Comments

@rlrossiter
Copy link
Contributor

Library name and version

Azure.Storage.Blobs 12.23.0

Describe the bug

The documentation for [BlobOpenWriteOptions.BufferSize](https://github.com/Azure/azure-sdk-for-net/blob/710a44d062b41c3e8a59ba8676618e939d2ce0c2/sdk/storage/Azure.Storage.Blobs/src/Models/BlobOpenWriteOptions.cs#L18-L22) says the default buffer used is 4MB and the max is 4000MB. That bufferSize is passed to StorageWriteStream, where the actual buffer size is calculated to be [Math.Min(Constants.MB, bufferSize](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.Common/src/Shared/StorageWriteStream.cs#L96).

This is because PooledMemoryStream ends up using an ArrayPool to back the buffer, which has a max array size of type int (and Shared in reality has a max of 1024 * 1024) so the Math.Min makes sense. It should just be caught in ValidateBufferSize() rather than silently changing what was requested.

Expected behavior

    private const long OneMB = 1024 * 1024;
    private static readonly BlobOpenWriteOptions Options = new BlobOpenWriteOptions
    {
        BufferSize = 1024 * OneMB,
    };

    static void Main(string[] args)
    {
        string BlobUri = $"https://sasenderscaletest0.blob.core.windows.net/fmstest/testblob";
        BlobClient blobClient = new BlobClient(new Uri(BlobUri), credential: new DefaultAzureCredential());
        using var stream = blobClient.OpenWriteAsync(true, options: Options).Result;
    }

Expected:
stream._buffer.MaxArraySize == stream._bufferSize == 1073741824

Actual behavior

Image

stream._bufferSize == 1073741824
stream._buffer.MaxArraySize == 1048576

Reproduction Steps

internal class Program
{
    private const long OneMB = 1024 * 1024;
    private static readonly BlobOpenWriteOptions Options = new BlobOpenWriteOptions
    {
        BufferSize = 1024 * OneMB,
    };

    static void Main(string[] args)
    {
        string BlobUri = $"https://mysa.blob.core.windows.net/container/testblob";
        BlobClient blobClient = new BlobClient(new Uri(BlobUri), credential: new DefaultAzureCredential());
        using var stream = blobClient.OpenWriteAsync(true, options: Options).Result;
    }
}

Environment

No response

@github-actions github-actions bot added Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Attention Workflow: This issue is responsible by Azure service team. Storage Storage Service (Queues, Blobs, Files) labels Jan 10, 2025
Copy link

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @xgithubtriage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Attention Workflow: This issue is responsible by Azure service team. Storage Storage Service (Queues, Blobs, Files)
Projects
None yet
Development

No branches or pull requests

1 participant