Skip to content

gcp_cloud_storage sink: timezone option not applied to key_prefix strftime rendering #25090

@yh1215

Description

@yh1215

A note for the community

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

The timezone option in the gcp_cloud_storage sink does not affect strftime specifiers in key_prefix. The date-based directory partitioning always uses UTC regardless of the configured timezone.
The same timezone option works correctly in the aws_s3 sink.

Observed Behavior
With timezone = "Asia/Taipei" (UTC+8) configured on the gcp_cloud_storage sink:

The 20260401/ folder's earliest file was created at 08:00 AM local time (Asia/Taipei).
The 20260331/ folder's latest file was created at 07:59 AM local time (Asia/Taipei).
This indicates the folder date boundary is at 00:00 UTC (= 08:00 Asia/Taipei), meaning key_prefix strftime rendering is using UTC and ignoring the timezone setting.

Expected Behavior
The folder date boundary should be at 00:00 Asia/Taipei (= 16:00 UTC the previous day), consistent with the configured timezone = "Asia/Taipei".

The aws_s3 sink with the identical timezone and key_prefix configuration produces the correct date-based partitioning, confirming the expected behavior.

Configuration

# gcp_cloud_storage (NOT working)
[sinks.gcs_output]
type = "gcp_cloud_storage"
timezone = "Asia/Taipei"
inputs = ["filtered"]
bucket = "my-bucket"
key_prefix = "%Y%m%d/"
filename_append_uuid = true
filename_extension = "gz"
compression = "gzip"

#aws_s3 (working correctly)
[sinks.minio_output]
type = "aws_s3"
timezone = "Asia/Taipei"
inputs = ["filtered"]
bucket = "my-bucket"
key_prefix = "%Y%m%d/"
filename_append_uuid = true
filename_extension = "gz"
compression = "gzip"
endpoint = "https://minio.example.com"
region = ""

Version

0.54.0

Debug Output


Example Data

No response

Additional Context

  • Container image: timberio/vector:0.54.0-alpine
  • Architecture: x86_64
  • Platform: Kubernetes (GKE)

The Alpine-based image (timberio/vector:0.54.0-alpine) does include tzdata, as confirmed by the aws_s3 sink correctly handling Asia/Taipei.
This issue is specific to the gcp_cloud_storage sink; the aws_s3 sink honors the timezone setting for key_prefix as expected.

References

PR #18506 added configurable filename timezone for file, aws_s3, and gcp_cloud_storage sinks. The aws_s3 implementation appears to work, but the gcp_cloud_storage implementation does not apply the timezone to key_prefix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions