Context
Follow-up from PR #171 review comment: #171 (comment)
PR #171 added S3 / object-store support to the documents connector (source path and image_store can be s3:// prefixes) and validated it with a live e2e run against real S3 (us-east-1). The object_store crate backing this also supports Azure Blob Storage and GCS, so the same code path should work against those providers — but it hasn't been exercised live.
Ask
Run the same live e2e coverage PR #171 did on S3, but against:
- Azure Blob Storage — we have Azure credits.
- GCP Cloud Storage — only if we already have GCP credits; otherwise skip/defer.
Scope
Mirror the S3 e2e checklist from the PR against each provider:
- Nested corpus scan with glob + prefix scoping (sibling-prefix decoy and non-matching extensions excluded).
image_store page-render / crop uploads land as real objects with the correct Content-Type, and the write-probe object is cleaned up.
- Self-ingestion guard holds when the
image_store prefix sits inside the source path.
- Negative startup cases fail with actionable errors: cross-container/cross-bucket
image_store, nonexistent container/bucket, credentials smuggled into options, missing region/credentials in env, and a write preflight failure on a read-only target.
Notes / open questions
- Confirm the env-only credential contract and region/endpoint config translate cleanly to Azure (
AZURE_STORAGE_ACCOUNT / AZURE_STORAGE_ACCESS_KEY etc.) and GCS (GOOGLE_SERVICE_ACCOUNT / ADC), and update docs/documents.md if the per-provider setup differs.
- The same-bucket constraint may need a per-provider analog (same container / same bucket).
Context
Follow-up from PR #171 review comment: #171 (comment)
PR #171 added S3 / object-store support to the
documentsconnector (sourcepathandimage_storecan bes3://prefixes) and validated it with a live e2e run against real S3 (us-east-1). Theobject_storecrate backing this also supports Azure Blob Storage and GCS, so the same code path should work against those providers — but it hasn't been exercised live.Ask
Run the same live e2e coverage PR #171 did on S3, but against:
Scope
Mirror the S3 e2e checklist from the PR against each provider:
image_storepage-render / crop uploads land as real objects with the correctContent-Type, and the write-probe object is cleaned up.image_storeprefix sits inside the sourcepath.image_store, nonexistent container/bucket, credentials smuggled intooptions, missing region/credentials in env, and a write preflight failure on a read-only target.Notes / open questions
AZURE_STORAGE_ACCOUNT/AZURE_STORAGE_ACCESS_KEYetc.) and GCS (GOOGLE_SERVICE_ACCOUNT/ ADC), and updatedocs/documents.mdif the per-provider setup differs.