Skip to content
This repository was archived by the owner on Aug 19, 2026. It is now read-only.

PREINSTALL.md

Latest commit

聽

History

History
72 lines (57 loc) 路 5.93 KB

PREINSTALL.md

File metadata and controls

72 lines (57 loc) 路 5.93 KB
Raw

Use this extension to store the locations and metadata for Cloud Storage Objects in your Firestore database in real-time.

When you Create, Update, Delete, or Archive an Object in your configured Cloud Storage Bucket, this extension will update the configured Firestore location to maintain a mirror that reflects the contents of Cloud Storage in real-time.

Clients can subscribe to this Mirror in Firestore to get real-time updates as Objects are modified in your Bucket. Firestore Queries can be performed using metadata fields that are copied over in order to filter for specific Objects.

For every Object or Folder in Cloud Storage, there should be one Document created in Firestore with the Object's metadata stored as Document fields.

Things you should know

  • The mirroring process only happens when new changes in Cloud Storage trigger the extension. This means that the extension WILL NOT back-fill any data that existed before it was installed or if there are any service outages. In order to back-fill/repair your Firestore mirror, please use the back-fill tool provided in the stress-test directory (npm run-script start -- backfill)
  • Please note that this Mirror is one-way (changes made in Firestore will not show up in Cloud Storage), and the contents should be treated as read-only. This is because the entire Document is replaced during an update and any changes will be overwritten. Also note that any Documents where the extension fields have been removed will be treated as non-existent.
  • Firestore has some Document naming restrictions. Because this extension will mirror the path of the Cloud Storage Object, the generated Firestore path may be longer than the maximum subcollection depth of 100 and so there is a limit on the number of delimiters in the name of Objects you intend to mirror. There is also a limit of 1 megabyte for Document size, this will not be a problem for regular Objects with no custom metadata but keep it in mind if you are intending to mirror custom metadata.
  • When Objects are deleted from Cloud Storage, the mirrored equivalent in Firestore will also be deleted. However, since the extension needs to keep track of the last time it updated the Document, it will create a "Tombstone" document that can be found in the configured subcollection, this Tombstone Document contains only the lastEvent field used by the extension to keep track of it's most recent update.
  • The extension will maintain all the active Prefixes by creating and deleting Prefix Documents representing paths that contain Objects within Cloud Storage. In effect, this means file-system-like traversal is possible where each Prefix Document acts like a directory and has nested sub-directories both of which may or may not contain files. When creating or updating Objects, the relevant Prefix Documents will be created, and when deleting or archiving Objects, the relevant Prefix Documents will be checked for deletion.
  • Although we describe the extension as being real-time, due to limitations related to Cloud Functions there may be a noticeable delay between updates to Cloud Storage and these updates showing up in Firestore depending on what your use-case looks like. Due to the cold-start time for Cloud Functions, there may be a delay when running a Cloud Storage operation after a long period of inactivity due to the function having to initialize. Because there is a limit to the number of concurrent running invocations of a function, there may be a delay when there is high load causing many function invocations to happen concurrently.
  • Note that the extension will mirror all metadata fields by default. This includes the custom metadata field for download tokens, which give read access. This means anyone with read access to a Firestore document will have read access to the GCS object it represents. You can disable this by setting the Custom Metadata Filter to ^(?!firebaseStorageDownloadTokens).*.
  • Although Cloud Storage stores Object metadata as string fields, this extension converts applicable fields to Number or Timestamp Firestore types like size or updated in order to enable querying.

Document Data

Documents created representing Objects will have the following fields:

  • A lastEvent field used by the extension to determine when the Document was last changed by the extension.
  • A gcsMetadata field that contains the metadata stored in Cloud Storage.
    • The following fields are converted to Number in Firestore: size, generation, metageneration
    • The following fields are converted to Timestamp in Firestore: timeCreated, updated, timeStorageClassUpdated

Documents created representing Prefixes will have the following fields:

  • A lastEvent field like above.
  • A childRef field that is used by the extension to determine whether the Prefix Document should be checked for deletion.

Documents created representing Tombstones for deleted Objects or Prefixes that have become empty will have the following fields:

  • A lastEvent field like above.

Additional Setup

Billing

This extension uses other Firebase or Google Cloud Platform services which may have associated charges:

  • Cloud Firestore
  • Cloud Storage
  • Cloud Functions

When you use Firebase Extensions, you're only charged for the underlying resources that you use. A paid-tier billing plan is only required if the extension uses a service that requires a paid-tier plan, for example calling to a Google Cloud Platform API or making outbound network requests to non-Google services. All Firebase services offer a free tier of usage. Learn more about Firebase billing..