Skip to content

Add Filesystem DataStore implementation for local storage #5891

Description

@leighmcculloch

What problem does your feature solve?

The datastore package currently only supports cloud storage backends (GCS and S3). There is no option to store data on the local filesystem.

This is a blocker for adding galexie meta lake support to quickstart (stellar/quickstart#835) and using quickstart for integration testing fork testing in the soroban-sdk (stellar/rs-soroban-sdk#1657).

Quickstart needs to publish a meta lake locally without requiring cloud storage configuration, similar to how it currently serves the local history archive.

What would you like to see?

A new FilesystemDataStore implementation that satisfies the DataStore interface, with the following capabilities:

  • Configuration via destination_path parameter specifying the root directory
  • Optional metadata storage in sidecar .metadata.json files (configurable via write_metadata param, defaulting to enabled)

Example configuration:

type = "Filesystem"
[params]
destination_path = "/var/lib/stellar/datastore"
write_metadata = "true"  # optional, defaults to true

What alternatives are there?

  • Run a local S3-compatible service (e.g., MinIO) alongside quickstart - adds complexity, resource overhead, and licensing concerns (MinIO uses AGPL)

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions