Skip to content

support rclone - #466

Open
happy-v587 wants to merge 1 commit into
Aiven-Open:mainfrom
happy-v587:rclone
Open

support rclone#466
happy-v587 wants to merge 1 commit into
Aiven-Open:mainfrom
happy-v587:rclone

Conversation

@happy-v587

@happy-v587 happy-v587 commented Aug 26, 2021

Copy link
Copy Markdown

By using rclone tool, we can support more storage.

https://rclone.org/

Just want to ask if this way is OK?
If it's OK, I'll sort out the documents and tests

@happy-v587
happy-v587 requested a review from a team August 26, 2021 07:25
@rdunklau

rdunklau commented Sep 9, 2021

Copy link
Copy Markdown
Contributor

Hello !

Thank you for contributing !

Out of curiosity, what is your use-case for rclone ? If it's a specific storage not already covered by PGHoard it might make sense to implement it directly.

As for direct rclone support, since it is a "meta-storage" so to speak, we don't feel like including it as-is.
What we could do instead would be to allow to specify a class name as the storage type, and then use this, making the storage layer extensible. You could just move your RCloneTransfer class to another package, and then use it from the config file: "storage_type": "mypackage.RCloneTransfer"

Would that suit your needs ?

@tobwen

tobwen commented Nov 14, 2021

Copy link
Copy Markdown

@happyuncle Thanks for this PR. Since it lacks of documentation, here's a simple example of how to use it:

{
  "backup_location": "/home/pghoard/backup/metadata",
  "backup_sites": {
    "example-site": {
      "nodes": [
        {
          "host": "127.0.0.1",
          "port": 5433,
          "user": "backup",
          "password": "secret",
          "application_name": "pghoard"
        }
      ],
      "object_storage": {
        "storage_type": "rclone",
        "remote_clone_config_path": "/home/pghoard/backup/rclone.conf",
        "source": "/home/pghoard/backup/rclone/",
        "destination": "onedrive-encrypted",
        "destination_path": "/backups/"
      },
      "pg_data_directory": "/var/lib/postgresql/14/main/"
    }
  }
}

Notes:

  • OneDrive is just cloud storage, it IS NOT a reliable backup target! Please use a real backup provider!
  • onedrive-encrypted is the target, which is defined in rclone.conf
  • the backup on the target located at: /backups/example-site/

@jankatins
jankatins removed the request for review from a team November 18, 2021 13:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants