|
| 1 | +============== |
| 2 | +Native Restore |
| 3 | +============== |
| 4 | + |
| 5 | +Native restore is an optimization similar to :doc:`native backup <../backup/native-backup>`. |
| 6 | + |
| 7 | +ScyllaDB Manager uses ScyllaDB Manager Agents deployed on each ScyllaDB node to coordinate restore. |
| 8 | +These agents serve as a proxy to `ScyllaDB REST API <https://docs.scylladb.com/manual/stable/operating-scylla/rest.html>`_ and also act as `rclone <https://github.com/scylladb/rclone>`_ servers responsible for communication between the node and backup location. |
| 9 | + |
| 10 | +Since rclone server is separate from ScyllaDB internal schedulers, yet they both live on the same machine, |
| 11 | +it is allocated with limited resources not to interfere with ScyllaDB performance. Moreover, having separate |
| 12 | +processes responsible for fetching files from backup location and streaming those files into the cluster results |
| 13 | +in the need of additional costly synchronization. |
| 14 | + |
| 15 | +This solution results in: |
| 16 | + |
| 17 | +* Saving files downloaded from backup location on disk before streaming them into the cluster (unnecessary disk writes before streaming) |
| 18 | +* Inefficient resource utilization (file download handled by rclone server while file streaming handled by ScyllaDB) |
| 19 | +* Longer restore duration (limited resources allocated to rclone server) |
| 20 | + |
| 21 | +Native restore aims to solve these problems by moving restore responsibilities from rclone server into ScyllaDB itself. |
| 22 | +Just like with native backup, both native and rclone restores performed by ScyllaDB Manager rely on the same :doc:`backup specification </backup/specification>`, |
| 23 | +so any backup (native or rclone) can be restored using any restore method (assuming it meets given restore method limitations). |
| 24 | +In the `Status`_ section you can find the parts of restore procedure already moved to ScyllaDB. |
| 25 | + |
| 26 | +Status |
| 27 | +====== |
| 28 | + |
| 29 | +This section contains the list of stages in the restore procedure now managed by ScyllaDB. |
| 30 | +All other stages are still performed by the rclone server. |
| 31 | +The ``ScyllaDB Version`` column describes the ScyllaDB version from which the functionality is considered production ready, |
| 32 | +even though the functionality might be available in earlier versions as well. |
| 33 | + |
| 34 | +.. list-table:: |
| 35 | + :widths: 25 15 50 25 |
| 36 | + :header-rows: 1 |
| 37 | + |
| 38 | + * - Functionality |
| 39 | + - ScyllaDB Version |
| 40 | + - Description |
| 41 | + - Limitations |
| 42 | + * - SSTable streaming directly from :doc:`s3 </backup/setup-s3-compatible-storage>` |
| 43 | + - 2026.1 |
| 44 | + - As this is the most time- and resource-consuming part of the restore procedure, moving it to ScyllaDB brings the most benefits. |
| 45 | + It also allows for not saving downloaded SSTables on disk before streaming them into the cluster. |
| 46 | + Unlike native backup, when performing restore on a cluster which doesn't currently serve user traffic, |
| 47 | + it's best not to throttle native restore with |
| 48 | + `stream_io_throughput_mb_per_sec <https://docs.scylladb.com/manual/stable/reference/configuration-parameters.html#confval-stream_io_throughput_mb_per_sec>`_ in `scylla.yaml` |
| 49 | + to obtain the best performance. |
| 50 | + - Does not support restoration of :ref:`versioned SSTables <backup-versioned-sstables>`. |
| 51 | + Does not support restoration of SSTables with integer based IDs. |
| 52 | + * - SSTable streaming directly from :doc:`gcs </backup/setup-gcs>` |
| 53 | + - 2026.1 |
| 54 | + - Same as above. |
| 55 | + - Same as above. |
| 56 | + |
| 57 | +Configuration |
| 58 | +============= |
| 59 | + |
| 60 | +Native restore requires the same ScyllaDB and ScyllaDB Manager Agent configuration as native backup. |
| 61 | +Follow the steps described in :ref:`configure-native-backup-in-scylla` to configure each ScyllaDB node. |
| 62 | +Throttling `stream_io_throughput_mb_per_sec <https://docs.scylladb.com/manual/stable/reference/configuration-parameters.html#confval-stream_io_throughput_mb_per_sec>`_ |
| 63 | +is not recommended, as explained in `Status`_ section. |
| 64 | + |
| 65 | +Usage |
| 66 | +===== |
| 67 | + |
| 68 | +The native restore usage is controlled with the :ref:`sctool restore --method <sctool-restore>` flag. |
| 69 | +It supports three values: ``rclone`` (default), ``native``, and ``auto``: |
| 70 | + |
| 71 | + * ``native``: Uses all native restore functionalities listed in the `Status`_ section. |
| 72 | + Use this value for native restore configuration validation and testing. |
| 73 | + Note that this will fail when: |
| 74 | + |
| 75 | + * ScyllaDB is not configured properly (see ``object_storage_endpoints`` in `Configuration`_) |
| 76 | + * Provider not supported by used ScyllaDB version is used (see limitations in `Status`_) |
| 77 | + * Restored backup contains versioned SSTables (see limitations in `Status`_) |
| 78 | + * Restored backup contains SSTables with integer based IDs (see limitations in `Status`_) |
| 79 | + |
| 80 | + * ``auto``: Uses native restore functionalities when possible, otherwise falls back to rclone restore. |
| 81 | + Use this value for production restores. It will use native restore functionality only when it is |
| 82 | + considered production ready (see version support in `Status`_). The fallback works on per restored batch basis, |
| 83 | + so it allows for utilizing native restore functionalities for most restored batches, |
| 84 | + even if a small subset of them contains SSTables not compatible with native restore. |
| 85 | + |
| 86 | + * ``rclone``: Uses rclone restore functionalities only. This effectively disables all native restore functionalities. |
| 87 | + |
| 88 | +Note that :ref:`sctool restore --rate-limit --transfers --unpin-agent-cpu <sctool-restore>` flags do not take effect when using native restore, |
| 89 | +as the streaming performance is controlled directly by ScyllaDB itself. To control streaming performance on the ScyllaDB side, |
| 90 | +configure `stream_io_throughput_mb_per_sec <https://docs.scylladb.com/manual/stable/reference/configuration-parameters.html#confval-stream_io_throughput_mb_per_sec>`_ in `scylla.yaml`. |
| 91 | + |
| 92 | +You can :ref:`create restore task <sctool-restore>` with the desired method: |
| 93 | + |
| 94 | +.. code-block:: none |
| 95 | +
|
| 96 | + sctool restore -c <cluster ID> -L <backup location> --snapshot-tag <tag> --method native |
0 commit comments