Skip to content

Commit ef24308

Browse files
docs(restore): add native restore docs
This commit adds native restore page similar to the native backup one. It also adjusts --method flag description so that it points to the newly created page. Fixes #4606
1 parent baef52b commit ef24308

5 files changed

Lines changed: 116 additions & 25 deletions

File tree

docs/source/restore/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ Restore
77
:maxdepth: 2
88

99
restore-tables
10+
native-restore
1011
restore-schema
1112
old-restore-schema
1213
examples
Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
==============
2+
Native Restore
3+
==============
4+
5+
Native restore is an optimization similar to :doc:`native backup <../backup/native-backup>`.
6+
7+
ScyllaDB Manager uses ScyllaDB Manager Agents deployed on each ScyllaDB node to coordinate restore.
8+
These agents serve as a proxy to `ScyllaDB REST API <https://docs.scylladb.com/manual/stable/operating-scylla/rest.html>`_ and also act as `rclone <https://github.com/scylladb/rclone>`_ servers responsible for communication between the node and backup location.
9+
10+
Since rclone server is separate from ScyllaDB internal schedulers, yet they both live on the same machine,
11+
it is allocated with limited resources not to interfere with ScyllaDB performance. Moreover, having separate
12+
processes responsible for fetching files from backup location and streaming those files into the cluster results
13+
in the need of additional costly synchronization.
14+
15+
This solution results in:
16+
17+
* Saving files downloaded from backup location on disk before streaming them into the cluster (unnecessary disk writes before streaming)
18+
* Inefficient resource utilization (file download handled by rclone server while file streaming handled by ScyllaDB)
19+
* Longer restore duration (limited resources allocated to rclone server)
20+
21+
Native restore aims to solve these problems by moving restore responsibilities from rclone server into ScyllaDB itself.
22+
Just like with native backup, both native and rclone restores performed by ScyllaDB Manager rely on the same :doc:`backup specification </backup/specification>`,
23+
so any backup (native or rclone) can be restored using any restore method (assuming it meets given restore method limitations).
24+
In the `Status`_ section you can find the parts of restore procedure already moved to ScyllaDB.
25+
26+
Status
27+
======
28+
29+
This section contains the list of stages in the restore procedure now managed by ScyllaDB.
30+
All other stages are still performed by the rclone server.
31+
The ``ScyllaDB Version`` column describes the ScyllaDB version from which the functionality is considered production ready,
32+
even though the functionality might be available in earlier versions as well.
33+
34+
.. list-table::
35+
:widths: 25 15 50 25
36+
:header-rows: 1
37+
38+
* - Functionality
39+
- ScyllaDB Version
40+
- Description
41+
- Limitations
42+
* - SSTable streaming directly from :doc:`s3 </backup/setup-s3-compatible-storage>`
43+
- 2026.1
44+
- As this is the most time- and resource-consuming part of the restore procedure, moving it to ScyllaDB brings the most benefits.
45+
It also allows for not saving downloaded SSTables on disk before streaming them into the cluster.
46+
Unlike native backup, when performing restore on a cluster which doesn't currently serve user traffic,
47+
it's best not to throttle native restore with
48+
`stream_io_throughput_mb_per_sec <https://docs.scylladb.com/manual/stable/reference/configuration-parameters.html#confval-stream_io_throughput_mb_per_sec>`_ in `scylla.yaml`
49+
to obtain the best performance.
50+
- Does not support restoration of :ref:`versioned SSTables <backup-versioned-sstables>`.
51+
Does not support restoration of SSTables with integer based IDs.
52+
* - SSTable streaming directly from :doc:`gcs </backup/setup-gcs>`
53+
- 2026.1
54+
- Same as above.
55+
- Same as above.
56+
57+
Configuration
58+
=============
59+
60+
Native restore requires the same ScyllaDB and ScyllaDB Manager Agent configuration as native backup.
61+
Follow the steps described in :ref:`configure-native-backup-in-scylla` to configure each ScyllaDB node.
62+
Throttling `stream_io_throughput_mb_per_sec <https://docs.scylladb.com/manual/stable/reference/configuration-parameters.html#confval-stream_io_throughput_mb_per_sec>`_
63+
is not recommended, as explained in `Status`_ section.
64+
65+
Usage
66+
=====
67+
68+
The native restore usage is controlled with the :ref:`sctool restore --method <sctool-restore>` flag.
69+
It supports three values: ``rclone`` (default), ``native``, and ``auto``:
70+
71+
* ``native``: Uses all native restore functionalities listed in the `Status`_ section.
72+
Use this value for native restore configuration validation and testing.
73+
Note that this will fail when:
74+
75+
* ScyllaDB is not configured properly (see ``object_storage_endpoints`` in `Configuration`_)
76+
* Provider not supported by used ScyllaDB version is used (see limitations in `Status`_)
77+
* Restored backup contains versioned SSTables (see limitations in `Status`_)
78+
* Restored backup contains SSTables with integer based IDs (see limitations in `Status`_)
79+
80+
* ``auto``: Uses native restore functionalities when possible, otherwise falls back to rclone restore.
81+
Use this value for production restores. It will use native restore functionality only when it is
82+
considered production ready (see version support in `Status`_). The fallback works on per restored batch basis,
83+
so it allows for utilizing native restore functionalities for most restored batches,
84+
even if a small subset of them contains SSTables not compatible with native restore.
85+
86+
* ``rclone``: Uses rclone restore functionalities only. This effectively disables all native restore functionalities.
87+
88+
Note that :ref:`sctool restore --rate-limit --transfers --unpin-agent-cpu <sctool-restore>` flags do not take effect when using native restore,
89+
as the streaming performance is controlled directly by ScyllaDB itself. To control streaming performance on the ScyllaDB side,
90+
configure `stream_io_throughput_mb_per_sec <https://docs.scylladb.com/manual/stable/reference/configuration-parameters.html#confval-stream_io_throughput_mb_per_sec>`_ in `scylla.yaml`.
91+
92+
You can :ref:`create restore task <sctool-restore>` with the desired method:
93+
94+
.. code-block:: none
95+
96+
sctool restore -c <cluster ID> -L <backup location> --snapshot-tag <tag> --method native

docs/source/sctool/partials/sctool_restore.yaml

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -101,16 +101,14 @@ options:
101101
- name: method
102102
default_value: rclone
103103
usage: |
104-
Specify the API used for restoring files:
105-
- 'auto': Use the native API when possible, otherwise use the Rclone API.
106-
- 'native': Scylla server streams directly from backup location (supports only S3 provider).
107-
- 'rclone': Scylla Manager Agent downloads from backup location and calls Scylla server for streaming afterwards.
104+
Control native restore method (See https://manager.docs.scylladb.com/stable/restore/native-restore):
108105
109-
Both methods require configuring the Scylla Manager Agent in 'scylla-manager-agent.yaml'.
110-
The native API also requires 'object_storage_endpoints' to be configured in 'scylla.yaml' (See https://docs.scylladb.com/manual/stable/operating-scylla/admin.html#object-storage-configuration).
106+
* 'auto': Use the supported native restore functionalities, otherwise use rclone restore.
107+
* 'native': Use the native restore functionalities only.
108+
* 'rclone': Use the rclone restore functionalities only.
111109
112-
'--rate-limit', '--transfers', '--unpin-agent-cpu' flags do not take effect when using '--method=native',
113-
as the streaming performance is controlled by the Scylla server.
110+
Using native restore functionalities requires additional configuration and has its own limitations,
111+
which are described in detail in https://manager.docs.scylladb.com/stable/restore/native-restore.
114112
- name: name
115113
usage: |
116114
Task name that can be used instead of ID.

docs/source/sctool/partials/sctool_restore_update.yaml

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -99,16 +99,14 @@ options:
9999
- name: method
100100
default_value: rclone
101101
usage: |
102-
Specify the API used for restoring files:
103-
- 'auto': Use the native API when possible, otherwise use the Rclone API.
104-
- 'native': Scylla server streams directly from backup location (supports only S3 provider).
105-
- 'rclone': Scylla Manager Agent downloads from backup location and calls Scylla server for streaming afterwards.
102+
Control native restore method (See https://manager.docs.scylladb.com/stable/restore/native-restore):
106103
107-
Both methods require configuring the Scylla Manager Agent in 'scylla-manager-agent.yaml'.
108-
The native API also requires 'object_storage_endpoints' to be configured in 'scylla.yaml' (See https://docs.scylladb.com/manual/stable/operating-scylla/admin.html#object-storage-configuration).
104+
* 'auto': Use the supported native restore functionalities, otherwise use rclone restore.
105+
* 'native': Use the native restore functionalities only.
106+
* 'rclone': Use the rclone restore functionalities only.
109107
110-
'--rate-limit', '--transfers', '--unpin-agent-cpu' flags do not take effect when using '--method=native',
111-
as the streaming performance is controlled by the Scylla server.
108+
Using native restore functionalities requires additional configuration and has its own limitations,
109+
which are described in detail in https://manager.docs.scylladb.com/stable/restore/native-restore.
112110
- name: name
113111
usage: |
114112
Task name that can be used instead of ID.

pkg/command/restore/res.yaml

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -86,13 +86,11 @@ dc-mapping: |
8686
Note: Only DCs that are provided in mappings will be restored.
8787
8888
method: |
89-
Specify the API used for restoring files:
90-
- 'auto': Use the native API when possible, otherwise use the Rclone API.
91-
- 'native': Scylla server streams directly from backup location (supports only S3 provider).
92-
- 'rclone': Scylla Manager Agent downloads from backup location and calls Scylla server for streaming afterwards.
93-
94-
Both methods require configuring the Scylla Manager Agent in 'scylla-manager-agent.yaml'.
95-
The native API also requires 'object_storage_endpoints' to be configured in 'scylla.yaml' (See https://docs.scylladb.com/manual/stable/operating-scylla/admin.html#object-storage-configuration).
89+
Control native restore method (See https://manager.docs.scylladb.com/stable/restore/native-restore):
90+
91+
* 'auto': Use the supported native restore functionalities, otherwise use rclone restore.
92+
* 'native': Use the native restore functionalities only.
93+
* 'rclone': Use the rclone restore functionalities only.
9694
97-
'--rate-limit', '--transfers', '--unpin-agent-cpu' flags do not take effect when using '--method=native',
98-
as the streaming performance is controlled by the Scylla server.
95+
Using native restore functionalities requires additional configuration and has its own limitations,
96+
which are described in detail in https://manager.docs.scylladb.com/stable/restore/native-restore.

0 commit comments

Comments
 (0)