Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
429da3a
Add comprehensive installation instructions
RulerOf Mar 26, 2018
0f669a3
Cleaned up Configuration section to match the rest of the document
RulerOf Mar 26, 2018
9b2c2f2
don't use hardcoded paths
phreaker0 Sep 12, 2018
265b32b
Merge branch 'master' into path
phreaker0 Dec 7, 2018
be50481
use more portable/POSIX compatible ways to check for command existence
phreaker0 Dec 7, 2018
941a770
updated comment regarding command existence check
phreaker0 Dec 7, 2018
89f1d4e
run compress commands bare too
phreaker0 Dec 8, 2018
e4e9065
removed debug output from excludes loop
phreaker0 Jan 7, 2019
271ede7
Revert "Revert "Zfs Recursion""
phreaker0 Jan 7, 2019
0fc16b4
fixed uninitialzed variable warning and clarified recursive value
phreaker0 Jan 7, 2019
ab12540
added message if zfs recursion is used and fixed ordinary recursion i…
phreaker0 Jan 7, 2019
6f5e846
Merge branch 'master' into silence-excludes
phreaker0 Jan 25, 2019
02219b0
Merge branch 'master' into path
phreaker0 Jan 25, 2019
d692972
Merge branch 'master' into zfs-recursion
phreaker0 Jan 25, 2019
85561dd
handle the case of mistyped dataset name nicely
phreaker0 Jan 25, 2019
132c765
error out if no datasets are processed with recursive arg
phreaker0 Jan 25, 2019
c5eebbe
implemented parsing of provided zfs send/recv options and whitelistin…
phreaker0 Jan 31, 2019
10b2012
removed debug output
phreaker0 Feb 4, 2019
5812b57
check for invalid send/receive option syntax
phreaker0 Feb 4, 2019
d3f19c8
Sanoid ebuild for Gentoo
thehaven Feb 17, 2019
234cc61
Merge pull request #351 from thehaven/master
jimsalterjrs Feb 17, 2019
c543f9a
use ordinary replication if clone recreation fails
phreaker0 Feb 19, 2019
25c5e2e
allow new zfs send/recv hold parameter (introduced in recent master) …
phreaker0 Feb 19, 2019
eefc659
update the install instruction with recent changes
phreaker0 Feb 20, 2019
d16ec24
Merge remote-tracking branch 'upstream/master' into RulerOf-master
phreaker0 Feb 20, 2019
1a374a3
Merge branch 'RulerOf-master'
phreaker0 Feb 20, 2019
e014fd7
check if ssh connection works
phreaker0 Feb 20, 2019
a97da7d
Added bwlimit units
klemens-u Feb 28, 2019
e0e7b0d
added bwlimit units
klemens-u Feb 28, 2019
736d031
initial replication without intermediate snapshots should use the cre…
phreaker0 Mar 14, 2019
df1e344
Fix gentoo package snafu - need a sanoid directory to store the files
thehaven Mar 29, 2019
ffa6ad3
Fix gentoo package snafu - need a sanoid directory to store the files
thehaven Mar 29, 2019
5a49358
Merge pull request #363 from thehaven/master
jimsalterjrs Mar 29, 2019
681aa61
clarify bandwidth limit units
jimsalterjrs Mar 29, 2019
77e1520
Merge pull request #357 from klemens-u/patch-2
jimsalterjrs Mar 29, 2019
56d7c59
clarify --bwlimit units
jimsalterjrs Mar 29, 2019
f163df2
Merge pull request #356 from klemens-u/patch-1
jimsalterjrs Mar 29, 2019
e86ac01
Merge pull request #360 from phreaker0/fix-initial-no-stream
jimsalterjrs Mar 29, 2019
7900e9f
Merge pull request #353 from phreaker0/check-ssh
jimsalterjrs Mar 29, 2019
5e0617f
Merge pull request #348 from phreaker0/fix-unitialized-variable
jimsalterjrs Mar 29, 2019
38ca1ef
Merge pull request #340 from phreaker0/fix-listing
jimsalterjrs Mar 29, 2019
189f933
Merge pull request #335 from phreaker0/catch-some-errors
jimsalterjrs Mar 29, 2019
4783470
Merge pull request #206 from RulerOf/master
jimsalterjrs Mar 29, 2019
68cc59a
Update INSTALL.md
jimsalterjrs Mar 29, 2019
31e46da
Merge pull request #343 from phreaker0/whitelist-send-recv-options
jimsalterjrs Mar 29, 2019
fd28239
Merge pull request #319 from phreaker0/silence-excludes
jimsalterjrs Mar 29, 2019
11b3af6
Merge pull request #334 from phreaker0/check-pool-resume
jimsalterjrs Mar 29, 2019
1219818
Merge pull request #352 from phreaker0/clone-fallback
jimsalterjrs Mar 29, 2019
e376ead
Merge pull request #342 from phreaker0/create-bookmark
jimsalterjrs Mar 29, 2019
c162e63
Merge pull request #320 from phreaker0/zfs-recursion
jimsalterjrs Mar 29, 2019
17e15c2
Fix systemd service definitions.
danielewood Mar 29, 2019
afb33e0
Merge pull request #365 from danielewood/patch-1
jimsalterjrs Mar 29, 2019
90b8c92
sanoid-prune.service is indeed referenced and run after the sanoid.se…
phreaker0 Mar 31, 2019
1900eca
added verbose cmd line flags for journald logging
phreaker0 Mar 31, 2019
111d93a
Merge branch 'master' into fix-bookmark-edge-cases
phreaker0 Mar 31, 2019
8d0efe2
Merge pull request #367 from phreaker0/revert
jimsalterjrs Mar 31, 2019
be1a1f5
Merge pull request #341 from phreaker0/fix-bookmark-edge-cases
jimsalterjrs Mar 31, 2019
a78faed
Merge branch 'master' into path
phreaker0 Mar 31, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 0 additions & 33 deletions INSTALL

This file was deleted.

172 changes: 172 additions & 0 deletions INSTALL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
# Installation

**Sanoid** and **Syncoid** are complementary but separate pieces of software. To install and configure them, follow the guide below for your operating system. Everything in `code blocks` should be copy-pasteable. If your OS isn't listed, a set of general instructions is at the end of the list and you can perform the process manually.

<!-- TOC depthFrom:1 depthTo:6 withLinks:1 updateOnSave:0 orderedList:0 -->

- [Installation](#installation)
- [Debian/Ubuntu](#debianubuntu)
- [CentOS](#centos)
- [FreeBSD](#freebsd)
- [Other OSes](#other-oses)
- [Configuration](#configuration)
- [Sanoid](#sanoid)

<!-- /TOC -->


## Debian/Ubuntu

Install prerequisite software:

```bash
apt install libconfig-inifiles-perl pv lzop mbuffer
```

Clone this repo, build the debian package and install it (alternatively you can skip the package and do it manually like described below for CentOS):

```bash
# Download the repo as root to avoid changing permissions later
sudo git clone https://github.com/jimsalterjrs/sanoid.git
cd sanoid
ln -s packages/debian .
dpkg-buildpackage -uc -us
apt install ../sanoid_*_all.deb
```

Enable sanoid timer:
```bash
# enable and start the sanoid timer
sudo systemctl enable sanoid.timer
sudo systemctl start sanoid.timer
```

## CentOS

Install prerequisite software:

```bash
# Install and enable epel if we don't already have it, and git too
sudo yum install -y epel-release git
# Install the packages that Sanoid depends on:
sudo yum install -y perl-Config-IniFiles perl-Data-Dumper lzop mbuffer mhash pv
```

Clone this repo, then put the executables and config files into the appropriate directories:

```bash
# Download the repo as root to avoid changing permissions later
sudo git clone https://github.com/jimsalterjrs/sanoid.git
cd sanoid
# Install the executables
sudo cp sanoid syncoid findoid sleepymutex /usr/local/sbin
# Create the config directory
sudo mkdir /etc/sanoid
# Install default config
sudo cp sanoid.defaults.conf /etc/sanoid
# Create a blank config file
sudo touch /etc/sanoid/sanoid.conf
# Place the sample config in the conf directory for reference
sudo cp sanoid.conf /etc/sanoid/sanoid.example.conf
```

Create a systemd service:

```bash
cat << "EOF" | sudo tee /etc/systemd/system/sanoid.service
[Unit]
Description=Snapshot ZFS Pool
Requires=zfs.target
After=zfs.target
ConditionFileNotEmpty=/etc/sanoid/sanoid.conf

[Service]
Environment=TZ=UTC
Type=oneshot
ExecStart=/usr/local/sbin/sanoid --take-snapshots --verbose
EOF

cat << "EOF" | sudo tee /etc/systemd/system/sanoid-prune.service
[Unit]
Description=Cleanup ZFS Pool
Requires=zfs.target
After=zfs.target sanoid.service
ConditionFileNotEmpty=/etc/sanoid/sanoid.conf

[Service]
Environment=TZ=UTC
Type=oneshot
ExecStart=/usr/local/sbin/sanoid --prune-snapshots --verbose

[Install]
WantedBy=sanoid.service
EOF
```

And a systemd timer that will execute **Sanoid** once per quarter hour
(Decrease the interval as suitable for configuration):

```bash
cat << "EOF" | sudo tee /etc/systemd/system/sanoid.timer
[Unit]
Description=Run Sanoid Every 15 Minutes
Requires=sanoid.service

[Timer]
OnCalendar=*:0/15
Persistent=true

[Install]
WantedBy=timers.target
EOF
```

Reload systemd and start our timer:
```bash
# Tell systemd about our new service definitions
sudo systemctl daemon-reload
# Enable and start the Sanoid timer
sudo systemctl enable sanoid.timer
sudo systemctl start sanoid.timer
```

Now, proceed to configure [**Sanoid**](#configuration)

## FreeBSD

Install prerequisite software:

```bash
pkg install p5-Config-Inifiles pv mbuffer lzop
```

**Additional notes:**

* FreeBSD may place pv and lzop in somewhere other than /usr/bin — syncoid currently does not check path.

* Simplest path workaround is symlinks, eg `ln -s /usr/local/bin/lzop /usr/bin/lzop` or similar, as appropriate to create links in **/usr/bin** to wherever the utilities actually are on your system.

* See note about mbuffer and other things in FREEBSD.readme

## Other OSes

**Sanoid** depends on the Perl module Config::IniFiles and will not operate without it. Config::IniFiles may be installed from CPAN, though the project strongly recommends using your distribution's repositories instead.

**Syncoid** depends on ssh, pv, gzip, lzop, and mbuffer. It can run with reduced functionality in the absence of any or all of the above. SSH is only required for remote synchronization. On newer FreeBSD and Ubuntu Xenial chacha20-poly1305@openssh.com, on other distributions arcfour crypto is the default for SSH transport since v1.4.6. Syncoid runs will fail if one of them is not available on either end of the transport.

### General outline for installation

1. Install prerequisites: Perl module Config::IniFiles, ssh, pv, gzip, lzop, and mbuffer
2. Download the **Sanoid** repo
3. Create the config directory `/etc/sanoid` and put `sanoid.defaults.conf` in there, and create `sanoid.conf` in it too
4. Create a cron job or a systemd timer that runs `sanoid --cron` once per minute

# Configuration

**Sanoid** won't do anything useful unless you tell it how to handle your ZFS datasets in `/etc/sanoid/sanoid.conf`.

**Syncoid** is a command line utility that doesn't require any configuration, with all of its switches set at runtime.

## Sanoid

Take a look at the files `sanoid.defaults.conf` and` sanoid.conf.example` for all possible configuration options. Also have a look at the README.md
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,11 +184,11 @@ As of 1.4.18, syncoid also automatically supports and enables resume of interrup

+ --source-bwlimit <limit t|g|m|k>

This is the bandwidth limit imposed upon the source. This is mainly used if the target does not have mbuffer installed, but bandwidth limites are desired.
This is the bandwidth limit in bytes (kbytes, mbytes, etc) per second imposed upon the source. This is mainly used if the target does not have mbuffer installed, but bandwidth limits are desired.

+ --target-bw-limit <limit t|g|m|k>

This is the bandwidth limit imposed upon the target. This is mainly used if the source does not have mbuffer installed, but bandwidth limites are desired.
This is the bandwidth limit in bytes (kbytes, mbytesm etc) per second imposed upon the target. This is mainly used if the source does not have mbuffer installed, but bandwidth limits are desired.

+ --no-command-checks

Expand Down
4 changes: 4 additions & 0 deletions packages/gentoo/sys-fs/sanoid/Manifest
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
AUX sanoid.cron 45 BLAKE2B 3f6294bbbf485dc21a565cd2c8da05a42fb21cdaabdf872a21500f1a7338786c60d4a1fd188bbf81ce85f06a376db16998740996f47c049707a5109bdf02c052 SHA512 7676b32f21e517e8c84a097c7934b54097cf2122852098ea756093ece242125da3f6ca756a6fbb82fc348f84b94bfd61639e86e0bfa4bbe7abf94a8a4c551419
DIST sanoid-2.0.1.tar.gz 106981 BLAKE2B 824b7271266ac9f9bf1fef5374a442215c20a4f139081f77d5d8db2ec7db9b8b349d9d0394c76f9d421a957853af64ff069097243f69e7e4b83a804f5ba992a6 SHA512 9d999b0f071bc3c3ca956df11e1501fd72a842f7d3315ede3ab3b5e0a36351100b6edbab8448bba65a2e187e4e8f77ff24671ed33b28f2fca9bb6ad0801aba9d
EBUILD sanoid-2.0.1.ebuild 772 BLAKE2B befbc479b5c79faa88ae21649ed31d1af70dbecb60416e8c879fffd9a3cdf9f3f508e12d8edc9f4e0afbf0e6ab0491a36fdae2af995a1984072dc5bffd63fe1d SHA512 d90a8b8ae40634e2f2e1fa11ba787cfcb461b75fa65b19c0d9a34eb458f07f510bbb1992f4a0e7a0e4aa5f55a5acdc064779c9a4f993b30eb5cbf39037f97858
EBUILD sanoid-9999.ebuild 752 BLAKE2B 073533436c6f5c47b9e8410c898bf86b605d61c9b16a08b57253f5a87ad583e00d935ae9ea90f98b42c20dc1fbda0b9f1a8a7bf5be1cf3daf20afc640f1428ca SHA512 40ad34230fdb538bbdcda2d8149f37eac2a0e2accce5f79f7ba77d8e62e3fd78e997d8143baa0e050f548f90ce1cb6827e50b536b5e3acc444c6032f170251be
1 change: 1 addition & 0 deletions packages/gentoo/sys-fs/sanoid/files/sanoid.cron
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* * * * * root TZ=UTC /usr/bin/sanoid --cron
36 changes: 36 additions & 0 deletions packages/gentoo/sys-fs/sanoid/sanoid-2.0.1.ebuild
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Copyright 2019 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2

EAPI=7

DESCRIPTION="Policy-driven snapshot management and replication tools for ZFS"
HOMEPAGE="https://github.com/jimsalterjrs/sanoid"
SRC_URI="https://github.com/jimsalterjrs/${PN}/archive/v${PV}.tar.gz -> ${P}.tar.gz"

LICENSE="GPL-3.0"
SLOT="0"
KEYWORDS="~x86 ~amd64"
IUSE=""

DEPEND="app-arch/lzop
dev-perl/Config-IniFiles
sys-apps/pv
sys-block/mbuffer
virtual/perl-Data-Dumper"
RDEPEND="${DEPEND}"
BDEPEND=""

DOCS=( README.md )

src_install() {
dobin findoid
dobin sanoid
dobin sleepymutex
dobin syncoid
keepdir /etc/${PN}
insinto /etc/${PN}
doins sanoid.conf
doins sanoid.defaults.conf
insinto /etc/cron.d
newins "${FILESDIR}/${PN}.cron" ${PN}
}
38 changes: 38 additions & 0 deletions packages/gentoo/sys-fs/sanoid/sanoid-9999.ebuild
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Copyright 2019 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2

EAPI=7

EGIT_REPO_URI="https://github.com/jimsalterjrs/${PN}.git"
inherit git-r3

DESCRIPTION="Policy-driven snapshot management and replication tools for ZFS"
HOMEPAGE="https://github.com/jimsalterjrs/sanoid"

LICENSE="GPL-3.0"
SLOT="0"
KEYWORDS="**"
IUSE=""

DEPEND="app-arch/lzop
dev-perl/Config-IniFiles
sys-apps/pv
sys-block/mbuffer
virtual/perl-Data-Dumper"
RDEPEND="${DEPEND}"
BDEPEND=""

DOCS=( README.md )

src_install() {
dobin findoid
dobin sanoid
dobin sleepymutex
dobin syncoid
keepdir /etc/${PN}
insinto /etc/${PN}
doins sanoid.conf
doins sanoid.defaults.conf
insinto /etc/cron.d
newins "${FILESDIR}/${PN}.cron" ${PN}
}
Loading