A full PBS, a server at end of life, a change of hosting provider: sooner or later, a datastore has to move. A sync job can do it, by filling a new datastore. When the datastore sits on ZFS, you can move the datastore itself, live, and suspend backups only for the time of one last incremental.
It is the same mechanism as our offline copy with zfs send, and it is how we move a datastore from one server to another: on arrival, the Proxmox VE host reads back and decrypts its backups on the new server with the same account, the same password and the same key.
What travels with the dataset, and what stays behind
Travels with zfs send | Stays on the old server, in /etc/proxmox-backup/ |
|---|---|
| chunks and indexes groups, snapshots and namespaces the owner of each group the verification state of snapshots | the datastore declaration — datastore.cfgaccounts — user.cfg, shadow.json, token.shadowpermissions — acl.cfgjobs — prune.cfg, verification.cfg, sync.cfgthe certificate — proxy.pem, proxy.key |
Good news for the right-hand column: a freshly installed PBS has no datastore.cfg, no acl.cfg, no token.shadow and no job files. They are created on first use. On the target, there is therefore nothing to merge with existing content: only what you created on the source needs carrying over.
Prerequisites
- ZFS on both sides, with the source datastore being a dataset of its own;
- the same major PBS version, ideally with the target at least as recent;
- the same datastore name on the target: that is the name the Proxmox VE hosts reference;
- key-based SSH access from the target to the source — the same setup as for the offline copy.
Step 1 — The inventory, before touching anything
proxmox-backup-manager version --verbose
proxmox-backup-manager datastore show clientX
zfs list -o name,used,avail,recordsize,atime,mountpoint rpool/datastores/clientX
proxmox-backup-manager user list
proxmox-backup-manager acl list
proxmox-backup-manager prune-job list
proxmox-backup-manager verify-job list
proxmox-backup-manager sync-job list
proxmox-backup-manager cert info # the fingerprint the Proxmox VE hosts have pinned
ls -l /etc/proxmox-backup/This list is what will tell you, at the end, whether anything was forgotten. On the target, prepare the parent that will receive the dataset:
zfs create -o mountpoint=/mnt/datastore tank/datastoresSteps 2 and 3 — Full copy, then incrementals, live
Backups keep running on the old server throughout this phase. The datastore is not yet declared on the target: no garbage collection and no prune can touch it.
The full copy, then as many incrementals as needed — each one shorter than the last —, are done exactly as for an offline copy: snapshot and bookmark on the old server, zfs send -L -c then zfs send -L -c -i from the bookmark, received with zfs receive -s -u (plus -x mountpoint on the first send). The commands and options are detailed in Offline copy of a PBS datastore with zfs send, steps 3 and 5; the target is simply the new PBS instead of a disk.
From here on, the snapshots are named migration-1, migration-2… and the received dataset is tank/datastores/clientX.
Step 4 — The cutover
Schedule it outside the backup window: during the cutover, a backup started against the old server will fail, and that is intended.
proxmox-backup-manager datastore update clientX --maintenance-mode type=read-only
proxmox-backup-manager task list # wait until no backup is still running
zfs snapshot rpool/datastores/clientX@migration-final
zfs bookmark rpool/datastores/clientX@migration-2 rpool/datastores/clientX#migration-2Maintenance mode does not stop what is already running
PBS blocks new writes, but lets operations started before the mode was enabled finish. A snapshot taken right after the command may therefore contain a half-written backup. Wait for the tasks to end, and only then take the final snapshot. In read-only mode, restores remain possible throughout the cutover.
set -o pipefail
ssh old-pbs zfs send -L -c -i rpool/datastores/clientX#migration-2 rpool/datastores/clientX@migration-final \
| zfs receive -s -u tank/datastores/clientX
echo "PIPESTATUS=${PIPESTATUS[*]}"
zfs mount tank/datastores/clientX # received with -u: it is not mounted
proxmox-backup-manager datastore create clientX /mnt/datastore/clientX --reuse-datastore trueExpected: Access time update check successful. then TASK OK. Without the zfs mount, the mountpoint does not exist yet and datastore create hits an empty directory: the copy is not lost, it simply is not mounted.
Step 5 — Accounts, passwords, tokens and permissions
The owner of each group arrived with the datastore; the account bearing that name did not. As long as it does not exist on the target, the client sees nothing. And the goal is for the client to have nothing to change: no password, no token.
The cleartext password is never needed. You carry over what PBS already stores: the password hash and the hashed token secrets. Nobody needs to know the password to perform the migration.
The whole server moves
The new PBS replaces the old one: take the files whole, after backing up the target's own copies.
cd /etc/proxmox-backup
mkdir -p /root/before-migration && cp -a user.cfg shadow.json /root/before-migration/
# from the old PBS, preserving owner and permissions
# user.cfg shadow.json token.shadow acl.cfg
# (+ domains.cfg and tfa.json if you use LDAP/AD or two-factor authentication)
proxmox-backup-manager user list
proxmox-backup-manager acl listOne datastore among others moves
If the old and new servers host other accounts, never copy the whole files: they contain everyone's accounts. Carry over four fragments, those of the one account concerned:
user.cfg → the "user: client@pbs" section and its lines
(and its "token: client@pbs!<id>" sections, if any)
shadow.json → the "client" entry ← WITHOUT the @pbs suffix
token.shadow → the "client@pbs!<id>" entry, if there are tokens
acl.cfg → the line "acl:1:/datastore/clientX:client@pbs:DatastoreBackup"Two costly traps
shadow.json is keyed without the realm. The entry is named client, not client@pbs. Looking it up with the suffix returns "absent" even though it exists.
user.cfg is a section-based file, and a section with no properties must be followed by a blank line. A token: section created by generate-token often has no properties at all. Paste a section right after it, and PBS answers syntax error — then rejects the entire file: no users, no ACLs left, including for accounts that were working. Back up before editing, and check with proxmox-backup-manager user list immediately afterwards.
Step 6 — The jobs
Prune, verify, sync, notifications: none of these followed. Their absence does not get in the way of any restore, but a datastore without prune silently fills up — all the faster if clients, being write-only, cannot prune by themselves. Recreate them from the step 1 inventory, and leave garbage collection aside until step 8.
Step 7 — What Proxmox VE sees: the name and the fingerprint
Proxmox VE pins the PBS certificate fingerprint in /etc/pve/storage.cfg. A new server has a new certificate: this is the point that decides whether the migration is invisible.
| Situation | What Proxmox VE sees | To do on the PVE side |
|---|---|---|
| a front end (reverse proxy) terminates TLS in front of the PBS servers | the front end's certificate, unchanged | nothing |
the old PBS certificate is reused on the new one (proxy.pem + proxy.key, then restart proxmox-backup-proxy) | the same certificate, hence the same fingerprint | nothing, apart from the address if it changes |
| new certificate, or a front end that merely passes TLS through | the new PBS certificate | pvesm set <storage> --fingerprint <fingerprint> on every PVE host |
The name: switch DNS, or change the address
If the Proxmox VE hosts target a DNS name, point that name at the new server, and the storage configuration does not change. Once DNS has propagated, disconnecting and reconnecting the datastore on the Proxmox VE side is enough for it to reach the new server — in the web interface (Datacenter → Storage → disable, then enable again) or from the command line:
pvesm set backup --disable 1
pvesm set backup --disable 0
pvesm status --storage backupOtherwise, change the address: pvesm set <storage> --server <new>. Either way, the fingerprint follows the rules in the table above. In the end, the storage entry differs at most by its server line — and with DNS switched, not even that one changes:
pbs: backup pbs: backup
datastore clientX datastore clientX ← identical
server pbs.example.com server pbs.example.com ← identical if DNS is switched
fingerprint <fingerprint> fingerprint <fingerprint> ← depends on the case above
username client@pbs username client@pbs ← identical
encryption-key <key fingerprint> encryption-key <fingerprint> ← identicalThe client encryption key does not move: it lives on the Proxmox VE side, in /etc/pve/priv/storage/<storage>.enc, and the new PBS never sees it any more than the old one did. Declaring a second storage instead of modifying the first is also possible: that is what reactivating a datastore describes.
Step 8 — Prove it works, before decommissioning
Before decommissioning the old server, you need proof that you can restore from the new one: a forced verify, extracting a VM configuration (which involves the encryption key), then a real restore to a new ID. These checks are detailed, commands included, in Reactivate a PBS datastore after losing the server (steps 3 and 6). For a migration, only the order matters:
- the proofs: forced verify, configuration extraction, restore;
- only then, the first garbage collection on the new datastore;
- jobs re-enabled, one night of successful backups, and only then decommissioning the old server.
Rolling back
As long as the old server exists, you just lift the maintenance mode — proxmox-backup-manager datastore update clientX --delete maintenance-mode — and point the Proxmox VE hosts back to the old address. Nothing was destroyed on the source: snapshots and bookmarks, that is all.
At Nimbus
This is how we move a datastore from one of our servers to another. We carry over the password hash and the tokens: you have no new password and no new token to enter. The only point that could concern you is the fingerprint pinned in your Proxmox VE, and that is precisely what we check before moving anything.
Frequently asked questions
Can a PBS datastore be migrated without interrupting backups?
Almost. The full copy and the incrementals run live, while backups keep going to the old server. The interruption is limited to the cutover: the source datastore goes into read-only maintenance, running tasks finish, then one last, short incremental goes to the new server. Restores remain possible throughout.
Do the old and new servers need the same PBS version?
The same major version. A target more recent than the source is not a problem. The other way round, in our test a datastore coming from PBS 4.2.5 read back on 4.2.0 caused no trouble, but that is no reason to migrate to an older release. On the ZFS side, a zfs send stream is received without difficulty on a newer OpenZFS; it is importing a disk that requires an OpenZFS aware of every active feature of the pool.
Do users and Proxmox VE hosts have to change their password or token?
No. The cleartext password is never needed: you carry over the hash (shadow.json) and the hashed token secrets (token.shadow), along with the account definitions (user.cfg) and their permissions (acl.cfg). Careful: shadow.json is keyed by the user name without the realm — the key is "client", not "client@pbs".
Why does Proxmox VE refuse to connect to the new PBS?
Most often, the certificate fingerprint. Proxmox VE pins it in /etc/pve/storage.cfg. A new server has a new certificate, hence a new fingerprint: either reuse the old server's certificate, or have a front end that terminates TLS keep presenting the same one, or update the fingerprint with pvesm set <storage> --fingerprint.
Sync job or zfs send to migrate a PBS?
A sync job does not require ZFS and remains the official route: it fills a new datastore snapshot by snapshot. But synced groups take the owner defined in the job (root@pam by default), which then has to be fixed for each client. zfs send moves the datastore itself, owners, namespaces and verification state included, provided you have ZFS on both sides.
An offsite PBS you will never have to migrate
Datastore in France, servers replaced and migrated by us, offline copy as an option. You keep your local PBS; we carry the offsite copy.