An offline copy is only used once: the day it has to be read. The disk comes out of the safe, or the server that held the datastore is gone, and the datastore has to start again on a PBS that has never seen it.
Here is how we reactivate a single datastore on a new PBS, from a dataset replicated with zfs send — the one from our offline copy.
What you need to gather
| Item | Why |
|---|---|
| the media, or the replicated dataset | the data: chunks, indexes, groups, owners |
| a PBS of the same major version | preferably at least as recent as the source |
| an OpenZFS that knows the pool's features | at least as recent as the one that created the pool, or as the compatibility level chosen when it was created — otherwise zpool import refuses the disk |
| the media key, if the media is encrypted | without it, the disk is just a block of bytes; it never travels with it |
| the client-side encryption key | PBS never had it and cannot rebuild it. It is the only item whose loss is permanent. |
| a Proxmox VE | to restore VMs and containers |
1. Import the pool and mount the dataset
zpool import # lists visible pools, imports nothing
zpool import airgap-A1
zfs list -r -o name,mounted,mountpoint airgap-A1
zfs mount airgap-A1/copies/clientX # received with -u: it is not mountedIf the media is encrypted, unlock it before the zfs mount, with the key that travelled separately. If zpool import refuses the pool because of unsupported features, this server's OpenZFS is too old for it: update it, nothing is lost on the disk.
The zfs mount is not optional. A dataset received with -u is not mounted, and its mountpoint does not even exist yet. Without it, declaring the datastore hits an empty directory — and you believe the copy is lost when it is intact.
Is the disk going back into rotation? Then do not declare the dataset itself: PBS would write to it, and the next incremental would be rejected. Work on a clone, as for the check before the safe.
2. Declare the datastore, without touching the content
proxmox-backup-manager datastore create clientX /mnt/datastore/clientX --reuse-datastore true
proxmox-backup-manager datastore listExpected output: Access time update check successful. then TASK OK. PBS checks along the way that atime works, because its garbage collector depends on it. The groups, snapshots and their owners are there. A PBS more recent than the source is not a problem; the other way round, in our test a datastore coming from 4.2.5 read back on 4.2.0 caused no trouble.
Use the same datastore name as on the old server if Proxmox VE hosts have to reconnect to it without changing their configuration.
3. Read it back before restoring
proxmox-backup-manager verify clientX --ignore-verified falsecurl -sk -X POST -H "Authorization: PBSAPIToken=<authid>:<secret>" \
-d backup-type=vm -d backup-id=<id> -d backup-time=<epoch> -d ignore-verified=0 \
"https://localhost:8007/api2/json/admin/datastore/clientX/verify"Without ignore-verified, the verify reads nothing
The verification state travelled with the dataset. A regular verify answers SKIPPED: … (recently verified) without reading a single byte. And until the verify and a first restore have passed, no garbage collector on this datastore.
4. Grant access
Two situations, which do not require the same thing.
An emergency PBS, for the time it takes to restore
Degraded, temporary mode: a new, read-only account. The DatastoreReader role reads and restores backups from every owner: no need to recreate the original account. It is set twice, on the user and on its token: a token's effective privileges are the intersection of its own and its user's. With the ACL on only one of the two, reading is refused (missing Datastore.Audit|Datastore.Backup). And give the permissions a few seconds to apply: a refusal right after acl update is not necessarily an error.
proxmox-backup-manager user create restore@pbs
proxmox-backup-manager user generate-token restore@pbs pve # the secret is the "value" field
proxmox-backup-manager acl update /datastore/clientX DatastoreReader --auth-id restore@pbs
proxmox-backup-manager acl update /datastore/clientX DatastoreReader --auth-id 'restore@pbs!pve'
proxmox-backup-manager cert info # the fingerprint to give Proxmox VEThe datastore goes back into service for the same customers
Here, nobody should have to change anything: accounts, their hashed passwords, their tokens and their permissions are carried over from the old server. It is exactly the "accounts" step of a PBS migration, with its two pitfalls.
5. Declare the datastore in Proxmox VE
pvesm add pbs pbs-secours \
--server <new-pbs> --datastore clientX \
--username 'restore@pbs!pve' --password '<token secret>' \
--fingerprint '<certificate fingerprint>' \
--encryption-key /etc/pve/priv/storage/<usual-storage>.enc \
--nodes <node> --content backup
pvesm status
pvesm list pbs-secours | wc -l- An additional storage, in no backup job: the PVE now sees the same backups twice, and nothing must write to the copy by mistake;
--encryption-keyaccepts a file path and copies the key under the new storage ID. On a fresh Proxmox VE, point it at the copy of the key you kept;- the copied
.encis one byte larger than the original:pvesmadds a newline. It is not a different key — the fingerprint written instorage.cfgis what counts.
6. Prove you can restore, then restore
Listing the backups proves nothing: indexes can be browsed without the key. What brings encryption into play is extracting a VM configuration:
pvesh get /nodes/<node>/vzdump/extractconfig --volume 'pbs-secours:backup/vm/<id>/<date>'
qmrestore pbs-secours:backup/vm/<id>/<date> <new-vmid> --storage local-lvm
pct restore <new-ctid> pbs-secours:backup/ct/<id>/<date> --storage local-lvmExpected, for encrypted backups: Using encryption key from file descriptor.. Fingerprint: …, then the VM configuration in clear text. Then restore always to a new ID, never on top of an existing VM. File-level restore is done from the Proxmox VE interface, with the File Restore button.
Restore from the disk, or rehydrate first?
A rotation disk is a single spindle: fast at reading end to end, slow at thousands of scattered reads — which is what a restore is. For one VM or a few files, restore directly: only the relevant chunks are read.
To bring everything back, you can first copy the dataset to fast storage, sequentially, and declare the datastore there:
zfs send -L -c airgap-A1/copies/clientX@<snapshot> | zfs receive -u tank/datastores/clientX
zfs mount tank/datastores/clientX
proxmox-backup-manager datastore create clientX <mountpoint> --reuse-datastore trueWhere the tipping point lies depends on the disk, the volume and the number of VMs: measure it on your hardware before you need it.
If the datastore goes back into service
Garbage collection only after the verify and a successful restore. For the rest — accounts carried over, jobs to recreate, the TLS fingerprint seen by Proxmox VE hosts —, it is a PBS migration (steps 5 to 7).
At Nimbus
With dedicated offline media, we can lend you your disks, subject to a deposit: this procedure is then the one you follow on your own PBS. With a dedicated offline PBS, what arrives is a server, ready to boot. Either way, the media key never travels in the parcel. And the pools of our offline media are created with a ZFS feature set frozen in advance, so they also import on a server less up to date than ours.
Frequently asked questions
Can a PBS datastore be read on a server that has never seen the old one?
Yes. A PBS datastore contains everything needed to be read: chunks, indexes, groups, snapshots and owners. proxmox-backup-manager datastore create with --reuse-datastore true declares it without touching the content, on any PBS of the same major version. Nothing from the old server is needed to restore, apart from the client-side encryption key if the backups are encrypted.
Why does datastore create find an empty directory?
Because the dataset is not mounted. A dataset received with zfs receive -u is not mounted, and its mountpoint does not even exist yet. A zfs mount before declaring it, and the content appears: the copy was never lost.
Do I need to recreate the account that owns the backups in order to restore?
No. The DatastoreReader role on the datastore can read and restore backups from every owner. The original account is only needed if the datastore has to go back into service for the same customers without them changing anything: accounts then have to be carried over, as in a migration.
What do I need besides the disk to restore?
A PBS of the same major version, the media key if the media is encrypted, and the client-side encryption key if the backups are. The client key is the only item whose loss is permanent: PBS never had it. On the Proxmox VE side, it lives in /etc/pve/priv/storage/<storage>.enc — a copy must exist outside the infrastructure being backed up.
An offline copy we can lend you
AirGapped Drive PBS at €34 excl. VAT/TB/month, with optional dedicated media — disks or offline PBS — that we lend you the day you need it, subject to a deposit. On quote.