Deduplication in Proxmox Backup Server: how it works, and what the factor shows

1. Two ways to split: fixed and dynamic chunks

VM disks: fixed 4 MiB chunks

For a block-based backup (VM disk, .img image), PBS splits the content into chunks of the same length: “typically 4 MiB”, says the documentation. In the code it is more than typical: the backup API has never created a fixed index with another size, and since PBS 4.2.8 (October 2, 2026) a fixed index of any other size is rejected on read and on write (pbs-datastore/src/fixed_index.rs). The last chunk of a disk can be smaller.

In practice, the --chunk-size option of proxmox-backup-client is useless for an image. Our bench found it before we read it in the code: below 4096 KB, every .img archive fails.

Files and containers: dynamic chunks from 1 to 16 MiB

For a file backup (pxar archive: LXC containers, servers backed up with the client), PBS places chunk boundaries according to the content, with a rolling hash: “We use a variant of Buzhash” (Technical Overview). An insertion in the middle of a file therefore does not shift all the following chunks.

• Default target average size: 4 MiB.
• Actual chunk size: from a quarter to four times the average, so 1 to 16 MiB by default (pbs-datastore/src/chunker.rs).
• --chunk-size (64 KiB to 4 MiB, power of 2) sets this average, not a maximum.

Each chunk, fixed or dynamic, is then compressed with zstd, when compression makes it smaller, before being encrypted if needed and written. For what follows, remember that compression adds to deduplication.

2. Scope: the whole datastore, but only one

A datastore's chunks live in a single .chunks/ directory, where a chunk's path depends only on its fingerprint. A chunk that is already there is not written again, whatever machine or namespace sends it. The documentation even presents namespaces as a way to reuse “a single chunk store deduplication domain for multiple sources” (Terminology).

• Storage: deduplication across the whole datastore, all machines and namespaces combined.
• Network: the client downloads the chunk list of the previous snapshot of the same group and does not upload the chunks it finds there. The first backup of a new VM therefore transfers its chunks, even though the server keeps a single copy.
• Across datastores: nothing. Each one has its own chunk store.

3. Encryption and deduplication

An encrypted chunk cannot be identified by the fingerprint of its encrypted content, otherwise nothing would deduplicate. PBS computes the fingerprint on the plain data, concatenated with a key derived from your encryption key (in the code: SHA-256(data ‖ id_key), with an id_key derived by PBKDF2, pbs-tools/src/crypt_config.rs). The documentation gives the reason: two identical chunks encrypted with different keys produce two different fingerprints (Encrypted Chunks).

• Same data, same key, several machines: normal deduplication.
• Two different keys: no deduplication between them.
• Encrypted and unencrypted backups: no deduplication. Turning encryption on later therefore sends all the data once more.

The client only encrypts the chunks it actually has to upload: deduplication is decided on your machine, and the server only receives encrypted chunks. At NimbusBackup: key not requested for backup hosting only; it stays with you. To create and keep it, see the PBS encryption key.

4. Reading the deduplication factor

In the PBS interface, on the datastore's Summary tab, the Stats from last Garbage Collection panel shows a Deduplication Factor. Three things to know to read it correctly:

1. It is only computed at the end of a garbage collection, and kept until the next one. Before the first one, it is 1.00.
2. It is a volume ratio: the cumulative logical size of all indexes of all snapshots (index-data-bytes), divided by the real space of the chunks on disk (disk-bytes). It therefore includes compression.
3. Each snapshot counts as a full copy. For a VM, that is the whole virtual disk size, empty blocks included. The more snapshots a datastore holds, the higher the factor, without deduplication being any “better”.

On the command line, the status of the last garbage collection gives both raw values; the factor is their quotient.

proxmox-backup-manager garbage-collection status <datastore> --output-format json
# factor = index-data-bytes / disk-bytes

5. Our measured figures

We do not publish a “typical rate”: it depends on your data. Here are two measurements we made ourselves, with their dates. Note that neither is the PBS factor described above: they are shares of data (or chunks) already present at the time of a backup.

MariaDB to PBS (SQL dumps, dynamic chunks averaging 1 MB)

Six databases, about 30 GiB of uncompressed dumps, one backup per night. Share of the dump's data already present on the datastore, computed as 1 − (new / total). Full details in our MariaDB to Proxmox Backup Server bench.

The 54.4% row is the most useful one: a routine database operation is enough to halve one night's deduplication. An average would have hidden it.

A 1 TB Windows server

On an 877 GB Windows server backed up with the Windows client, the 2nd run (18→19/04/2026) found 99.97% of its chunks already present, and only sent 1.73 GB. Here the percentage counts chunks, not bytes. Details in the 1 TB Windows case study.

6. Garbage collection: why space does not come back right away

A chunk shared by several backups cannot be erased as soon as one of them no longer needs it. PBS leaves this to garbage collection, in two phases (Maintenance):

1. Mark: all indexes are read, and the access time (atime) of every referenced chunk is updated.
2. Sweep: only chunks whose access time is older than the cutoff are erased. By default the cutoff is 24 h 5 min before the start of the garbage collection, or earlier if a backup is still running. Chunks within this grace period are reported at the end of the task as Pending removals.

The delay comes from the relatime mount option, which only updates a file's access time at most once every 24 h. It can be tuned per datastore with the gc-atime-cutoff option, in minutes (1 to 2880). In practice, a chunk that is no longer needed still takes up space until the garbage collection that runs more than 24 h after its last marking.

7. What it means for pricing at NimbusBackup

At NimbusBackup, you reserve a capacity in TB. What fills that reservation is the chunks, stored once and compressed, not the sum of your backups: deduplication decides how many restore points fit in the reserved TB. To size it, start from your data volume, then adjust the reservation after the first garbage collections, once the space actually used is known.

To compare plans and estimate your needs, see choose my backup.

Frequently asked questions