What could cause much higher upload progress than possible given bandwidth?

[tredondo]

Half an hour ago, I started the first snapshot backup into a new instance ID within a repo. My connection is 10Mbps up (1.5MiB/s average upload speed).

Backrest already shows that 66 GiB have been uploaded. That seems impossible because even with compression, it would mean that 66GiB have been compressed into 30 * 60 * 1.5 ~ 2.7GiB. I don't have such compressible files (25:1) among the paths to back up, and even if I had, it's highly unlikely that restic managed to upload exactly those files in the first 30 minutes.

What could explain this? Could restic find files with the same hash among the snapshots of a previous instance ID? (Most of the uploaded files were actually already uploaded to a previous instance ID).

[m-reiter]

AFAIK, deduplication in restic is indeed on a per-repository basis. This forum post seems to confirm that.

[garethgeorge]

This is exactly right-- very likely that you're benefiting from content based chunking & dedupe here.