Restore from Hetzner S3 bucket exits with "error 1" but restored file is accessible

[toskium]

Hello there,

TL;DR; backrest is able to create backups on a Hetzner S3 bucket but reports "error 1" when restoring even though the file is successfully restored. Restoring to my local downloads folder works without any error. Any ideas? :-)

My setup looks like this:

backrest runs as a custom app on TrueNAS Scale 25.0.4.2.4, the config file looks like this:

services:
  backrest:
    environment:
      BACKREST_CONFIG: /config/config.json
      BACKREST_DATA: /data
      NVIDIA_VISIBLE_DEVICES: void
      TMPDIR: /tmp
      TZ: Europe/Berlin
      XDG_CACHE_HOME: /cache
    group_add:
      - 568
    image: garethgeorge/backrest:latest
    platform: linux/amd64
    ports:
      - '9898:9898'
    privileged: False
    pull_policy: missing
    restart: 'no'
    stdin_open: False
    tty: False
    volumes:
      - bind:
          create_host_path: False
          propagation: rprivate
        read_only: False
        source: /mnt/pool0/userhomes/testuser
        target: /mnt/pool0/userhomes/testuser
        type: bind
      - bind:
          create_host_path: True
          propagation: rprivate
        read_only: False
        source: /mnt/pool0/docker/backrest/data
        target: /data
        type: bind
      - bind:
          create_host_path: True
          propagation: rprivate
        read_only: False
        source: /mnt/pool0/docker/backrest/config
        target: /config
        type: bind
      - bind:
          create_host_path: True
          propagation: rprivate
        read_only: False
        source: /mnt/pool0/docker/backrest/cache
        target: /cache
        type: bind
      - bind:
          create_host_path: True
          propagation: rprivate
        read_only: False
        source: /mnt/pool0/docker/backrest/tmp
        target: /tmp
        type: bind
      - bind:
          create_host_path: True
          propagation: rprivate
        read_only: False
        source: /mnt/pool0/docker/backrest/rclone
        target: /root/.config/rclone
        type: bind
volumes: {}

The container runs, is accessible and the configuration survives restarts, so I think that container wise everything is in order.

I have successfully created a Hetzner S3 bucket and added it as a repository, this is the redacted json config from backrests webui:

{
  "id": "s3-hetzner-testbucket01",
  "guid": "0677e144f8eff1442e041d5b166c9f858efb6a1b8ae147864728619e39d7d8a8",
  "uri": "s3:https://fsn1.your-objectstorage.com/testbucket01",
  "password": "...",
  "env": [
    "AWS_ACCESS_KEY_ID=key",
    "AWS_SECRET_ACCESS_KEY=key"
  ],
  "flags": [],
  "prunePolicy": {
    "maxUnusedPercent": 10,
    "schedule": {
      "clock": "CLOCK_LAST_RUN_TIME",
      "cron": "0 0 1 * *"
    }
  },
  "checkPolicy": {
    "readDataSubsetPercent": 0,
    "schedule": {
      "clock": "CLOCK_LAST_RUN_TIME",
      "cron": "0 0 1 * *"
    }
  },
  "commandPrefix": {
    "ioNice": "IO_DEFAULT",
    "cpuNice": "CPU_DEFAULT"
  },
  "autoUnlock": false,
  "hooks": []
}

Initializing the repository works, creating backups works, unfortunately restores don't work as expected.
When I do a test restore of some file this happens:

[repo-manager] 13:39:54.368Z	debug	restore snapshot	{"repo": "s3-hetzner-testbucket01", "snapshot": "87da36555a589973242de869a8e8e53aade7229b0411205667b9fb637b304761", "target": "/mnt/pool0/userhomes/testuser/s3test/test.zip-backrest-restore-87da3655"}

[restic] 13:39:54.368Z	debug	command: "/bin/restic restore --json 87da36555a589973242de869a8e8e53aade7229b0411205667b9fb637b304761:/mnt/pool0/userhomes/testuser/s3test -o sftp.args=-oBatchMode=yes --target /mnt/pool0/userhomes/testuser/s3test/test.zip-backrest-restore-87da3655 --include test.zip"

[tasklog] 13:39:56.108Z	error	restore snapshot "87da36555a589973242de869a8e8e53aade7229b0411205667b9fb637b304761" in repo "s3-hetzner-testbucket01"	task failed	{"error": "restore snapshot \"87da36555a589973242de869a8e8e53aade7229b0411205667b9fb637b304761:/mnt/pool0/userhomes/testuser/s3test\" for repo s3-hetzner-testbucket01: command \"/bin/restic restore --json 87da36555a589973242de869a8e8e53aade7229b0411205667b9fb637b304761:/mnt/poo...\" failed: exit code 1: restore failed\noutput processing: Fatal: There were 1 errors\n", "duration": 1.7405773}

But even though restic reports a failure the file is successfully restored.
Restoring the file to my client's download folder works without any error.
I suspect some permission issue with the host mounted dataset.

Does anyone have an idea what is going on?

[toskium]

I just found the solution. In case someone else wants to try this with TrueNAS Scale.

All you have to do is to add

user: '568:568'

To the container config and add the user "apps" and/or the group "apps" to the dataset with modification permissions.