CIFS storage LXC move/clone/backup errors

[openaspace]

Hello.
When i try to backup or copy or move a vm or lxc, sometimes the cifs storage report error.

If I clone or move a VPS it works correctly.
any idea?

mount: /var/lib/lxc/403/.copy-volume-2: cannot mount /dev/loop0 read-only.
Specified filename /var/lib/lxc/403/.copy-volume-1 does not exist.
TASK ERROR: command 'mount -o ro /dev/loop0 /var/lib/lxc/403/.copy-volume-2//' failed: exit code 32

 

[openaspace]

Cloning a container for test, it report a strange error of a read only file inside the container ...

create full clone of mountpoint rootfs (local-zfs:subvol-105-disk-0)
Formatting '/mnt/pve/storagebox/images/103/vm-103-disk-0.raw', fmt=raw size=161061273600 preallocation=off
Creating filesystem with 39321600 4k blocks and 9830400 inodes
Filesystem UUID: 9719b634-fac4-43a5-890e-d8a01e9c5394
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
    4096000, 7962624, 11239424, 20480000, 23887872
rsync: [receiver] write failed on "/var/lib/lxc/103/.copy-volume-1/var/www/nextcloud-data/admin/files/Cloudstation/quote.php": Read-only file system (30)
rsync error: error in file IO (code 11) at receiver.c(378) [receiver=3.2.3]
rsync: [sender] write error: Broken pipe (32)
TASK ERROR: clone failed: command 'rsync --stats -X -A --numeric-ids -aH --whole-file --sparse --one-file-system '--bwlimit=0' /var/lib/lxc/103/.copy-volume-2/ /var/lib/lxc/103/.copy-volume-1' failed: exit code 11

 

[openaspace]

create full clone of mountpoint rootfs (local-zfs:subvol-105-disk-0)
Formatting '/mnt/pve/storagebox/images/103/vm-103-disk-0.raw', fmt=raw size=161061273600 preallocation=falloc
Creating filesystem with 39321600 4k blocks and 9830400 inodes
Filesystem UUID: 981104a7-efa4-458c-9cf4-54d774bb4f0e
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
    4096000, 7962624, 11239424, 20480000, 23887872
rsync: [receiver] write failed on "/var/lib/lxc/103/.copy-volume-1/var/www/nextcloud-data/admin/files/Cloudstation/.debris/2022-02-26/2022-01-13 alle 6.37.06 PM.mov": Read-only file system (30)
rsync error: error in file IO (code 11) at receiver.c(378) [receiver=3.2.3]
rsync: [sender] write error: Broken pipe (32)
TASK ERROR: clone failed: command 'rsync --stats -X -A --numeric-ids -aH --whole-file --sparse --one-file-system '--bwlimit=0' /var/lib/lxc/103/.copy-volume-2/ /var/lib/lxc/103/.copy-volume-1' failed: exit code 11

 

[openaspace]

Received From: px7->/var/log/syslog Rule: 1002 fired (level 2) -> "Unknown problem somewhere in the system." Portion of the log(s): Apr 17 20:02:12 px7 kernel: [181635.082151] CIFS: VFS: \\192.168.10.10 crypt_message: Failed to init sg

Received From: px7->/var/log/messages Rule: 1002 fired (level 2) -> "Unknown problem somewhere in the system." Portion of the log(s): Apr 17 21:01:42 px7 kernel: [185205.787066] kworker/u16:3: page allocation failure: order:4, mode:0x40cc0(GFP_KERNEL|__GFP_COMP), nodemask=(null),cpuset=/,mems_allowed=0

 

[openaspace]

no way to use a container on a CIFS storage?

 

[openaspace]

Failed LXC Backup Jobs on CIFS storage.

105    cloud    FAILED    00:18:48    command 'set -o pipefail && tar cpf - --totals --one-file-system -p --sparse --numeric-owner --acls --xattrs '--xattrs-include=user.*' '--xattrs-include=security.capability' '--warning=no-file-ignored' '--warning=no-xattr-write' --one-file-system '--warning=no-file-ignored' '--directory=/var/tmp/vzdumptmp2360287_105' ./etc/vzdump/pct.conf ./etc/vzdump/pct.fw '--directory=/mnt/vzsnap0' --no-anchored '--exclude=lost+found' --anchored '--exclude=./tmp/?*' '--exclude=./var/tmp/?*' '--exclude=./var/run/?*.pid' ./ >/mnt/pve/storagebox/dump/vzdump-lxc-105-2022_04_20-04_02_12.dat' failed: exit code 2

command 'set -o pipefail && tar cpf - --totals --one-file-system -p --sparse --numeric-owner --acls --xattrs '--xattrs-include=user.*' '--xattrs-include=security.capability' '--warning=no-file-ignored' '--warning=no-xattr-write' --one-file-system '--warning=no-file-ignored' '--directory=/var/tmp/vzdumptmp2360287_401' ./etc/vzdump/pct.conf ./etc/vzdump/pct.fw '--directory=/mnt/vzsnap0' --no-anchored '--exclude=lost+found' --anchored '--exclude=./tmp/?*' '--exclude=./var/tmp/?*' '--exclude=./var/run/?*.pid' ./ >/mnt/pve/storagebox/dump/vzdump-lxc-401-2022_04_20-04_36_33.dat' failed: exit code 2

command 'set -o pipefail && lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- tar cpf - --totals --one-file-system -p --sparse --numeric-owner --acls --xattrs '--xattrs-include=user.*' '--xattrs-include=security.capability' '--warning=no-file-ignored' '--warning=no-xattr-write' --one-file-system '--warning=no-file-ignored' '--directory=/var/tmp/vzdumptmp2360287_600' ./etc/vzdump/pct.conf ./etc/vzdump/pct.fw '--directory=/mnt/vzsnap0' --no-anchored '--exclude=lost+found' --anchored '--exclude=./tmp/?*' '--exclude=./var/tmp/?*' '--exclude=./var/run/?*.pid' ./ >/mnt/pve/storagebox/dump/vzdump-lxc-600-2022_04_20-05_25_12.dat' failed: exit code 2

 

[fabian]

an informed guess given the name - is this hetzner's storagebox product you are using? reports on this forum indicate it's barely suitable as backup target, I'd not use it for storing the guest volumes themselves.

 

[openaspace]

an informed guess given the name - is this hetzner's storagebox product you are using? reports on this forum indicate it's barely suitable as backup target, I'd not use it for storing the guest volumes themselves.

Thank you for the support.
Yes I'm trying to use it only as backup storage location..
even if the vps work without any problem. The use of containers is instead impossible,

 

[fabian]

you are definitely not only using it as backup storage location according to the logs you posted.. in any case, if the samba mount goes read-only there is nothing PVE can do about it except error out - talk to Hetzner

 

[openaspace]

you are definitely not only using it as backup storage location according to the logs you posted.. in any case, if the samba mount goes read-only there is nothing PVE can do about it except error out - talk to Hetzner

Hello thank you.
But also using it only as backup,some LXC are not execute the backup job....

Question, why is proxmox go to readonly when backuping...it allow to use a storage like this to host vm storage?

 

[fabian]

it's not Proxmox that goes readonly, but the storage like I said, please look at the journal/syslog and go complain to Hetzner, it's likely the samba server not responding (fast enough) that causes this.