[SOLVED] Cannot restore CT No space left on device

[totalimpact]

This is just a lab environment, I have 2 PVE, PVE1 is using sshfs to mount backup storage on PVE2 to backup CT to other host. Now I want to restore the backups from PVE1 to a CT on PVE2... no matter what I do I keep getting an error that it cannot create dirs or no space left on device, which has even happened on a nearly empty 500gb SSD, I created a new 16GB debian CT on that SSD fine, but restoring to it fails, doesnt matter if its dir storage or block.




Just ran dist-upgrade to 8.3.2 , still same.

VM config on PVE1:

[/etc/pve/lxc]# cat 118.conf
arch: amd64
cores: 4
features: keyctl=1,nesting=1
hostname: dns2.mylab.com
lock:
memory: 4096
nameserver: 1.1.1.1 8.8.8.8
net0: name=eth0,bridge=vmbr0,gw=10.0.9.1,hwaddr=BC:24:11:EF:7E:E2,ip=10.0.9.9/24,tag=9,type=veth
onboot: 1
ostype: debian
parent: vzdump
rootfs: VMData1:subvol-118-disk-0,size=8G
searchdomain: mylab.com
swap: 512
tags:
unprivileged: 1

The CT is just a pihole, only has 3GB used of 8, dump is 3GB-

df -

 

[Kingneutron]

Did you do any searching?

https://search.brave.com/search?q=p...summary=1&conversation=0b99238ee46127c7ca84d3

 

[wahmed]

No space left does not always mean literal no space left. You will see the message when restore process is unable to write to the disk. This can happen if Container option has not been selected for the storage. Verify that both Disk Image and Container options are selected for the storage from Datacenter > Storage.

 

[totalimpact]

Sorry, I did do some searching, didnt mention that I checked the inodes, I dont see any excessive usage - especially since I have no other CTs running. and I have tried all of these storage locations, including a new SSD, same result.

@wahmed If CT is not enabled on the storage it wont be offered in the Restore list.

 

[totalimpact]

I just tried creating a new CT on PVE2 and it works fine, boots I can login and make files. I just cannot restore one.

 

[waltar]

Take a look here as increasing the ct resolve a no space on device error during restore

B

[SOLVED] Post in thread 'LXC Restore - no space left on device'

Oct 6, 2023

TL;DR: Solved, issue found:

* Storage A compresses more than storage B. VM's filesystem is actually larger than bounds of the disk but fit (on storage A) since it compresses well
* Storage B compresses less (or not at all), running out of space in the disk image it creates that is now too small to fit the un-/insufficiently compressed filesystem.
* Manual CLI restore or do magic below to increase disk size & restore again
* greeat success

---

I've figured this out (apologies should this overlap with what has already been said above): I kept wondering how, or why, it would run...

• bossi

• 

 

[totalimpact]

Take a look here as increasing the ct resolve a no space on device error during restore

B

[SOLVED] Post in thread 'LXC Restore - no space left on device'

Oct 6, 2023

TL;DR: Solved, issue found:

* Storage A compresses more than storage B. VM's filesystem is actually larger than bounds of the disk but fit (on storage A) since it compresses well
* Storage B compresses less (or not at all), running out of space in the disk image it creates that is now too small to fit the un-/insufficiently compressed filesystem.
* Manual CLI restore or do magic below to increase disk size & restore again
* greeat success

---

I've figured this out (apologies should this overlap with what has already been said above): I kept wondering how, or why, it would run...

• bossi

• 

Yes, thanks Waltar, that was the problem, the source was on ZFS compressed storage.... It was a 8GB disk, showed 4.5GB used, but on the new non-zfs storage it had 9GB used.

I adjusted the restore rootfs size to 12GB:

pct restore 218 vzdump-lxc-118-2024_12_28-17_32_14.tar.zst --rootfs 12 --storage TempSSD --unique --hostname dns2.mydomain.com

The proper solution would have been to expand the rootfs to 12GB, back it up again and then restore from GUI.