'zfs destroy' of single disk killed all VEs

RollMops

Member
Jul 17, 2014
30
0
6
with disk-space having become short, I wanted to clean up my server a bit. I deleted a virtual disk, that was not actually being used (VE1100 uses subvol-1100-disk-2) and could not be deleted via GUI, so I did a:

Code:
zfs destroy -f rpool/subvol-1100-disk-1

as a result all other subvolumes sort of vanished, no VE could be started, cuz the files of the subvolumes/disks kinda disappeared, e.g.:
Code:
lxc-start -lDEBUG -o logme.log -F -n 222
cannot open directory //rpool/subvol-222-disk-1: No such file or directory
lxc-start: conf.c: run_buffer: 347 Script exited with status 2
lxc-start: start.c: lxc_init: 450 Failed to run lxc.hook.pre-start for container "222".
lxc-start: start.c: __lxc_start: 1313 Failed to initialize container "222".
lxc-start: tools/lxc_start.c: main: 360 The container failed to start.
lxc-start: tools/lxc_start.c: main: 364 Additional information can be obtained by setting the --logfile and --logpriority options.
I had to restore from backup to make the VEs work again. This is just a test server and I am glad this didn't happen on my production system.

Posting this JFYI, dunno if this is a bug.
 

dietmar

Proxmox Staff Member
Staff member
Apr 28, 2005
16,803
408
103
Austria
www.proxmox.com
as a result all other subvolumes sort of vanished, no VE could be started

That is only possible if you use linked clones? In that case it is a bad idea to force removal of the base volume using the force flag. But this is not a zfs problem.
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
5,176
849
163
That is only possible if you use linked clones? In that case it is a bad idea to force removal of the base volume using the force flag. But this is not a zfs problem.

even then ZFS does not remove a dataset with children unless you pass -r (to destroy snapshots) or -R (to destroy snapshots and clones).. -f is just for force unmounting the dataset.

if you want to find out what happened, "zpool history rpool" will probably give you the answer..
 

RollMops

Member
Jul 17, 2014
30
0
6
...linked clones?
No idea - it's a simple, standard proxmox VE 4.3-12 w. some containers :)

@fabian:
Code:
root@pve:~# zpool history rpool
...
2016-12-12.12:17:12 zfs destroy -r rpool/subvol-666-disk-2
2016-12-12.12:17:26 zfs destroy -r rpool/vm-555-disk-1
2016-12-12.12:17:34 zfs destroy -r rpool/vm-777-disk-1
2016-12-12.12:38:02 zfs destroy -f rpool/subvol-1100-disk-1
2016-12-12.12:42:29 zpool import -N rpool
2016-12-12.12:50:19 zpool import -N rpool
2016-12-12.12:55:53 zfs create -o acltype=posixacl -o xattr=sa -o refquota=83886080k rpool/subvol-222-disk-2
2016-12-12.12:56:08 zfs destroy -r rpool/subvol-222-disk-1
2016-12-12.13:01:21 zfs create -o acltype=posixacl -o xattr=sa -o refquota=260046848k rpool/subvol-100-disk-2
2016-12-12.13:01:32 zfs destroy -r rpool/subvol-100-disk-1
2016-12-12.13:08:33 zfs create -o acltype=posixacl -o xattr=sa -o refquota=104857600k rpool/subvol-333-disk-2
2016-12-12.13:08:41 zfs destroy -r rpool/subvol-333-disk-1
2016-12-12.13:17:23 zfs create -o acltype=posixacl -o xattr=sa -o refquota=8388608k rpool/subvol-444-disk-2
2016-12-12.13:17:41 zfs destroy -r rpool/subvol-444-disk-1
2016-12-12.13:39:49 zfs create -o acltype=posixacl -o xattr=sa -o refquota=335544320k rpool/subvol-1100-disk-1
2016-12-12.13:39:54 zfs destroy -r rpool/subvol-1100-disk-2
2016-12-12.14:00:41 zfs create -o acltype=posixacl -o xattr=sa -o refquota=8388608k rpool/subvol-555-disk-1
2016-12-12.14:01:53 zfs destroy -r rpool/subvol-555-disk-1

...where rpool/subvol-1100-disk-1 in line #1 is the one I intentionally killed via console (bc I couldn't via GUI, option ghosted out).
100, 222, 333, 444 and 1100 "vanished" - I didn't ask to kill them, neither via console, nor via GUI.
Initially I tried to remove 555, 666 and 777 via GUI which didn't work at first, not even after reboot. Destroy generated errors. After I restored 100, 222, 333, 444 and 1100 from backup, also 555, 666 and 777 went away, afair without again doing anything to remove them.

~R.
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
5,176
849
163
well the log shows that you destroyed the 555 666 and 777 volumes (probably with "remove disk"), then manually destroyed the 1100-disk-1 volume, and then created and destroyed the other disks (probably an attempted and failed restore from backup)
 

RollMops

Member
Jul 17, 2014
30
0
6
I reckon the "destructions" of 100, 222, 333, 444 and 1100 happened automatically, when I restored them from backup which re-created the disks/subvolumes. I never ordered to destroy these. Restore happened only once for each VE and without errors.
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
5,176
849
163
ah sorry, I misread. yes, the destructions were destructions of the old volumes after the restore was done! maybe the datasets where unmounted after the manual deletion? I see two imports in the history, did you manually import the pool? did you reboot?
 

RollMops

Member
Jul 17, 2014
30
0
6
except for the "zfs destroy -f rpool/subvol-1100-disk-1" I did nothing manually. Yes, rebooted twice.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!