Can't create LXC on ZFS pool

tycoonbob

Member
Aug 25, 2014
67
0
6
Fresh install of PVE 4.2:
Code:
root@jormungandr:~# pveversion --verbose
proxmox-ve: 4.2-48 (running kernel: 4.4.6-1-pve)
pve-manager: 4.2-2 (running version: 4.2-2/725d76f0)
pve-kernel-4.4.6-1-pve: 4.4.6-48
lvm2: 2.02.116-pve2
corosync-pve: 2.3.5-2
libqb0: 1.0-1
pve-cluster: 4.0-39
qemu-server: 4.0-72
pve-firmware: 1.1-8
libpve-common-perl: 4.0-59
libpve-access-control: 4.0-16
libpve-storage-perl: 4.0-50
pve-libspice-server1: 0.12.5-2
vncterm: 1.2-1
pve-qemu-kvm: 2.5-14
pve-container: 1.0-62
pve-firewall: 2.0-25
pve-ha-manager: 1.0-28
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u1
lxc-pve: 1.1.5-7
lxcfs: 2.0.0-pve2
cgmanager: 0.39-pve1
criu: 1.6.0-1
zfsutils: 0.6.5-pve9~jessie

Created a new ZFS pool:
Code:
root@jormungandr:~# zfs list
NAME                   USED  AVAIL  REFER  MOUNTPOINT
rpool                 9.24G   216G    96K  /rpool
rpool/ROOT             756M   216G    96K  /rpool/ROOT
rpool/ROOT/pve-1       756M   216G   756M  /
rpool/data              96K   216G    96K  /rpool/data
rpool/swap            8.50G   224G    64K  -
tank_vm_01             719M   860G    19K  /tanks/tank_vm_01
tank_vm_01/kvm          19K   860G    19K  /tanks/tank_vm_01/kvm
tank_vm_01/lxc          19K   860G    19K  /tanks/tank_vm_01/lxc
tank_vm_01/templates   719M   860G   719M  /tanks/tank_vm_01/templates
root@jormungandr:~# zpool list
NAME         SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
rpool        232G   758M   231G         -     0%     0%  1.00x  ONLINE  -
tank_vm_01   888G   719M   887G         -     0%     0%  1.00x  ONLINE  -
root@jormungandr:/tanks# zpool status
  pool: rpool
state: ONLINE
  scan: scrub repaired 0 in 0h0m with 0 errors on Thu May  5 16:24:43 2016
config:

    NAME        STATE     READ WRITE CKSUM
    rpool       ONLINE       0     0     0
     mirror-0  ONLINE       0     0     0
       sda2    ONLINE       0     0     0
       sdd2    ONLINE       0     0     0

errors: No known data errors

  pool: tank_vm_01
state: ONLINE
  scan: scrub repaired 0 in 0h0m with 0 errors on Thu May  5 16:24:23 2016
config:

    NAME        STATE     READ WRITE CKSUM
    tank_vm_01  ONLINE       0     0     0
     mirror-0  ONLINE       0     0     0
       sdb     ONLINE       0     0     0
       sdc     ONLINE       0     0     0

errors: No known data errors
root@jormungandr:~# lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0 232.9G  0 disk
├─sda1   8:1    0  1007K  0 part
├─sda2   8:2    0 232.9G  0 part
└─sda9   8:9    0     8M  0 part
sdb      8:16   0 894.3G  0 disk
├─sdb1   8:17   0 894.3G  0 part
└─sdb9   8:25   0     8M  0 part
sdc      8:32   0 894.3G  0 disk
├─sdc1   8:33   0 894.3G  0 part
└─sdc9   8:41   0     8M  0 part
sdd      8:48   0 232.9G  0 disk
├─sdd1   8:49   0  1007K  0 part
├─sdd2   8:50   0 232.9G  0 part
└─sdd9   8:57   0     8M  0 part
zd0    230:0    0     8G  0 disk [SWAP]

Created storage through PVE GUI:
Code:
root@jormungandr:~# cat /etc/pve/storage.cfg
dir: local
    disable
    path /var/lib/vz
    content iso
    maxfiles 0

zfspool: local-zfs
    disable
    pool rpool/data
    content images
    sparse

zfspool: zfs_lxc
    pool tank_vm_01/lxc
    content rootdir

zfspool: zfs_kvm
    pool tank_vm_01/kvm
    content images

dir: templates
    path /tanks/tank_vm_01/templates
    content vztmpl,iso
    maxfiles 0

root@jormungandr:~#

And this is the error I am getting when trying to create a new LXC through the GUI.
Code:
mount: special device /tank_vm_01/lxc/subvol-101-disk-1 does not exist
mounting container failed
TASK ERROR: command 'mount -o bind /tank_vm_01/lxc/subvol-101-disk-1 /var/lib/lxc/101/rootfs/' failed: exit code 32

Something else; I can create a VM using my ZFS dataset (tank_vm_01/kvm), but I notice the vm doesn't have a mountpoint. Is that expected behavior?
Code:
root@jormungandr:/tanks# zfs list
NAME                           USED  AVAIL  REFER  MOUNTPOINT
rpool                         9.24G   216G    96K  /rpool
rpool/ROOT                     756M   216G    96K  /rpool/ROOT
rpool/ROOT/pve-1               756M   216G   756M  /
rpool/data                      96K   216G    96K  /rpool/data
rpool/swap                    8.50G   224G    64K  -
tank_vm_01                    8.95G   851G    19K  /tanks/tank_vm_01
tank_vm_01/kvm                8.25G   851G    19K  /tanks/tank_vm_01/kvm
tank_vm_01/kvm/vm-102-disk-1  8.25G   860G     8K  -
tank_vm_01/lxc                  19K   851G    19K  /tanks/tank_vm_01/lxc
tank_vm_01/templates           719M   851G   719M  /tanks/tank_vm_01/templates

Thanks!!
 
Something else; I can create a VM using my ZFS dataset (tank_vm_01/kvm), but I notice the vm doesn't have a mountpoint. Is that expected behavior?
Code:
root@jormungandr:/tanks# zfs list
NAME                           USED  AVAIL  REFER  MOUNTPOINT
rpool                         9.24G   216G    96K  /rpool
rpool/ROOT                     756M   216G    96K  /rpool/ROOT
rpool/ROOT/pve-1               756M   216G   756M  /
rpool/data                      96K   216G    96K  /rpool/data
rpool/swap                    8.50G   224G    64K  -
tank_vm_01                    8.95G   851G    19K  /tanks/tank_vm_01
tank_vm_01/kvm                8.25G   851G    19K  /tanks/tank_vm_01/kvm
tank_vm_01/kvm/vm-102-disk-1  8.25G   860G     8K  -
tank_vm_01/lxc                  19K   851G    19K  /tanks/tank_vm_01/lxc
tank_vm_01/templates           719M   851G   719M  /tanks/tank_vm_01/templates

Thanks!!

yes, because for Qemu/KVM we use ZVOLs, and the VM can use whatever filesystem inside ;)
 
The driver uses ZFS standard mount path, so it expects to find the data at /tank_vm_01/lxc/
I think I understand. So since my dataset (tank_vm_01/lxc) is not mounted at /tank_vm_01/lxc (currently mounted at /tanks/tank_vm_01/lxc), it isn't working. It's specifically looking for the path /tank_vm_01/lxc, and not just the dataset.

Why is this different behavior than KVM? A QEMU/KVM instance can use a designated zpool, regardless of mount path.
At the least, you'd think there would be a check when creating a zfs storage and setting it to containers, that if it wasn't mounted to the standard mount path, that it'd throw an error.

yes, because for Qemu/KVM we use ZVOLs, and the VM can use whatever filesystem inside ;)
Cool, thanks...that makes sense.
 
Last edited: