FreeNAS/FreeBSD hot-plugging of disks not working

This sounds reasonable since virtio-scsi-single uses a controller for each disk which implies adding and removing a PCIe controller each time a disk is hot-plugged. I think adding or removing a controller to/from a running system is not supported.

yes this is the reason I asked to check if the SCSI controller is of type virtio and not virtio-single

since FreeBSD has a very clean output when detecting device we can see it the boot log very clearly:

here three disks using VirtIO SCSI

(da0:vtscsi0:0:0:0): UNMAPPED
(da1:vtscsi0:0:0:1): UNMAPPED
(da2:vtscsi0:0:0:2): UNMAPPED

here three disks using VirtIO SCSI single
we notice the different SCSI controllers:

(da0:vtscsi0:0:0:0): UNMAPPED
(da1:vtscsi1:0:0:1): UNMAPPED
(da2:vtscsi2:0:0:2): UNMAPPED

@BloodyIron:

do you see any kernel activity after pluggin the disk ?
what the last lines of 'dmesg' after adding a disk ?
 
Sorry I disappeared folks, the last month+ has been busy for me in all the right kinda ways. I'm going to come back soon when I get a chance to properly respond and test, but that's not this moment in time. Thanks for the help so far! I hope we get this figured out, and sorry for my delays :D
 
Okay, so the controller is set to VirtIO SCSI, not VirtIO SCSI Single...

The disks I am adding are not actual disks, they are qcow2 files to help me lab this, unsure if that changes anything.

When I add the disks to the VM, I'm tail'ing /var/run/dmesg.boot and I don't see any changes, nor do any new devices show up in /dev

I've tried doing this two ways, differing when I add the disk:
  1. Bus VirtIO
  2. Bus SCSI
Neither of these options result in the disks showing up while FreeNAS is running. They only show up after rebooting the VM.

Still unsure why this is happening. Please advise.
 
Switching the OS to "other" and the cpu arch to Default (kvm64) and disabling NUMA did not "fix" this. Drives still not showing up as added until reboot.
 
Okay I spun up a new VM with FreeNAS 10-beta, which is IIRC based on FreeBSD 10.3. I attached the disks in the same way, trying SCSI/VirtIO, and neither one is detected as hot-swappable. The controller is VirtIO SCSI, not VirtIO SCSI Single.

So I'm very confident this is a ProxmoxVE or something like that issue. :(
 
Oops, forgot to mention, also:

Code:
proxmox-ve: 4.4-80 (running kernel: 4.4.40-1-pve)
pve-manager: 4.4-12 (running version: 4.4-12/e71b7a74)
pve-kernel-4.4.35-1-pve: 4.4.35-77
pve-kernel-4.2.6-1-pve: 4.2.6-36
pve-kernel-4.4.35-2-pve: 4.4.35-79
pve-kernel-4.4.21-1-pve: 4.4.21-71
pve-kernel-4.4.24-1-pve: 4.4.24-72
pve-kernel-4.2.2-1-pve: 4.2.2-16
pve-kernel-4.4.19-1-pve: 4.4.19-66
pve-kernel-4.4.40-1-pve: 4.4.40-80
lvm2: 2.02.116-pve3
corosync-pve: 2.4.2-1
libqb0: 1.0-1
pve-cluster: 4.0-48
qemu-server: 4.0-109
pve-firmware: 1.1-10
libpve-common-perl: 4.0-92
libpve-access-control: 4.0-23
libpve-storage-perl: 4.0-74
pve-libspice-server1: 0.12.8-1
vncterm: 1.3-1
pve-docs: 4.4-3
pve-qemu-kvm: 2.7.1-3
pve-container: 1.0-94
pve-firewall: 2.0-33
pve-ha-manager: 1.0-40
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u3
lxc-pve: 2.0.7-3
lxcfs: 2.0.6-pve1
criu: 1.6.0-1
novnc-pve: 0.5-8
smartmontools: 6.5+svn4324-1~pve80
zfsutils: 0.6.5.9-pve15~bpo80
 
> I attached the disks in the same way, trying SCSI/VirtIO, and neither one is detected as hot-swappable.

from where do you get this information ? how should a hot swappable disk be displayed

adding remove/disks work with FreeBSD 11 and all KVM guests, I am pretty sure it should work with FreeNAS 10 too
maybe you should bring your issue to the FreeNAS forums
 
I'm watching DMESG and the FreeNAS GUI. I'm confident this is not an issue on their end. When I attach a disk to the VM DMESG does not output any new entries. When using bare metal it would.

> I attached the disks in the same way, trying SCSI/VirtIO, and neither one is detected as hot-swappable.

from where do you get this information ? how should a hot swappable disk be displayed

adding remove/disks work with FreeBSD 11 and all KVM guests, I am pretty sure it should work with FreeNAS 10 too
maybe you should bring your issue to the FreeNAS forums
 
lol I got an alert that Vito Reiter posted in here telling me to not run FreeNAS in a VM, and now the post is gone? perhaps they saw that I'm doing this for labbing purposes? :p
 
Yeah I did, just looking out for the community haha. No reason to call me out publicly though ;( I'll learn to read thoroughly prior to posting now.

Thanks for the PR,
Vito Reiter
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!