Multiple HDDs in VMs

[namron]

Hi,I'm planning on running a home NAS and a firewall as VMs on a proxmox host.The storage for the NAS are 4 HDDs of different types and sizes, which I want to put in the machine running proxmox via SATA. The host itself runs on a seperate SSD.How would I go about making the HDDs available for the NAS VM?Regards,NormanEdit: Formatting doesn't seem to work

 

[t.lamprecht]

https://pve.proxmox.com/wiki/Pci_passthrough

You could try to use PCI pass-through for the HDDs.

With virtio paravirtualization you should also have an acceptable performance with virtual disks, those are also a bit more flexible and you can manage them through the PVE interface.

 

[namron]

With virtio paravirtualization you should also have an acceptable performance with virtual disks, those are also a bit more flexible and you can manage them through the PVE interface.

I plugged in a hdd for testing. fdisk shows it, but how can I access it from the PVE interface to add it as a virtual disk?

 

[t.lamprecht]

mount it somewhere (configure fstab so that it get's mounted automatically), after that go to the PVE interface, to the Datacenter->Storage tab and add a new storage, select directory and entry your mount point.

Then you can add a virtual disk to your NFS VM with the storage location on the added drive. There select virtio for best performance.

 

[namron]

Thank you. That's what I was looking for.What parameters other than virtio should I set when adding the drive? Is RAW disk format alright for my purpose? And what about the cache?

 

[t.lamprecht]

Yes RAW should fit your need.

With cache it's a trade off between security/integrity and performance.
This should give you an overview: https://pve.proxmox.com/wiki/Performance_Tweaks#Disk_Cache

 

[namron]

I also gave PCI passthrough a try but can't get it to work. After editing GRUB and checking with "dmesg | grep -e DMAR -e IOMMU" I get "IOMMU enabled". But when I add "hostpci0: 04:00.0" (which really is my SATA PCI- cards ID) to the node config, the VM doesn" boot anymore, showing this:

TASK ERROR: start failed: command '/usr/bin/kvm -id 101 -chardev 'socket,id=qmp,path=/var/run/qemu-server/101.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/101.vnc,x509,password -pidfile /var/run/qemu-server/101.pid -daemonize -smbios 'type=1,uuid=aea8db38-4c87-4fee-a204-f632ba13b757' -name fileserver -smp '4,sockets=1,cores=4,maxcpus=4' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000' -vga cirrus -cpu kvm64,+lahf_lm,+x2apic,+sep -m 6000 -k de -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'pci-assign,host=04:00.0,id=hostpci0,bus=pci.0,addr=0x10' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:5c42c7db8ec1' -device 'ahci,id=ahci0,multifunction=on,bus=pci.0,addr=0x7' -drive 'file=/var/lib/vz/images/101/vm-101-disk-1.qcow2,if=none,id=drive-sata0,format=qcow2,aio=native,cache=none,detect-zeroes=on' -device 'ide-drive,bus=ahci0.0,drive=drive-sata0,id=sata0,bootindex=100' -drive 'file=/var/lib/vz/template/iso/debian-8.1.0-amd64-netinst.iso,if=none,id=drive-ide2,media=cdrom,aio=native' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -drive 'file=/media/hdd1/images/101/vm-101-disk-2.raw,if=none,id=drive-virtio0,format=raw,aio=native,cache=none,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa' -netdev 'type=tap,id=net0,ifname=tap101i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown' -device 'e1000,mac=3E:86:FD:29:24:35,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300'' failed: exit code 1

PS: any idea why line break doesn't work here for me?

 

[t.lamprecht]

Did you set the machine type to q35?

Something like this should be in your VM's config file.

machine: q35

btw.: no idea why new line isn't working.

 

[namron]

This is the whole config file:

bootdisk: sata0cores: 4ide2: local:iso/debian-8.1.0-amd64-netinst.iso,media=cdrommemory: 6000name: fileservernet0: e1000=3E:86:FD:29:24:35,bridge=vmbr1numa: 0onboot: 1ostype: l26sata0: local:101/vm-101-disk-1.qcow2,format=qcow2,size=12Gsmbios1: uuid=aea8db38-4c87-4fee-a204-f632ba13b757sockets: 1virtio0: hdd1:101/vm-101-disk-2.raw,format=raw,backup=no,size=229G#hostpci0: 04:00.0

 

[t.lamprecht]

Please add the line from my post above to your config and retry.

 

[namron]

Ah, I overlooked something. Here is the beginning of the log:

kvm: -device pci-assign,host=04:00.0,id=hostpci0,bus=pci.0,addr=0x10: PCI region 5 at address 0xd0680000 has size 0x400, which is not a multiple of 4K.  You might experience some performance hit due to that.

kvm: -device pci-assign,host=04:00.0,id=hostpci0,bus=pci.0,addr=0x10: No IOMMU found.  Unable to assign device "hostpci0"

kvm: -device pci-assign,host=04:00.0,id=hostpci0,bus=pci.0,addr=0x10: Device initialization failed.

kvm: -device pci-assign,host=04:00.0,id=hostpci0,bus=pci.0,addr=0x10: Device 'kvm-pci-assign' could not be initialized

 

[namron]

I use an Intel Celeron J1900 CPU. May that be the problem here? Edit: Yes, the J1900 doesn't support VT-d and therefore isn't capable of passing through devices. So I"ll stick to virtio drives .

 

[t.lamprecht]

Ah OK, that makes sense.

Yes, then VirtIO should be the best choice and almost on par with pass-through.

 

[jim.bond.9862]

Why don't you try passing disks only, not the controller but disk.
I do not have the script but when I get home I see if I can find the syntax. What I had done in the past is create a vm with virtual drive for system, than edit the config and pass the drives as scisi drives in vm.

use this comands to get the drives id

lsblk -l
ls -l /dev/disk/by-id
blkid

than edit vm config.
I usually run this command on host when it is runnig
replace the "100" with id of your vm and drive id with yours
the command below will add single drive to scsi0.
it definitely works. but YMMV

qm set 100 -scsi0 /dev/disk/by-id/ata-ST2000DL003-XXXXXXX_5YD3XYXZ

Sent from my phone

 

[namron]

than edit vm config.I usually run this command on host when it is runnig

Can you explain this furtherI get the following error when trying your method:

root@prox:~# qm set 101 -scsi0 /dev/disk/by-id/ata-ST3250820AS_5QE5AVV2

update VM 101: -scsi0 /dev/disk/by-id/ata-ST3250820AS_5QE5AVV2400 Parameter verification failed.

scsi0: hotplug problem - error on hotplug device 'scsihw0'

qm set  [OPTIONS]

EDIT: And why isn't the full space on a mounted folder storage available when creating a RAW image?

 

[jim.bond.9862]

I do not think you can do pass through on web interface. On web gui you can only define and add virtual drives. I was talking about doing a full physical hard drive pass through to vm. Op has problem passing the pci controller in to vm, I think that maybe he can still pass the hard drives separately.
The example I give adds a real physical drive to vm. You can chosse the interface to use, it can be scsi, or sata or virtio or ide. And it is totally independent from what your drive actually using but better to get a related type. Like for sata drive I would use sata or scsi over ide.
In my case I use scsi since it has more devic count limit over sata. Like 16 devices over 6.

Sent from my phone

 

[namron]

Please see my edit above

 

[jim.bond.9862]

Well I guess it was a good try than. The pass through is not working for you. Sorry.
I am not sure what you mean by full space on mounted storage but check your vm controller type. If you are using ide controller the drive image size is limited to 2tb I believe. It ia the limit of ide. Try switching controller type to sata.

Sent from my phone

 

[namron]

Ok, I got it working now. But why would you use iscsi for a sata hdd? Wouldn't it make more sense to passthrough a sata hdd as a sata hdd? And where would I put this line so it is executed at every startup?

 

[jim.bond.9862]

Oh you don't need to execute it at every startup. The line I give you modifies the vm config from host cli and adds the drive permanently to vm. The way you do that is you either type the whole thing in host console on web ui, or ssh into host and do it there.
I am not sure exactly, but when dealing with vm en environment sata and scsi can be interchangeable most of the time when creating drives in vm. In fact you can use anything supported in vm environment you use. For virtual disks I personally like to use virtio or sata. And only if it gives me an issue I fall back on ide.
In the case above I used scsi because I have a set up that can support 24 drives, so I want the room for future expansion if needed. Also I did not use iscsi, it is a regular scsi controller/drive emulation. Iscsi is something else.

Sent from my phone