IOMMU with ZFS

[To0wnn]

Hi,

When i enble IOMMU to cmdline and reboot it cannot mount my ZFS pool to start proxmox.
If removed from command line everything is working again.

This is the error:


Any suggestions?

 

[oguz]

hi,

what kernel command line options did you use?

have you tried with iommu=pt option in your cmdline as well?

 

[To0wnn]

Yes i did
this is what i have

root=ZFS=rpool/ROOT/pve-1 boot=zfs rootdelay=20 quiet intel_iommu=on iommu=pt

 

[oguz]

* is VT-d enabled in your BIOS settings?

* if you remove quiet you should also see if there are any error messages while booting

* why did you add rootdelay?

* also it might be possible to use the zfs cache to import the device instead of scanning for it

 

[To0wnn]

Server Dell R740

* is VT-d enabled in your BIOS settings?
This is enabled
Virtualization Technology
I/OAT DMA Engine
SR-IOV Global Enable

* if you removequietyou should also see if there are any error messages while booting
will check

* why did you add rootdelay?
did find that on the internet and tried it hooped with delay it would find it

* also it might be possible to use the zfs cache to import the device instead of scanning for it
it looked ok because i saw the foldes and such so not really sure. only ssh was accessible

 

[LnxBil]

* why did you add rootdelay?
did find that on the internet and tried it hooped with delay it would find it

I also need that with an even higher number, so that my HBAs initialize all my disks correctly.

 

[LnxBil]

View attachment 38178

This implied only that you have readed systemd, so you will not run into initrd problems. What if you just try to import your pool manually? Please post error message and last entried of kernel ring buffer (dmesg).

 

[To0wnn]

echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
this is the issue somehow

 

[oguz]

echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
this is the issue somehow

what do you mean?

setting these options causes the issue? or removing them?

where did you see if any error message?

please elaborate...

 

[To0wnn]

if i add this it will not mount the zfs automatically.

but i try to forward the gpu to my windows machine.
That is working now but get the code 43 on my windows machine

 

[oguz]

if i add this it will not mount the zfs automatically.

okay, so the boot problem goes away if you disable unsafe interrupts...

but i try to forward the gpu to my windows machine.
That is working now but get the code 43 on my windows machine

that's something else, probably better to open another thread for it with:
* gpu model, output of lspci -nnk | grep -i vga -C 3
* vfio config files
* qm config VMID of your VM