Hello,
I've recently decided to move from VMware to Proxmox as my server core os. So far I've managed to perfectly recreate my old setup (on vmware) however I encountered an issue whilst trying to setup my NAS.
On my old setup I ran Truenas scale as a VM on Vmware while passing though SSDs to Truenas using a PCI Sata expanssion card. I wanted to recreate this setup on Proxmox however if I launch the truenas VM with the PCI card as a passthrough device the server freezes and I have to reboot it manually.
Hardware wise I'm running a MSI z170a SLI plus with an I7 6700k cpu. Both supports passthrough (It worked on VMware) and the VT-d option is enabled in the bios.
When I'm running this command on proxmox I get this result :
Output:
In my list of PCI devices each device belongs to a unique group, including the Sata expanssion card :
According to the wiki I should be good to go but I keep getting this error.
I tried multiple workarounds. The only one that works is the following :
- Shut down the server ;
- Unplug the drive that is connected to the PCI expanssion card;
- Boot up the server ;
- Boot the the Truenas VM with the expanssion card passed through but without any drive plugged to it ;
- Hot plugging the drive back.
This workaround works pretty well and allows me to access my drive in Truenas. However I have to respect this procedure each time I'm rebooting either Truenas or Proxmox, which is far from ideal.
Does anyone have a lead about the potential issue here?
Thanks a lot!
I've recently decided to move from VMware to Proxmox as my server core os. So far I've managed to perfectly recreate my old setup (on vmware) however I encountered an issue whilst trying to setup my NAS.
On my old setup I ran Truenas scale as a VM on Vmware while passing though SSDs to Truenas using a PCI Sata expanssion card. I wanted to recreate this setup on Proxmox however if I launch the truenas VM with the PCI card as a passthrough device the server freezes and I have to reboot it manually.
Hardware wise I'm running a MSI z170a SLI plus with an I7 6700k cpu. Both supports passthrough (It worked on VMware) and the VT-d option is enabled in the bios.
When I'm running this command on proxmox I get this result :
dmesg | grep -e DMAR -e IOMMUOutput:
Code:
[ 0.006923] ACPI: DMAR 0x000000008A452538 0000A8 (v01 INTEL SKL 00000001 INTL 00000001)
[ 0.006940] ACPI: Reserving DMAR table memory at [mem 0x8a452538-0x8a4525df]
[ 0.049240] DMAR: IOMMU enabled
[ 0.125197] DMAR: Host address width 39
[ 0.125198] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[ 0.125203] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 7e3ff0505e
[ 0.125204] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[ 0.125207] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[ 0.125208] DMAR: RMRR base: 0x0000008a23a000 end: 0x0000008a259fff
[ 0.125209] DMAR: RMRR base: 0x0000008b800000 end: 0x0000008fffffff
[ 0.125211] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 1
[ 0.125212] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[ 0.125213] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.126744] DMAR-IR: Enabled IRQ remapping in x2apic mode
[ 0.383656] DMAR: No ATSR found
[ 0.383657] DMAR: No SATC found
[ 0.383658] DMAR: IOMMU feature fl1gp_support inconsistent
[ 0.383659] DMAR: IOMMU feature pgsel_inv inconsistent
[ 0.383659] DMAR: IOMMU feature nwfs inconsistent
[ 0.383660] DMAR: IOMMU feature eafs inconsistent
[ 0.383661] DMAR: IOMMU feature prs inconsistent
[ 0.383661] DMAR: IOMMU feature nest inconsistent
[ 0.383662] DMAR: IOMMU feature mts inconsistent
[ 0.383662] DMAR: IOMMU feature sc_support inconsistent
[ 0.383663] DMAR: IOMMU feature dev_iotlb_support inconsistent
[ 0.383664] DMAR: dmar0: Using Queued invalidation
[ 0.383665] DMAR: dmar1: Using Queued invalidation
[ 0.385392] DMAR: Intel(R) Virtualization Technology for Directed I/OIn my list of PCI devices each device belongs to a unique group, including the Sata expanssion card :
According to the wiki I should be good to go but I keep getting this error.
I tried multiple workarounds. The only one that works is the following :
- Shut down the server ;
- Unplug the drive that is connected to the PCI expanssion card;
- Boot up the server ;
- Boot the the Truenas VM with the expanssion card passed through but without any drive plugged to it ;
- Hot plugging the drive back.
This workaround works pretty well and allows me to access my drive in Truenas. However I have to respect this procedure each time I'm rebooting either Truenas or Proxmox, which is far from ideal.
Does anyone have a lead about the potential issue here?
Thanks a lot!
Last edited: