Hi all,
As the title states, I have a 2nd NIC on my HP ProLiant DL360p Gen8 server (The first NIC is a quad 1Gbps NIC). Upon getting my hands on the NIC, I installed Proxmox 7.1-7 and ran the following script I wrote:
Upon rebooting and uploading the ISO to the VM, I added the NICs after configuring the VMs hardware:
However, upon attempting to boot the OPNsense VM, I got the following error:
It should be noted that I tried booting the VM with "All Functions" both enabled and disabled. The end result was the same. I checked
As seen above,
Can I request your assistance in solving this? I've been dealing with this issue for well over several hours now and I can't seem to solve it.
Thanks in advance!
As the title states, I have a 2nd NIC on my HP ProLiant DL360p Gen8 server (The first NIC is a quad 1Gbps NIC). Upon getting my hands on the NIC, I installed Proxmox 7.1-7 and ran the following script I wrote:
Bash:
#!/bin/bash
# Configure apt sources and install necessary packages:
/usr/bin/echo -e "\n# not recommended for production use\ndeb http://download.proxmox.com/debian/pve bullseye pve-no-subscription" >> /etc/apt/sources.list
/usr/bin/sed -e '/deb/ s/^#*/#/' -i /etc/apt/sources.list.d/pve-enterprise.list
/usr/bin/apt update && /usr/bin/apt install -y curl git htop neofetch tmux wget zsh
# Change shell to zsh:
/usr/bin/chsh -s /usr/bin/zsh
# Delete local-lvm:
/usr/sbin/pvesm remove local-lvm
/usr/sbin/lvremove /dev/pve/data
/usr/sbin/lvresize -l +100%FREE /dev/pve/root
/usr/sbin/resize2fs /dev/mapper/pve-root
# Install PVE Discord Dark Theme:
/usr/bin/bash <(curl -s https://raw.githubusercontent.com/Weilbyte/PVEDiscordDark/master/PVEDiscordDark.sh ) install
# Install oh-my-zsh:
/usr/bin/sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)" "" --unattended
/usr/bin/git clone https://github.com/zsh-users/zsh-autosuggestions ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions
/usr/bin/git clone https://github.com/zsh-users/zsh-completions ${ZSH_CUSTOM:-${ZSH:-~/.oh-my-zsh}/custom}/plugins/zsh-completions
/usr/bin/wget https://gitlab.com/Arszilla/dotfiles/-/raw/main/etc/skel/.zshrc -O .zshrc
# Delete unnecessary lines in .zshrc:
/usr/bin/sed -i '18d;21,24d' /root/.zshrc
# Add GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt" to /etc/default grub:
/usr/bin/sed -i 's/^\(GRUB_CMDLINE_LINUX_DEFAULT\)=.*$/\1="quiet intel_iommu=on iommu=pt"/' /etc/default/grub
# Add the necessary lines to /etc/modules:
echo -e "\nvfio\nvfio_iommu_type1\nvfio_pci\nvfio_virqfd\n" >> /etc/modules
# Disable the HP ProLiant iLO4 driver:
echo "blacklist hpwdt" >> /etc/modprobe.d/blacklist-hp.conf
# Update initramfs:
/usr/sbin/update-initramfs -k all -u
# Update GRUB:
/usr/sbin/update-grubUpon rebooting and uploading the ISO to the VM, I added the NICs after configuring the VMs hardware:
However, upon attempting to boot the OPNsense VM, I got the following error:
Code:
kvm: -device vfio-pci,host=0000:07:00.0,id=hostpci0.0,bus=pci.0,addr=0x10.0,multifunction=on: vfio 0000:07:00.0: failed to setup container for group 33: Failed to set iommu for container: Operation not permitted
TASK ERROR: start failed: QEMU exited with code 1It should be noted that I tried booting the VM with "All Functions" both enabled and disabled. The end result was the same. I checked
dmesg | grep -e DMAR -e IOMMU and saw the following
Code:
# dmesg | grep -e DMAR -e IOMMU
[ 0.008104] ACPI: DMAR 0x00000000BDDAD200 000558 (v01 HP ProLiant 00000001 \xd2? 0000162E)
[ 0.008164] ACPI: Reserving DMAR table memory at [mem 0xbddad200-0xbddad757]
[ 0.160223] DMAR: IOMMU enabled
[ 0.402359] DMAR: Host address width 46
[ 0.402360] DMAR: DRHD base: 0x000000fbefe000 flags: 0x0
[ 0.402366] DMAR: dmar0: reg_base_addr fbefe000 ver 1:0 cap d2078c106f0462 ecap f020fe
[ 0.402368] DMAR: DRHD base: 0x000000f3ffe000 flags: 0x1
[ 0.402372] DMAR: dmar1: reg_base_addr f3ffe000 ver 1:0 cap d2078c106f0462 ecap f020fe
[ 0.402374] DMAR: RMRR base: 0x000000bdffd000 end: 0x000000bdffffff
[ 0.402376] DMAR: RMRR base: 0x000000bdff6000 end: 0x000000bdffcfff
[ 0.402377] DMAR: RMRR base: 0x000000bdf83000 end: 0x000000bdf84fff
[ 0.402378] DMAR: RMRR base: 0x000000bdf7f000 end: 0x000000bdf82fff
[ 0.402379] DMAR: RMRR base: 0x000000bdf6f000 end: 0x000000bdf7efff
[ 0.402380] DMAR: RMRR base: 0x000000bdf6e000 end: 0x000000bdf6efff
[ 0.402381] DMAR: RMRR base: 0x000000000f4000 end: 0x000000000f4fff
[ 0.402383] DMAR: RMRR base: 0x000000000e8000 end: 0x000000000e8fff
[ 0.402384] DMAR: [Firmware Bug]: No firmware reserved region can cover this RMRR [0x00000000000e8000-0x00000000000e8fff], contact BIOS vendor for fixes
[ 0.402451] DMAR: [Firmware Bug]: Your BIOS is broken; bad RMRR [0x00000000000e8000-0x00000000000e8fff]
[ 0.402453] DMAR: RMRR base: 0x000000bddde000 end: 0x000000bdddefff
[ 0.402455] DMAR: ATSR flags: 0x0
[ 0.402458] DMAR-IR: IOAPIC id 10 under DRHD base 0xfbefe000 IOMMU 0
[ 0.402460] DMAR-IR: IOAPIC id 8 under DRHD base 0xf3ffe000 IOMMU 1
[ 0.402461] DMAR-IR: IOAPIC id 0 under DRHD base 0xf3ffe000 IOMMU 1
[ 0.402463] DMAR-IR: HPET id 0 under DRHD base 0xf3ffe000
[ 0.402464] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[ 0.402465] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[ 0.402992] DMAR-IR: Enabled IRQ remapping in xapic mode
[ 0.787528] DMAR: No SATC found
[ 0.787533] DMAR: dmar0: Using Queued invalidation
[ 0.787544] DMAR: dmar1: Using Queued invalidation
[ 0.911779] DMAR: Intel(R) Virtualization Technology for Directed I/O
[ 4.687679] DMAR: DRHD: handling fault status reg 2
[ 4.687758] DMAR: [DMA Write] Request device [04:00.0] PASID ffffffff fault addr bdf7f000 [fault reason 05] PTE Write access is not set
[ 9.687799] DMAR: DRHD: handling fault status reg 102
[ 9.687863] DMAR: [DMA Write] Request device [04:00.0] PASID ffffffff fault addr bdf7f000 [fault reason 05] PTE Write access is not set
[ 14.687835] DMAR: DRHD: handling fault status reg 202
[ 14.687938] DMAR: [DMA Write] Request device [04:00.0] PASID ffffffff fault addr bdf7f000 [fault reason 05] PTE Write access is not set
[ 19.687921] DMAR: DRHD: handling fault status reg 302
[ 19.687998] DMAR: [DMA Write] Request device [04:00.0] PASID ffffffff fault addr bdf7f000 [fault reason 05] PTE Write access is not set
[ 23.301034] vfio-pci 0000:07:00.0: DMAR: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor.
[ 24.688074] DMAR: DRHD: handling fault status reg 402
[ 24.688141] DMAR: [DMA Write] Request device [04:00.0] PASID ffffffff fault addr bdf7f000 [fault reason 05] PTE Write access is not set
[ 29.688158] DMAR: DRHD: handling fault status reg 502
[ 29.688225] DMAR: [DMA Write] Request device [04:00.0] PASID ffffffff fault addr bdf7f000 [fault reason 05] PTE Write access is not set
[ 34.688235] DMAR: DRHD: handling fault status reg 602
[ 34.688300] DMAR: [DMA Write] Request device [04:00.0] PASID ffffffff fault addr bdf7f000 [fault reason 05] PTE Write access is not set
[ 39.688332] DMAR: DRHD: handling fault status reg 702As seen above,
[ 0.160223] DMAR: IOMMU enabled is present. I am unsure why the server is constantly throwing PTE Write access is not set though. Checking dmesg | grep -e vfio I saw the following:
Code:
# dmesg | grep -e vfio
[ 23.301034] vfio-pci 0000:07:00.0: DMAR: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor.Can I request your assistance in solving this? I've been dealing with this issue for well over several hours now and I can't seem to solve it.
Thanks in advance!
Last edited:
