Hi,
I recently purchased a Terramaster F8 SSD Plus and added 8 NVMe drives, some of them are behind an onboard ASM2806.
I plan to use it as a home server and need pcie-passthrough, therefore VT-d is a must.
Unfortunately, I am not able to start the install process of proxmox if VT-d is enabled in bios.
When starting the installer it freezes at
Proxmox 8.3.2, Kernel 6.8.12-6-pve
What else I have tested:
Thanks in advance!
Proxmox installer boot log:
lspci:
ASM2806: lspci -s 04:00.0 -vvv
I recently purchased a Terramaster F8 SSD Plus and added 8 NVMe drives, some of them are behind an onboard ASM2806.
I plan to use it as a home server and need pcie-passthrough, therefore VT-d is a must.
Unfortunately, I am not able to start the install process of proxmox if VT-d is enabled in bios.
When starting the installer it freezes at
Waiting for /dev to be fully populated ... (see logs below)Proxmox 8.3.2, Kernel 6.8.12-6-pve
What else I have tested:
- Installing with VT-d disabled works perfectly and everything beside pcie-passthrough works great. If I then enable VT-d the system freezes during boot.
- Starting other linux distros also does not work with VT-D enabled
- Installing Truenas Core (BSD), Windows 11 and also the preinstalled OS TOS6 work with VT-d enabled
- In Truenas Scale forums people describe the same issue. They suspect it has to do with the ASM2806 chip.
I have also tried several of the recommended grub kernel parameters mentioned there but was not successful.
Thanks in advance!
Proxmox installer boot log:
Code:
Welcome to the Proxmox VE 8.3 installer
initial setup startup
mounting proc filesystem
mounting sys filesystem
EFI boot mode detected, mounting efivars filesystem
boot comandline: BOOT_IMAGE=/boot/linux26 ro ramdisk_size=16777216 rw quiet splash=silent
loading drivers: nume wmi intel_pmc_core mac_hid acpi_pad video acpi_tad pinctrl_alderlake spi_intel_pci xhci_pci i2c_i801 acpi_cpufreq rtc_cmos efi_pstore serio_raw psmouse pcspkr intel_cstate rapl aesni_intel sha1_ssse3 sha256_ssse3 ghash_clmulni_intel polyval_clmulni crc32_pclmul crct10dif_pclmul kum_intel intel_powerclamp x86_pkg_temp_thermal intel_tcc_cooling intel_rapl_common
modprobe: ERROR: could not insert 'intel_tcc_cooling': No such device
searching for block device containing the ISO proxmox-ve-8.3-1
with ISO ID '20c7412e-a788-11ef-9ee4-4f61d2460cfb'
testing again in 1 seconds
testing device '/dev/sda' for ISO
found Proxmox VE ISO
preparing installer mount points and working environment
switching root from initrd to actual installation system
Starting Proxmox installation
EFI boot mode detected, mounting efivars filesystem
Installing additional hardware drivers
Starting hotplug events dispatcher: systemd-udeud.
Synthesizing the initial hotplug events (subsystems) ... done.
Synthesizing the initial hotplug events (devices) ... done.
Waiting for /dev to be fully populated ...lspci:
Code:
00:00.0 Host bridge: Intel Corporation Device 4617
00:02.0 VGA compatible controller: Intel Corporation Alder Lake-N [UHD Graphics]
00:14.0 USB controller: Intel Corporation Alder Lake-N PCH USB 3.2 xHCI Host Controller
00:14.2 RAM memory: Intel Corporation Alder Lake-N PCH Shared SRAM
00:16.0 Communication controller: Intel Corporation Alder Lake-N PCH HECI Controller
00:1c.0 PCI bridge: Intel Corporation Device 54b8
00:1c.3 PCI bridge: Intel Corporation Device 54bb
00:1c.6 PCI bridge: Intel Corporation Device 54be
00:1d.0 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port
00:1d.2 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port
00:1d.3 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port
00:1f.0 ISA bridge: Intel Corporation Alder Lake-N PCH eSPI Controller
00:1f.3 Audio device: Intel Corporation Alder Lake-N PCH High Definition Audio Controller
00:1f.4 SMBus: Intel Corporation Alder Lake-N SMBus
00:1f.5 Serial bus controller: Intel Corporation Alder Lake-N SPI (flash) Controller
01:00.0 Ethernet controller: Aquantia Corp. AQC113C NBase-T/IEEE 802.3an Ethernet Controller [Marvell Scalable mGig] (rev 03)
02:00.0 Non-Volatile memory controller: Sandisk Corp Device 5049 (rev 01)
03:00.0 Non-Volatile memory controller: Sandisk Corp Device 5049 (rev 01)
04:00.0 PCI bridge: ASMedia Technology Inc. ASM2806 4-Port PCIe x2 Gen3 Packet Switch (rev 01)
05:00.0 PCI bridge: ASMedia Technology Inc. ASM2806 4-Port PCIe x2 Gen3 Packet Switch (rev 01)
05:02.0 PCI bridge: ASMedia Technology Inc. ASM2806 4-Port PCIe x2 Gen3 Packet Switch (rev 01)
05:06.0 PCI bridge: ASMedia Technology Inc. ASM2806 4-Port PCIe x2 Gen3 Packet Switch (rev 01)
05:0e.0 PCI bridge: ASMedia Technology Inc. ASM2806 4-Port PCIe x2 Gen3 Packet Switch (rev 01)
06:00.0 Non-Volatile memory controller: Sandisk Corp Device 5049 (rev 01)
07:00.0 Non-Volatile memory controller: Sandisk Corp Device 5049 (rev 01)
08:00.0 Non-Volatile memory controller: Sandisk Corp Device 5049 (rev 01)
09:00.0 Non-Volatile memory controller: Sandisk Corp Device 5049 (rev 01)
0a:00.0 Non-Volatile memory controller: Sandisk Corp Device 5049 (rev 01)
0b:00.0 Non-Volatile memory controller: Sandisk Corp Device 5049 (rev 01)ASM2806: lspci -s 04:00.0 -vvv
Code:
04:00.0 PCI bridge: ASMedia Technology Inc. ASM2806 4-Port PCIe x2 Gen3 Packet Switch (rev 01) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 16
Bus: primary=04, secondary=05, subordinate=09, sec-latency=0
I/O behind bridge: 0000f000-00000fff [disabled] [32-bit]
Memory behind bridge: 80a00000-80dfffff [size=4M] [32-bit]
Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff [disabled] [64-bit]
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16+ MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [80] Express (v2) Upstream Port, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ SlotPowerLimit 25W
DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 256 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 8GT/s, Width x8, ASPM L0s L1, Exit Latency L0s <4us, L1 <64us
ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; Disabled- CommClk+
ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 8GT/s, Width x2 (downgraded)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis- NROPrPrP- LTR+
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS-
AtomicOpsCap: Routing-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ 10BitTagReq- OBFF Disabled,
AtomicOpsCtl: EgressBlck-
LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis+
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+ EqualizationPhase1+
EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [c0] Subsystem: Device 0000:0000
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 14, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 05000001 0000000f 05080005 00000000
Capabilities: [140 v1] Power Budgeting <?>
Capabilities: [160 v1] Latency Tolerance Reporting
Max snoop latency: 3145728ns
Max no snoop latency: 3145728ns
Capabilities: [180 v1] Multicast
McastCap: MaxGroups 64, ECRCRegen-
McastCtl: NumGroups 1, Enable-
McastBAR: IndexPos 0, BaseAddr 0000000000000000
McastReceiveVec: 0000000000000000
McastBlockAllVec: 0000000000000000
McastBlockUntransVec: 0000000000000000
McastOverlayBAR: OverlaySize 0 (disabled), BaseAddr 0000000000000000
Capabilities: [1c0 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: 0
Capabilities: [200 v1] L1 PM Substates
L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1+ ASPM_L1.2- ASPM_L1.1+ L1_PM_Substates+
L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
L1SubCtl2:
Kernel driver in use: pcieport
Last edited: