I am having a similar issue with a R630 and the X520/I350 rNDC, C63DV and Ubuntu 26.04.1 (kernel 7.0) and Ubuntu 24.04.4 (hwe kernel 7.0).
- disabling the igb kernel module yields good results, but ofc disables usage of the 1G nics of the card
- booting an older kernel with a older igb module yields good results as well, but for obvious reasons that is not a longer term solution
- updating the ROM via intel bootutil to the latest versions did not yield any results (and is officially discouraged for LOMs)
- downloading the X520/I350 update package (21.5.0) from the R640 and flashing it did not yield any results
- fw update + X2APIC on, I/OAT DMA on, UEFI on, and Secure Boot off/on did not yield any results
- fw update + X2APIC on, I/OAT DMA on, UEFI on, and Secure Boot off and SR-IOV Global Enable on actually led to the machine booting fine