Failed pci passthrough attempt: stuck at initramfs prompt.

[fuzzyduck]

Hello,

I wanted to add a GPU to my box and decided to take the plunge to attempt to enable passthrough and it failed. Im on the latest 6.4 proxmox.

My steps:
1.

added intel_iommu=on to nano /etc/default/grub making it GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"

updating grub gave me:

root@pve:~# update-grub
Generating grub configuration file ...
W: This system is booted via proxmox-boot-tool:
W: Executing 'update-grub' directly does not update the correct configs!
W: Running: 'proxmox-boot-tool refresh'

Copying and configuring kernels on /dev/disk/by-uuid/B939-AD79
        Copying kernel and creating boot-entry for 5.4.128-1-pve
        Copying kernel and creating boot-entry for 5.4.143-1-pve
Copying and configuring kernels on /dev/disk/by-uuid/B93A-0225
        Copying kernel and creating boot-entry for 5.4.128-1-pve
        Copying kernel and creating boot-entry for 5.4.143-1-pve
Found linux image: /boot/vmlinuz-5.4.143-1-pve
Found initrd image: /boot/initrd.img-5.4.143-1-pve
Found linux image: /boot/vmlinuz-5.4.128-1-pve
Found initrd image: /boot/initrd.img-5.4.128-1-pve
Found linux image: /boot/vmlinuz-5.4.106-1-pve
Found initrd image: /boot/initrd.img-5.4.106-1-pve
Found memtest86+ image: /ROOT/pve-1@/boot/memtest86+.bin
Found memtest86+ multiboot image: /ROOT/pve-1@/boot/memtest86+_multiboot.bin
done

2.

then i added

 vfio
 vfio_iommu_type1
 vfio_pci
 vfio_virqfd

to /etc/modules

which gave me:

root@pve:~# update-initramfs -u -k all
update-initramfs: Generating /boot/initrd.img-5.4.143-1-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/B939-AD79
        Copying kernel and creating boot-entry for 5.4.128-1-pve
        Copying kernel and creating boot-entry for 5.4.143-1-pve
Copying and configuring kernels on /dev/disk/by-uuid/B93A-0225
        Copying kernel and creating boot-entry for 5.4.128-1-pve
        Copying kernel and creating boot-entry for 5.4.143-1-pve
update-initramfs: Generating /boot/initrd.img-5.4.128-1-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/B939-AD79
        Copying kernel and creating boot-entry for 5.4.128-1-pve
        Copying kernel and creating boot-entry for 5.4.143-1-pve
Copying and configuring kernels on /dev/disk/by-uuid/B93A-0225
        Copying kernel and creating boot-entry for 5.4.128-1-pve
        Copying kernel and creating boot-entry for 5.4.143-1-pve
update-initramfs: Generating /boot/initrd.img-5.4.106-1-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/B939-AD79
        Copying kernel and creating boot-entry for 5.4.128-1-pve
        Copying kernel and creating boot-entry for 5.4.143-1-pve
Copying and configuring kernels on /dev/disk/by-uuid/B93A-0225
        Copying kernel and creating boot-entry for 5.4.128-1-pve
        Copying kernel and creating boot-entry for 5.4.143-1-pve
root@pve:~#

Then i rebooted. Now im stuck on a blinking cursor after the disk initialisation i see every time as seen here. The cursor is blinking in the last row..

imgpaste.net/image/KJr6y2

EDIT: After inserting usb keyboard i noticed in fact that im on a (initramfs) prompt

Anyway to undo/fix this?

 

[fuzzyduck]

I have yet to reboot a second time.

I have raid1 zfs as boot and a zfs 4 disk raid10 as storage.

I might have choosen the wrong file to edit since I have a bios and NOT uefi.

/etc/kernel/cmdline contained 1 line regarding zfs pools I think

 

[fuzzyduck]

i rebooted and tried the older kernel .128 but same problem. update-initramfs -u -k all -k all flag broke both kernels?

any way to undo my actions?

 

[kromberg]

Unfortunately you updated ALL the initrd modules. It is good practice to make a copy of the current one and then only update it.

You might be able to boot off a rescue disk, mount the proxmox root and boot filesystems, make the needed updates, chroot to the mounted proxmox filesystem, rebuild the initrd, and pray it works.

 

[fuzzyduck]

thank you for your reponse! that sounds like pretty tough. Is there some guide available about this? Its all new stuff for me to digest.

Ill go and make me a 6.4 bootable usb stick.

 

[fuzzyduck]

Im not sure if its related to my topic at all anymore. Maybe update-initramfs -u -k all broke more then i ever imagined, even a zpool on secondary disks?

I found out the following:
1. disconnecting 4 sata disks(software zfs raid10) makes proxmox boot correctly.
2. reconnecting them lands on initramfs prompt.
3. reinstalling proxmox (7.1) --> no difference. (!!!!!!!!!)

2 questions:

1.Why is the bootsequence of the bootdisk with a fresh proxmox 7.1 install (2x ssd zfs raid1) dependent on ( the absence of ) those disks to boot correctly? What's it doing there. Reading a (corrupt?) Pool?

2. I booted to Ubuntu Live cd and im trying to mount the zpool but zpool list shows no pools available. How do i search for pools on the the disks? all 4 are visible using lsblk