macos Ventura stuck on apple logo (1/3 progress bar) with gpu passthrough

[biskero]

So the issue appears only when I have assign a pci with the gpu.
I tried all kinds of configuration for Bluetooth removing usb entries and dongle (logitech mx).

Also as soon as I remove the pci entry, I can boot into venture via VMWare display.

 

[aldariz]

Have you tried vga/display set to none?
and potentially you have
https://www.nicksherlock.com/2020/11/working-around-the-amd-gpu-reset-bug-on-proxmox/

 

[biskero]

yes, nothing happen, from console I do see anything and on the screen attached to the rx580 nothing comes up so I am
blind on what is happening. Am I doing something wrong? How do I know if it's doing something?

I read it yesterday but that reset won't work for rx580:
"Unfortunately, with my RX 580, this module didn’t solve the reset issue for me, at least with macOS guests."

 

[aldariz]

yes, nothing happen, from console I do see anything and on the screen attached to the rx580 nothing comes up so I am
blind on what is happening. Am I doing something wrong? How do I know if it's doing something?

I read it yesterday but that reset won't work for rx580:
"Unfortunately, with my RX 580, this module didn’t solve the reset issue for me, at least with macOS guests."

Have you tried, as I've got more than one GPU, I never had any issues.
https://www.reddit.com/r/VFIO/comments/dc3mu3/rx580_my_experiences_in_passthrough/
https://github.com/edalquist/proxmox/blob/master/gpu_passthrough.md

 

[biskero]

so if I use the GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt nofb nomodeset video=efifbff",
I had GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"

I get the following error and I won't get to the selection of the macos.
The error gets into a loop!

03:00.0 is the RX580
03:00.1 is the RX580 audio

"
[ +0.000014] vfio-pci 0000:03:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID)
[ +0.001111] vfio-pci 0000:03:00.0: device [1002:67df] error status/mask=00100000/00000000
[ +0.001074] vfio-pci 0000:03:00.0: [20] UnsupReq (First)
[ +0.001070] vfio-pci 0000:03:00.0: AER: TLP Header: 40000008 000003ff b02f4000 00000000
[ +0.001147] pcieport 0000:00:02.0: AER: device recovery successful
[ +0.000003] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.1
[ +0.000007] vfio-pci 0000:03:00.1: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID)
[ +0.001085] vfio-pci 0000:03:00.1: device [1002:aaf0] error status/mask=00100000/00000000
[ +0.001085] vfio-pci 0000:03:00.1: [20] UnsupReq (First)
[ +0.001064] vfio-pci 0000:03:00.1: AER: TLP Header: 40000008 000000ff b02f3000 00000000
[ +0.001104] pcieport 0000:00:02.0: AER: device recovery successful
[ +0.000002] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.1
[ +0.000006] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.1
[ +0.000005] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.1
[ +0.000006] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.1
[ +0.000006] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.1
[ +0.000005] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.1
[ +0.000006] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.1

 

[aldariz]

so if I use the GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt nofb nomodeset video=efifbff",
I had GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"

I get the following error and I won't get to the selection of the macos.
The error gets into a loop!

03:00.0 is the RX580
03:00.1 is the RX580 audio

"
[ +0.000014] vfio-pci 0000:03:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID)
[ +0.001111] vfio-pci 0000:03:00.0: device [1002:67df] error status/mask=00100000/00000000
[ +0.001074] vfio-pci 0000:03:00.0: [20] UnsupReq (First)
[ +0.001070] vfio-pci 0000:03:00.0: AER: TLP Header: 40000008 000003ff b02f4000 00000000
[ +0.001147] pcieport 0000:00:02.0: AER: device recovery successful
[ +0.000003] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.1
[ +0.000007] vfio-pci 0000:03:00.1: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID)
[ +0.001085] vfio-pci 0000:03:00.1: device [1002:aaf0] error status/mask=00100000/00000000
[ +0.001085] vfio-pci 0000:03:00.1: [20] UnsupReq (First)
[ +0.001064] vfio-pci 0000:03:00.1: AER: TLP Header: 40000008 000000ff b02f3000 00000000
[ +0.001104] pcieport 0000:00:02.0: AER: device recovery successful
[ +0.000002] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.1
[ +0.000006] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.1
[ +0.000005] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.1
[ +0.000006] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.1
[ +0.000006] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.1
[ +0.000005] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.1
[ +0.000006] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.1

I think if you add "vfio-pci ids=XXXX:XXXX,YYYY:YYYY initcall_blacklist=sysfb_init textonly"
to get the ids you need to run lspci -n -s 03:00
Update Grub and then reboot Proxmox

 

[biskero]

so I add GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt vfio-pci ids=1002:67df,1002:aaf0 initcall_blacklist=sysfb_init textonly"

it does boot but I need to uncheck ROM and I still need use Display VMWare so it very slow and nothing from Display attached to RX580.
Ventura report can see the RX580, just the resolution is off since I have a 34" display it stays at 1920x1080.

If I set display to none in VM Hardware, I cannot do anything since VMC is not working and Display attached to RX580 is blank.

So looks like a progress but not finished!

 

[biskero]

Here are the info from Ventura about PCI Graphics cards that it see.
So not sure it is using the RX580 but it uses the onboard video?
Not sure but that would explain why I do not see anything on the Display when I use none in VM Hardware since the hdmi cable is attached to the RX580. Cannot attach anything to the motherboard since it does not have a video ouput, is a workstation mb.

Am I correct or just rumbling !?

 

[duxnobis13]

@biskero hi mate, any progress? I have now restarted my hackintosh project. I left it as I did not manage to pass the GPU through. Now with ventura I wanted to give it another go. Installed all as per Nick Sherlocks guide and all worked smooth. I have to say that I am using gpu with another vm where i run w11 and all works like a charm in regard to pass through the gpu. Of course I do not use them on parallel (even though apparently this should somehow work, but I use em separate). I stop one vm and start the other in case. anyway, if I start the ventura vm I reach the select start disk screen, select my ventura installation and get to the load screen with apple logo but nothing moves. I am not sure if in the background ventura actually starts or reaches the login screen, but at least I see nothing but the load screen with apple logo and no progress. Anyone ideas?

Just to add...with my installation it works totally fine with console but stucks at logo screen when trying to run with monitors. just to be on the same page.

I am running AS Rock Z690er Taichi board with an intel i9 12900k, 64GB ram, lots of disk space (nvme ssd's), asus radeon rx 6800 xt 16g., 3x LG UltraGear 27GP850-B Monitors

 

[biskero]

I still have the GPU pass through issue with my RX 580. My guess is that I need to update my GPU since this one is too old.
Trying to figure out which one is the best. Beside that I was able to do several Ventura macos updates without issues. Also
installed TrueNAS and Syncthing. Just need to get the GPU working.

 

[duxnobis13]

Yeah the working in ventura is not an issue. the system works smooth on console. I just need to get the gpu running as I really think it has to do with some silly little tweaking in prox vm. I dont think its an update of GPU thing otherwise I wouldnt have the same issue with a relatively new gpu such as the rx 6800 xt is. I just gave it a new try and always stuck at the progress bar with "NO" progress at all. On console...all smooth. i get crazy

 

[duxnobis13]

I just ended in another thread where someone talks about fixing this with the clocksource change but with me this does not yet works as if I prompt the code in shell I dont get the expected output but a new line with root@...: Will see if I can get some more inputs there. Thanks so far.

 

[ghiamar]

Hi, for me, unchecking the PCI Express check on the GPU entry solved the problem

 

[filippocamillo]

Hello @biskero. Any solution on that issue?

Thank you

 

[asded]

@filippocamillo @biskero follow my process https://forum.proxmox.com/threads/gpu-passthrough-on-proxmox-ve-macos-monterey-part-04x04.140579/

 

[asded]

is for Monterey, but the process remains the same