Proxmox VE 7.3-4 / Lenovo P300 / GPU Pci passthrough / Nvidia Quadro K2200 / Xeon E3-1241 v3 / MacOS Monterrey - Ventura / Windows 10

D

drx1

New Member
Jan 27, 2023
4
0
1
Hello, I have tried to find a solution in the forums here or on the web but all solutions done by others are not working for my setup.
I am new to Proxmox.

It all started with watching videos on Youtube about how awesome and easy it is to use Proxmox to run a MacOS VM. We all have different hardware that we are working with and this is the issue for someone like me, i cannot just follow the steps others have done from A to Z, I have to either jump steps or add steps for my particular hardware.

Things I have done:
  • I followed the official tutorial --> https://pve.proxmox.com/wiki/Pci_passthrough
    • Enable the IOMMU (systemd-boot, Intel CPU) --> [ 0.043989] DMAR: IOMMU enabled
    • Required Modules
    • IOMMU Interrupt Remapping --> [ 0.125129] DMAR-IR: Enabled IRQ remapping in x2apic mode
    • Verify IOMMU Isolation (All Xeon processor support them (E3,E5)) Installed in the PC is the Xeon E3-1241 v3 -->
      /sys/kernel/iommu_groups/7/devices/0000:00:1c.0 /sys/kernel/iommu_groups/5/devices/0000:00:1a.0 /sys/kernel/iommu_groups/3/devices/0000:00:16.0 /sys/kernel/iommu_groups/3/devices/0000:00:16.3 /sys/kernel/iommu_groups/11/devices/0000:03:00.0 /sys/kernel/iommu_groups/1/devices/0000:00:01.0 /sys/kernel/iommu_groups/1/devices/0000:01:00.0 /sys/kernel/iommu_groups/1/devices/0000:01:00.1 /sys/kernel/iommu_groups/8/devices/0000:00:1c.3 /sys/kernel/iommu_groups/6/devices/0000:00:1b.0 /sys/kernel/iommu_groups/4/devices/0000:00:19.0 /sys/kernel/iommu_groups/2/devices/0000:00:14.0 /sys/kernel/iommu_groups/10/devices/0000:00:1f.2 /sys/kernel/iommu_groups/10/devices/0000:00:1f.0 /sys/kernel/iommu_groups/10/devices/0000:00:1f.3 /sys/kernel/iommu_groups/0/devices/0000:00:00.0 /sys/kernel/iommu_groups/9/devices/0000:00:1d.0
    • Determine your PCI card address, and configure your VM -- >
      01:00.0 VGA compatible controller: NVIDIA Corporation GM107GL [Quadro K2200] (rev a2) 01:00.1 Audio device: NVIDIA Corporation GM107 High Definition Audio Controller [GeForce 940MX] (rev a1)
      Config file -->
      hostpci0: 0000:01:00.0,pcie=1,romfile=NVIDIA.QuadroK2200.4096.140922_1.rom hostpci1: 0000:01:00.1,pcie=1,romfile=NVIDIA.QuadroK2200.4096.140922_1.rom [*]
    • dmesg errors -->
      [ 2180.667640] vfio-pci 0000:01:00.0: vfio_ecap_init: hiding ecap 0x1e@0x258 [ 2180.811649] vfio-pci 0000:01:00.0: vfio_ecap_init: hiding ecap 0x19@0x900 [ 2180.955193] vfio-pci 0000:01:00.0: BAR 3: can't reserve [mem 0xf0000000-0xf1ffffff 64bit pref] [ 2181.099083] resource sanity check: requesting [mem 0x000c0000-0x000dffff], which spans more than PCI Bus 0000:00 [mem 0x000d0000-0x000dffff window] [ 2181.385639] caller pci_map_rom+0x7f/0x1f0 mapping multiple BARs [ 2181.526912] vfio-pci 0000:01:00.0: No more image in the PCI ROM
    • BAR 3: can't reserve [mem] error, added kernel commandline option --> video=efifb:off
    • I removed the ROM file because the guide says : you usually do not need the ROM-file dumping mentioned at the end
    • GPU Passthrough /etc/modprobe.d/vfio.conf --> options vfio-pci ids=10de:13ba,10de:0fbc disable_vga=1
    • Blacklisted the drivers in /etc/modprobe.d/pve-blacklist.conf -->
      blacklist radeon blacklist nouveau blacklist nvidia blacklist nvidiafb
    • UEFI bootable rom, To be UEFI compatible, you need a "type 3" in the result.
      I have used the nvflash command to obtain the rom and rom-parser to verify it -->
      ./rom-parser /usr/share/kvm/NVIDIA.QuadroK2200.4096.140922_1.rom Valid ROM signature found @600h, PCIR offset 190h PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 13ba, class: 030000 PCIR: revision 0, vendor revision: 1 Valid ROM signature found @fc00h, PCIR offset 1ch PCIR: type 3 (EFI), vendor: 10de, device: 13ba, class: 030000 PCIR: revision 3, vendor revision: 0 EFI: Signature Valid, Subsystem: Boot, Machine: X64 Last image
    • If i try to obtain the rom file with the linux echo command, rom-parser shows and error -->
      Error, ran off the end
    • I reflashed a verified and checked GPU rom back on the K2200 just to make sure.
    • USB Passthrough, done, added USB v1.0 keyboard/mouse
    • VT-d enabled in Bios
  • I created a Recovery dmg file for Monterrey
  • I created the MacOS VM with settings from youtube videos, i tried different settings.
  • Installed Monterrey and booted, starts without opencore.iso, detached both .img and .iso, and The VM runs no trouble with Display : VMware compatible,, When set to (none) and try to see if the K2200 is passedthrough,, all i see is a black screen on the connected screen.
I am out of ideas as to what else to do in configs to make this work. Please help if you have things to suggest I try. Thanks!
.
 
Last edited:
You can't use a Nvidia GPUs with MacOS since years. Want a hackintosh, baremetal or virtualized with gpu passthrough, you should ask in corresponding places.
AMD RX580, RX5700XT and RX6900XT is the way to go for now (RX7900 not supported yet and probably will never be). But Apple could drop support in coming MacOS versions in favor to Apple Silicon transition. IIRC last support for nvidia was available in Mojave and/or Catalina.
See compatibility list https://dortania.github.io/GPU-Buyers-Guide/modern-gpus/nvidia-gpu.html#kepler-series-gtx-6xx-7xx
 
Last edited:
  • Like
Reactions: leesteken and drx1
If you don’t need Ventura, you can use a GT710/720 (I think 730 as well but not sure, i will add a link to a list later) as well. They are supported by Big Sur natively and can easily be patched to be used with Monterey. Just be sure to buy a Kepler version, the really old ones do not work. Around 20-30€ (idk where you live) if you buy them used. I have the same plugged into a 1440p and 1080p monitor and it works fine for web browsing, YouTube videos and light work.
EDIT:
Just saw that the link I searched for is already in here but here are the Kepler gpus listed which are supported natively.
 
Last edited:
Thank you everyone for your replies.

I should try OpenCoreLegacy as LnxBil mentioned perhaps and see how that goes.

I got a Windows 10 Pro VM created yesterday and tried to pass-through the graphics card, still no luck.. It seems like the combination of CPU and GPU are not working well together.

The video card seems to be in IOMMU group 11 (thanks to user leesteken for the command outputs more detailed info) -->
[ 0.125672] DMAR-IR: Enabled IRQ remapping in x2apic mode ... IOMMU group 11 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM107GL [Quadro K2200] [10de:13ba] (rev a2) IOMMU group 11 01:00.1 Audio device [0403]: NVIDIA Corporation GM107 High Definition Audio Controller [GeForce 940MX] [10de:0fbc] (rev a1)

From the proxmox tutorial :
To have separate IOMMU groups, your processor needs to have support for a feature called ACS (Access Control Services). Make sure you enable the corresponding setting in your BIOS for this.

All Xeon processor support them (E3,E5) excluding Xeon E3-1200.
Click to expand...
I am not an expert but it seems like I need ACL support from the CPU to allow the GPU pass-through to a VM. However, this is the error message i see in "dmesg" --> [ 78.912447] systemd-journald[1154]: Failed to set ACL on /var/log/journal/48e20b088cca468e9eacc7ea5990688e/user-1000.journal, ignoring: Operation not supported

# lscpu : Model name: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz

My kernel command does not seem to be applied to fix the error : BAR 3: can't reserve [mem] error
[ 0.213780] pci 0000:01:00.0: BAR 3: assigned to efifb [ 519.479779] vfio-pci 0000:01:00.0: BAR 3: can't reserve [mem 0xf0000000-0xf1ffffff 64bit pref]

My file : /etc/kernel/cmdline
root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream nofb nomodeset video=efifb:off video=vesafb:off

Inside Windows 10 device manager, I can see only my video card name in Display adapters. I downloaded the NVIDIA drivers and installed them, rebooted, still nothing, I have also disabled RDP display driver in gpedit.msc and in proxmox display to none.

There are a lot of things I have done, and I thought this would have been an easy click and done with Proxmox. I was reading Unraid has a vfio-pci config plugin to make all this way easier..

Doing more research on this, i found a post from user Lefuneste, to look at the memory reservation ranges.
$ cat /proc/iomem
Returns:
00000000-00000000 : PCI Bus 0000:00 00000000-00000000 : PCI Bus 0000:01 00000000-00000000 : 0000:01:00.0 00000000-00000000 : vfio-pci 00000000-00000000 : 0000:01:00.0 00000000-00000000 : BOOTFB 00000000-00000000 : simplefb 00000000-00000000 : PCI Bus 0000:01 00000000-00000000 : 0000:01:00.0 00000000-00000000 : vfio-pci 00000000-00000000 : 0000:01:00.1
Inside Windows 10 VM, i can hear the sound and play youtube videos with sound, so this means at least 0000:01:00.1 (audio) is being passed-through lol but not the actual gpu 0000:01:00.0 because it is already reserved.

After doing the changes and rebooted Proxmox, and started Windows 10 VM, i get this now :
00000000-00000000 : PCI Bus 0000:00 00000000-00000000 : PCI Bus 0000:01 00000000-00000000 : 0000:01:00.0 00000000-00000000 : vfio-pci 00000000-00000000 : 0000:01:00.0 00000000-00000000 : vfio-pci 00000000-00000000 : PCI Bus 0000:01 00000000-00000000 : 0000:01:00.0 00000000-00000000 : vfio-pci 00000000-00000000 : 0000:01:00.1
However, i still do not have hardward acceleration, reinstalling NVIDIA drivers right now on VM.. rebooting now.

Success!! GPU pass-through is working on Windows 10 VM (confirmed with 3D spinning logo).

Sorry, I know this topic was about MacOS system and not Windows. However, I can now see the Nvidia card in Task Manager reporting stats properly. Added Windows 10 in the topic.

---

BAR 3 error is now gone (properly assigned to 1:00.0 nvidia):
[ 0.213874] pci 0000:01:00.0: BAR 3: assigned to efifb [ 10.757404] pci 0000:01:00.0: BAR 3: assigned [mem 0xf0000000-0xf1ffffff 64bit pref]

A reply from dece03 did the trick for me.
By the way, I also found better solution on a reddit post. It is adding "initcall_blacklist=sysfb_init" to kernel parameter. No need "video=efifb: off" or "video=simplefb: off" in kernel parameter. I also tested, it does solve the problem!
Click to expand...

[ 0.214006] pci 0000:01:00.0: BAR 3: assigned to efifb

Then --> sudo proxmox-boot-tool refresh
Reboot.
 
Last edited:
drx1 said:
Can you elaborate please and do you know a guide/video ?
Click to expand...
I just wanted to state, that the cards are supported (even for use in old legacy Apple hardware with a modern OS).
For your hackintosh (hardware that is NOT Apple), you will need OpenCore.
 
@drx1, macos is not supported by any virtualization that is not running on apple hardware and has corresponding licenses. your issue is not proxmox side, not the passthorough. it is really hardware issue (apple vs nvidia, there are no webdrivers from nvidia anymore, and no official drivers for older nvidia card in osx since mojave or so. yes, as LnxBil told, you can(should) go with OpenCore, which has extremely advanced on-the-fly patching capabilities in comparison to clover or chameleon. but it is not really native and you won't achieve a vanilla like installation, that would be update savvy or even expect it to reboot cleanly (it can, but no promises).
i am also a hackintosher since the early days (since tiger!) i don't want to scare you off here, this is just the wrong place to ask, since offtopic. and if we continue to fix hackintosh issues here, it could heavily explode and we could get some penalties from staff :)
please, take a look at dortanias blog i linked above, use an external search engine... there is a youtube channel, that is specifically about macos on proxmox, there are some very good forums and discord servers.
you got some nice keywords, here a list: opencore, osx86, amdosx (they have channels for non amd), hackintosh paradise, and some older as insanelymac, tonymacx, applelife, and there is more.
good luck!
 
Last edited:
flames said:
macos is not supported by any virtualization that is not running ....
Click to expand...
Fully agree ... I can also give you an additional search term nick sherlock proxmox macos ... his guide's are awesome and straight forward especially for PVE. I tried them just for fun a few times with different versions and it worked very well. I passthroughed a K2000 and USB3 on a Xeon workstation and it worked as far as I tested it. I often had PVE host crashes with any passthrough, so I stopped using passthrough at all. If you want to have a MacOS, just go and buy a MacMini ... no hassle and you'll love it.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back