[SOLVED] Very low performance of MacOS Big Sur on Proxmox v7, Ryzen 2700X

[nahero]

I managed to get Big Sur up and running on Proxmox v7, and while I'm getting acceptable results on CPU and GPU benchmarks, the SSD and overall performance is just terrible. For example if I play a 4K movie file everything becomes very sluggish, cursor lag gets so bad I can barely click on buttons. 4K playback from Youtube doesn't affect the system in this way, but there is a problem with Realtek network controller which is stuck at 10Mbit, and I had that problem earlier on hackintosh install so it might be a hardware problem.

My setup:
Ryzen 2700X, Gigabyte Aorus M B450, 32GB g.skill 3200mhz RAM, Sapphire RX 580 8gb

Here is what I did:
- installed Proxmox and then Big Sur VM on same Samsung 970 Evo NVMe SSD (chose xfs as file system)
- setup GPU passthrough (blacklisted Radeon drivers, set display to none)
- USB passthrough working but cursor very laggy (Logitech unifying receiver), regardless if I passthrough device or port. Then I passed through the whole USB3.0 controller PCIE device, also works but lag is the same. Can't tell if keyboard is actually laggy, it feels only the mouse is the issue
- Set CPU cores to 16 (either 2 sockets / 8 cores or 1 socket / 16 cores, results are the same). CPU type is Penryn.
- Set VM RAM to 30Gb, if I set it to max 32Gb VM won't start

On Geekbench 5 I get around 950 single core speed and 6200 multi-core, while on the same system with a bare-metal hackintosh install I was getting 1070 single and 7000 multi-core. GPU Compute score is around 35000 which is also 10-15% lower than expected, but this is not important for me.

With Novabench I noticed that something is wrong with SSD - I got around 200MBs write and 4000MBs read on first test, then every other test was much worse - down to 100MBs write and 300MBs read. I didn't test USB speeds with any tool but copying a 8GB from a USB stick file felt fast as expected.

I'm guessing there is something wrong with PCIE or even with my MBO, but is there a way to test this or some settings I should do in Proxmox?

-----
Here is my full config:

.../qemu-server/100.conf
args: -device isa-applesmc,osk="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc" -smbios type=2 -device usb-kbd,bus=ehci.0,port=2 -cpu Penryn,kvm=on,vendor=GenuineIntel,+kvm_pv_unhalt,+kvm_pv_eoi,+hypervisor,+invtsc,+pcid,+ssse3,+sse4.2,+popcnt,+avx,+avx2,+aes,+fma,+fma4,+bmi1,+bmi2,+xsave,+xsaveopt,check bios: ovmf boot: order=virtio0 cores: 8 cpu: Penryn efidisk0: local-lvm:vm-100-disk-1,size=4M hostpci0: 0000:07:00,pcie=1,x-vga=1 hostpci1: 0000:08:00,pcie=1 hotplug: usb machine: q35 memory: 30720 name: MacOSBigSur net0: vmxnet3=BE:EF:97:E2:DE:AB,bridge=vmbr0,firewall=1 numa: 0 ostype: other scsihw: virtio-scsi-pci smbios1: uuid=e927b933-74c3-44bd-9354-e0475bd08a63 sockets: 2 vga: none virtio0: local-lvm:vm-100-disk-0,cache=unsafe,iothread=1,size=640G vmgenid: 88fcd323-5ecf-48e5-bb2d-fabf1f5e67d3

/etc/default/grub:
GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt amd_iommu=on video=efifb:off"

/etc/modprobe.d/blacklist.conf:
blacklist radeon
blacklist amdgpu

/etc/modules:
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

/etc/modprobe.d/vfio.conf:
options vfio-pci ids=1002:67df,1002:aaf0 disable_vga=1

MBO settings:
SVM Mode is on, CSM (legacy boot) is off

 

[jasonsansone]

Post full config for the VM. What storage controller type, caching options, etc all matter.

 

[nahero]

Updated my question with full config

 

[nahero]

Any ideas why the ssd performance is so low? Could it be because I installed VM on the same disk as Proxmox? Also can the PCIE performance be tested somehow, as I think it could be the problem

 

[nahero]

Decided to experiment on my own and got the ssd to run properly, at least in Blackmagic Disk Speed: I changed cache from Write Back (unsafe) to Default (No cache) and left Discard on - now I'm hitting 2400MB/s write and over 3000MB/s read.
But if I leave the Blackmagic test running for a few minutes I can see the write speed falling down to about 1500MB/s, which probably means that TRIM is not enabled properly. I did enable it with sudo trimforce enable to see what happens but didn't notice any difference.
Also, if I test with Novabench I still see a low result of 300MB/s and 2000MB/s.

 

[jasonsansone]

Somehow I never received prior email notifications of responses. I apologize I didn't respond. I didn't know the thread was active. Since there is no guest agent or QEMu tools available, trim isn't likely ever going to function properly.

If I allow the test to keep running through multiple loops, I see an average sustained speed of 1500 MB/s write, 3000 MB/s read. I use writeback cache with io_uring on a Ceph cluster.

 

[nahero]

If trim isn't working properly how come there are not more mentions of this on the forums? What I see in every tutorial on creating MacOS VM in Proxmox is to set writeback (unsafe) and check the Discard option (for trim to function), and this is coming from people who also use nvme ssd like me. There is something wrong if I'm getting much better ssd results without caching.

Also, there is another issue that I see now in MacOS, I have stutter and glitches when watching youtube videos, and generally when I put some strain on the ssd the whole system starts having these glitches and it can only be partially solved by restarting. But after a few minutes of work it starts again.

PS what is io_uring you mention? Do you mean IO Thread option on Hard disk? I tried with and without it and it seems to make no difference. Documentation states that performance-wise best option for hard disk is SCSI driver (newer than VirtIO Block) with VirtIO SCSI controller and IO Thread on, but MacOS doesn't recognize the disk if using anything else than VirtIO Block. It basically makes no difference using VirtIO SCSI or any other controller with VirtIO Block.

 

[jasonsansone]

If trim isn't working properly how come there are not more mentions of this on the forums? What I see in every tutorial on creating MacOS VM in Proxmox is to set writeback (unsafe) and check the Discard option (for trim to function), and this is coming from people who also use nvme ssd like me. There is something wrong if I'm getting much better ssd results without caching.

macOS has a comparatively small user base and an even smaller enterprise user base. Very few run a macOS VM in production. The only hypervisor (to my knowledge) that officially supports macOS is vmWare and that is only on official Apple hardware. Running a Mac VM is more fun / novelty / hobbyist. As you know, you still have to use OpenCore. To extrapolate the problem even further, you are using a processor and motherboard that Apple has never (and will never) support. You don't see tickets or forum posts because no enterprise is actively testing macOS VM's on Proxmox on AMD consumer hardware.

I don't know "why" all of the internet tutorials elect to use writeback cache (unsafe), but I wouldn't personally ever use that setting (hence the word unsafe). I have extensively tested various configurations in order to create a macOS VM which supports live migration and HA. Many of the configuration settings floating around out there are not required, wrong, or modifiable. However, I am running a test cluster with identical Xeons, so I can't speak to your exact hardware.

PS what is io_uring you mention? Do you mean IO Thread option on Hard disk? I tried with and without it and it seems to make no difference. Documentation states that performance-wise best option for hard disk is SCSI driver (newer than VirtIO Block) with VirtIO SCSI controller and IO Thread on, but MacOS doesn't recognize the disk if using anything else than VirtIO Block. It basically makes no difference using VirtIO SCSI or any other controller with VirtIO Block.

The SCSI controllers aren't supported by any macOS kexts. They won't work, as you noted. I was referring to the async I/O options. See image.

 

[nahero]

jasonsansone thanks for the reply, I see I have to experiment to get my system working properly, and now it all seems to be working fine.
After switching to No cache I got my ssd working but had a lot of stutter whenever the system was under load. Solved this by unchecking ROM-Bar from the PCIE USB 3.0 controller that I passed through to the VM (just like the GPU).

 

[cagnulein]

so @nahero @@jasonsansone which is your current configuration?