I did some PCI pass through and used USB pass through for devices which had only windows support - I even upgraded the firmware of my workstations (uses PVE) monitor over a Windows VM with USB passthrough (while continuing to use said monitor), so in my experience, both can work quite stable.
I mean, naturally you can have bad luck with your setup, and have cursed firmware and or hardware, but more simple passthrough should be doable by most HW nowadays.
In your situation I'd try the USB passthrough route first, you probably don't need to buy extra HW for that as a USB hub is more often available than a PCI USB expander card, you should get a feeling about how flaky it is quickly and if it real has issues you can still get you a PCI card then.