USB passthrough is always a bit finicky, sometimes hardware devices implement some weird switches or hubs which can trip up QEMU. Try disabling USB3 for the passthrough, or use the manufacturer/device ID to identify the device. Note also that the device probably has to be on already before adding the device or starting the VM.
An alternative option would be to use PCIe passthrough to attach an external or onboard (support varying between motherboards and chipsets) USB controller to the VM. This also increases performance significantly, but is more involved to set up and can't be shared between VMs as easily.