What can work is passing a GPU and keyboard through and connecting the display to that GPU, that way you'll see the VM's graphic output 1:1 directly, nothing in between, some monitors even support some KVM-switch like functionality, like it has two downstream USB ports, and you can switch which one is used depending on display source selected, so a single keyboard/mouse can be toggled between PVE host and Ubuntu VM.
The simplest and flexible solution will probably still be to install a minimal graphic environment onto PVE and just use SPICE/virt-viewer in full-screen mode, that's quite fast and integrated.