First you should find out what the problem is. For example if your VM is actually fast, but just the remote desktop connection is bad, so it looks slow/laggy to you. In case you got GPU passed though you could connect a display directly to that GPU and see if it is still slow.
Then I would check network, disk, RAM, CPU, GPU benchmarks inside the VM to see if there is a botleneck, in case it is not the remote desktop connection.
Also make sure "KVM hardware acceleration" is enabled in the VMs options, that you don't run nested virtualization, that the CPU type is set to host and that you use virtio SCSI as the disk controller and virtio as the NIC.
And keep in mind that without a passed through GPU all graphics will have to be rendered in software by the CPU which is super slow and even a 720p youtube playback might stutter.