Thanks for the tip, Mira. I had thought PVE used Debian hence the choice to use the Debian live USB.
Had to find some time to get that all set up, but it does seem like the kernel difference could be at least contributing to this. Using the Ubuntu 20.04 Live USB, I was seeing similar 6Gbps...