Actually i not understand your question, can you give more details?
but i assuming as i understand, you can do that by creating a user and give him permission to control on a specific VM or XCL.
I don't have like this Network card, but i suggest you to install PVC 6.x on top of a fresh installation of debian buster:
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_install_proxmox_ve_on_debian
EDIT: or look if any error in syslog