Hi,
what error are you seeing when trying to delete the VM through the GUI? AFAIK it should be enough to delete the config file from /etc/pve/qemu-server/<vm>.conf, maybe also the firewall config in /etc/pve/firewall/<vmid>.fw if you configured it for the VM.