Hi,
what error are you seeing when trying to delete the VM through the GUI? AFAIK it should be enough to delete the config file from /etc/pve/qemu-server/<vm>.conf
, maybe also the firewall config in /etc/pve/firewall/<vmid>.fw
if you configured it for the VM.