I just wanted to note this down for reference, it may help some of you also.
Adding some NVME drives to a VM using the 'Add PCI Device' option can crash Proxmox.
For example the Samsung 960 Pro drive I had this problem with, other drives were fine.
With help from chatgpt it seemed to indicate...