There's a number of discussions on the forum about how to backup PVE itself, mainly focussed on the config files.
Being a scripting tard I've asked our friend ChatGTP to help me making a simple script. There's probably a way better method, but this covers my needs.
So for those looking for...