Tracking PVE QEMU VM configuration drift/evolution over time

I would like to be able to track and log how configuration of all our QEMU VMs across our clusters develop and evolve over time. I plan to do that by periodically snapshotting all qemu instance configs via the PVE cluster API, and stuffing the structured response result in a git repository, stored in one result file per cluster-and-vmid (or maybe vm name; I am not sure yet). I have used this technique with great results in the past for other kinds of resources that cannot meaningfully be managed by a "gitops"-like procedure and process.

Do any of you have a procedure in place to do such a thing? If so, what exactly do you do or use to implement it?
 
  • Like
Reactions: Johannes S
We played around with different tools on a simple vc level and also with tools like etckeeper but hae not found a better solution than just do auto-add/remove/commits. We also tried it on /etc/pve itself, but also with an rsynced copy.
 
  • Like
Reactions: Johannes S
You could use pvereport for the config export.
You would have to parse that file somehow to geht the individual VM's or config settings of your node.
Thanks, that's a great recommendation to also include PVE node information - but I am looking for something with more structured output and came quite far with a simple client to the REST API PVE provides already.

I have also found https://github.com/Corsinvest/cv4pve-report which seems like it can absolutely provide nice results and much more than what I managed to build in a hurry, but I am just not very comfortable with relying on a C# tool in my daily dealings %-)