Hello,
I've been having sporadic issues for a couple weeks with our cluster.
Our VMs slow to a crawl or stop responding completely they can only be restored to normal functionality by shutting down (or stopping if completely unresponsive) or migrating the VM to another node (no shutdown/stop necessary).
I'm not certain if its related to snapshots or not, there are certain VMs it happens to more than others. We are mostly a Windows shop (almost completely off VMware at this point), the machines that are frequently impacted are File Servers, running Server 2022, created directly on Proxmox, not migrated (so no possible vmware tools remnants) and an application server running a lot of JVM apps with postgre (migrated from VMware, but thouroughly cleaned/inspected).
We use the CV4-PVE Admin tool to take snapshots at regular intervals (for these servers 3 times over the workday, held for 3 days).
For backup we use PBS and backup every night.
About our environment:
PVE: 8.2.4
Kernel: 6.8.8-2
Ceph: 18.2.2
Storage Type: Ceph Reef on Proxmox, VMs using RBD
Disk Type: Mix of SAS12 and NVMe Drives - all the same series 4 7.68TB per host
Ceph Private Networking: 2x 25gb LAG
Ceph Public Networking: 2x10gb LAG
Guest Networks & backup: 4x10gb LAG
Current node count: 4 (3xR730, 1xR760)
I'm kind of purplexed and not really sure where to start.
I do have support through proxmox and a local service provider but figured I'd start here in case anyone else was experiencing the same thing.
I've been having sporadic issues for a couple weeks with our cluster.
Our VMs slow to a crawl or stop responding completely they can only be restored to normal functionality by shutting down (or stopping if completely unresponsive) or migrating the VM to another node (no shutdown/stop necessary).
I'm not certain if its related to snapshots or not, there are certain VMs it happens to more than others. We are mostly a Windows shop (almost completely off VMware at this point), the machines that are frequently impacted are File Servers, running Server 2022, created directly on Proxmox, not migrated (so no possible vmware tools remnants) and an application server running a lot of JVM apps with postgre (migrated from VMware, but thouroughly cleaned/inspected).
We use the CV4-PVE Admin tool to take snapshots at regular intervals (for these servers 3 times over the workday, held for 3 days).
For backup we use PBS and backup every night.
About our environment:
PVE: 8.2.4
Kernel: 6.8.8-2
Ceph: 18.2.2
Storage Type: Ceph Reef on Proxmox, VMs using RBD
Disk Type: Mix of SAS12 and NVMe Drives - all the same series 4 7.68TB per host
Ceph Private Networking: 2x 25gb LAG
Ceph Public Networking: 2x10gb LAG
Guest Networks & backup: 4x10gb LAG
Current node count: 4 (3xR730, 1xR760)
I'm kind of purplexed and not really sure where to start.
I do have support through proxmox and a local service provider but figured I'd start here in case anyone else was experiencing the same thing.
Last edited: