Hi,
Recently started experiencing a lot of issues with a docker VM hosted in Proxmox, which is hosting majority of home services.
Same VM has been running for multiple years just fine and changed 3 different physical hosts without a problem since last few weeks.
I got many random glitches related to connectivity to this particular VM - occasional drops and self restore after 5/10/15 minutes.
Proxmox host reboot does seems to help making those glitches a bit rare, but after a week of uptime you get them every couple of hours.
Setup - Small cluster of 3 nodes. Lately keep two of them up, as there is no good cooling in the rack.
Only thing changed to the VM is adding a PCI Pass-through GPU. (RTX A2000).
Some high level observations:
- Connection from PVE host to VM works 100% of the times, no drops, no broken pipes.
- Connections from my laptop to VM drops quite often. When connecting from laptop to PVE and then to VM - all good.
Tried to disable the pve-firewall - no effect.
Journalctl does not give many interesting things, apart of :
I was changing LAN cables, used a different ports on the switch and so far no result. Have idea to change the switch as well on Sunday.
It does not indicate to be the switch, as the connection from laptop to host is 100% reliable based on several days stats with no single packet drop, while the VM inside the PVE are dropping.
My Plan B is to drop the cluster completely and see if stand alone PVE instance would have the same behavior. Have to admit the cluster is not NW redundant yet and network is a flat one. Struggling to find the time to optimize it and introduce vlans, but first priority is to improve the reliability of the hosted services.
P.S: Running latest pve:
Recently started experiencing a lot of issues with a docker VM hosted in Proxmox, which is hosting majority of home services.
Same VM has been running for multiple years just fine and changed 3 different physical hosts without a problem since last few weeks.
I got many random glitches related to connectivity to this particular VM - occasional drops and self restore after 5/10/15 minutes.
Proxmox host reboot does seems to help making those glitches a bit rare, but after a week of uptime you get them every couple of hours.
Setup - Small cluster of 3 nodes. Lately keep two of them up, as there is no good cooling in the rack.
Only thing changed to the VM is adding a PCI Pass-through GPU. (RTX A2000).
Some high level observations:
- Connection from PVE host to VM works 100% of the times, no drops, no broken pipes.
- Connections from my laptop to VM drops quite often. When connecting from laptop to PVE and then to VM - all good.
When glitches start - VM is only reachable from PVE host itself - not from my laptop or from any other LXC/VMs running on other PVE host.
Tried to disable the pve-firewall - no effect.
Dmesg does not show anything useful, I have seen before the vmbr0 messages as well, but it has not been an issue.
Journalctl does not give many interesting things, apart of :
I was changing LAN cables, used a different ports on the switch and so far no result. Have idea to change the switch as well on Sunday.
It does not indicate to be the switch, as the connection from laptop to host is 100% reliable based on several days stats with no single packet drop, while the VM inside the PVE are dropping.
My Plan B is to drop the cluster completely and see if stand alone PVE instance would have the same behavior. Have to admit the cluster is not NW redundant yet and network is a flat one. Struggling to find the time to optimize it and introduce vlans, but first priority is to improve the reliability of the hosted services.
P.S: Running latest pve:
VM was restored from backup to the new physical host. Nothing was changed on the NW side - router, switches and etc. Only changed LAN cables after the glitches started.
Last edited: