Web and SSH access are sometimes unavailable

[JoannF]

Hi,

I’m experiencing an intermittent issue accessing my Proxmox host, both via the Web interface (port 8006) and SSH (port 22). The environment is installed on an OVH bare-metal server.

Under normal conditions everything can work fine for several days, then suddenly access to the Proxmox host becomes impossible: neither the Web UI nor SSH is reachable.

This server has two public IP addresses:

• The main public IP is configured on the bridge used to access the Proxmox host (Web UI + SSH).
• A second public IP is associated with a vMAC and is used by an OPNsense VM; the vMAC is configured on that VM’s network interface.

When the problem occurs, only connections to the main IP (the host IP) fail. However, access to the VMs behind OPNsense via the second public IP continues to work.

I investigated the network behavior. On the main IP, during connection attempts (SSH or Web), the server does receive the SYN packets and replies correctly with SYN/ACK packets, but the final ACK is never received on the client side. As a result, the TCP handshake never completes.

An important point: a full reboot of the server (reboot of the host) immediately restores access. Restarting only the network interface does not fix the issue and additionally makes the virtual machines unreachable. After a reboot, Web/SSH access usually works again for a few days before the issue comes back.

I also tried disabling all firewalls when the issue occurs, but it didn’t change anything—both the OVH firewall and the Proxmox firewall (where allowed client IPs are configured).

Thanks for your help.

Network interface configuration :

auto lo
iface lo inet loopback

iface enp10s0f0np0 inet manual

iface enp10s0f1np1 inet manual

# Bridge on public interface (used by the Proxmox host and the OPNsense VM for network access)
auto vmbr0
iface vmbr0 inet static
        address xx.xx.xx.xx/32
        gateway 100.64.0.1
        bridge-ports enp10s0f0np0
        bridge-stp off
        bridge-fd 0

# Bridge to the LAN physical interface (not currently used)
auto vmbr1
iface vmbr1 inet static
        address 192.168.0.1/24
        bridge-ports enp10s0f1np1
        bridge-stp off
        bridge-fd 0

# Used by the OPNsense VM
auto vmbr2
iface vmbr2 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0

# Used by the OPNsense VM
auto vmbr3
iface vmbr3 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0

source /etc/network/interfaces.d/*
source /etc/network/interfaces.d/sdn

 

[bbgeek17]

I investigated the network behavior. On the main IP, during connection attempts (SSH or Web), the server does receive the SYN packets and replies correctly with SYN/ACK packets, but the final ACK is never received on the client side. As a result, the TCP handshake never completes.

If you see the ACK leaving the physical port, then it is likely that the problem is further down the line in the infrastructure. Have you contacted OVH with your issue/findings?

An important point: a full reboot of the server (reboot of the host) immediately restores access. Restarting only the network interface does not fix the issue

Do you have console access via OVH? Perhaps you can try to "ip link down/up" the ports as an experiment. It sounds like some sort of ARP/IP table in play.
Also, you can try not starting any VMs for a while to see if the issue reproduces when you are only running with single interface/IP/MAC

Also, isn't there something in OVH interface where you have to register MAC addresses if you have multiple?

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[JoannF]

If you see the ACK leaving the physical port, then it is likely that the problem is further down the line in the infrastructure. Have you contacted OVH with your issue/findings?

Do you have console access via OVH? Perhaps you can try to "ip link down/up" the ports as an experiment. It sounds like some sort of ARP/IP table in play.
Also, you can try not starting any VMs for a while to see if the issue reproduces when you are only running with single interface/IP/MAC

Also, isn't there something in OVH interface where you have to register MAC addresses if you have multiple?

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

I was planning to contact OVH to see if they have any leads on the root cause of the issue, but I haven’t done so yet—this will be my next step.
I was able to connect to the server using Serial over LAN and tried an ip link down/up, but it didn’t change anything.

Indeed, OVH provides a vMAC configuration in their interface, where you can choose for which IP to generate a vMAC. For my second IP, I generated a vMAC and assigned it to the virtual NIC of the OPNsense VM. However, I’m not able to generate a vMAC for the main IP address.

Also, you just made me realize that I haven’t tried stopping the OPNsense VM when the issue occurs, even though it’s the VM that uses the additional IP.
I can also try running the host without starting any VMs for a few days to see whether the problem still happens.

If the additional IP/vMAC setup turns out to be the root cause, I could do without it by switching to an internal NAT/routing setup, but that wouldn’t be a very clean configuration.

 

[JeanL]

Hi,

Have you checked if your server is under DDoS mitigation when this happened ?

I have the exact setup as you, and no issue (it's a SYS server).

Also, isn't there something in OVH interface where you have to register MAC addresses if you have multiple?

You can only generate MAV for additional IP; the MAC of the server is already known by OVH (because they used it to put restrictions on the TOR switch), and if they change the motherboard, they give you the new MAC in the incident ticket to check your config.

Best regards,

 

[JoannF]

Have you checked if your server is under DDoS mitigation when this happened ?

I checked the DDoS protection history table and it is empty.
So apparently that’s not the cause.

However, I made some progress on the issue today, since it happened again overnight, less than 24 hours after the last reboot of the Proxmox host.

I started wondering whether there could be a link with MAC address assignment, so I tried adding the setting below to my network configuration, specifying the MAC address of the physical network interface on the Proxmox vmbr0 bridge.
I added it, reloaded the interfaces, and I immediately regained access to the web interface and SSH.

Now, whether this actually solved the problem, or whether it was something else that got fixed by reloading the interfaces, remains to be seen.
Time will tell, but I’m hopeful that this has resolved it.

Thank you all for your help.


New configuration of vmbr0 :

auto vmbr0
iface vmbr0 inet static
        address xx.xx.xx.xx/32
        gateway 100.64.0.1
        bridge-ports enp10s0f0np0
        bridge-stp off
        bridge-fd 0
        hwaddress ether xx:xx:xx:xx:xx:xx

 

[JoannF]

I’ve just lost access again.

 

[zeppie]

Experiencing the same issue. If it helps, the symptoms in my case are that ACKs packets disappear when the VMs are on the same host but using different networks. If the VMs are in different hosts, the ACKs are sent normally.

 

[JoannF]

Experiencing the same issue. If it helps, the symptoms in my case are that SYN packets disappear when the VMs are on the same host but using different networks. If the VMs are in different hosts, the SYNs are sent normally.

I performed a complete reinstallation of Proxmox last weekend, and it seems to have resolved the issue, as I haven’t experienced any additional crashes since then.