Losing network seemingly randomly

P

ProjectInfinity

New Member
May 12, 2017
2
0
1
33
Hi, I'm not gonna pretend that I know my way around networking, it's always been the bane of my existence. However after finding proxmox I am determined to make it work as I like the product and if someone can help me figure out the issues with the networking I would be more than happy to support the project (to also get subscriber repo access :p)

What happens is that the server seemingly at random loses connectivity to the outside world, as I am writing this my idling server lost all connection and the only way to bring it back is to go to hetzner and force a reboot.

However this can also happen at random when I start or stop containers (but it may just be unlucky and that the stars aligned just as I used the commands).

Truth be told I had a similar problem on my home server with proxmox with a completely standard network setup, Ubuntu did not have that problem however.

Host network
auto lo
iface lo inet loopback

iface lo inet6 loopback

auto eth0
iface eth0 inet static
address 88.99.241.206
netmask 255.255.255.192
gateway 88.99.241.193
pointopoint 88.99.241.193
up route add -net 88.99.241.192 netmask 255.255.255.192 gw 88.99.241.193 dev eth0
# route 88.99.241.192/26 via 88.99.241.193

iface eth0 inet6 static
address 2a01:4f8:10b:215::2
netmask 64
gateway fe80::1

auto vmbr0
iface vmbr0 inet static
address 10.10.10.1
netmask 255.255.255.0
bridge_ports eth0
bridge_stp off
bridge_fd 0

post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o eth0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o eth0 -j MASQUERADE
Click to expand...

Container network
auto lo
iface lo inet loopback
iface lo inet6 loopback

# interfaces(5) file used by ifup(8) and ifdown(8)
# Include files from /etc/network/interfaces.d:
source-directory /etc/network/interfaces.d

auto eth0
iface eth0 inet static
address 10.10.10.2
netmask 255.255.255.255
# --- BEGIN PVE ---
post-up ip route add 10.10.10.1 dev eth0
post-up ip route add default via 10.10.10.1 dev eth0
pre-down ip route del default via 10.10.10.1 dev eth0
pre-down ip route del 10.10.10.1 dev eth0
# --- END PVE ---
Click to expand...

Request timeout for icmp_seq 69 - At this point I had restarted the server and was waiting for it to come back up
Request timeout for icmp_seq 70
Request timeout for icmp_seq 71
Request timeout for icmp_seq 72
Request timeout for icmp_seq 73
Request timeout for icmp_seq 74
Request timeout for icmp_seq 75
Request timeout for icmp_seq 76
64 bytes from 88.99.241.206: icmp_seq=77 ttl=55 time=48.668 ms - It finally came up.
64 bytes from 88.99.241.206: icmp_seq=78 ttl=55 time=46.605 ms
64 bytes from 88.99.241.206: icmp_seq=79 ttl=55 time=46.323 ms
64 bytes from 88.99.241.206: icmp_seq=80 ttl=55 time=45.643 ms
64 bytes from 88.99.241.206: icmp_seq=81 ttl=55 time=46.112 ms
64 bytes from 88.99.241.206: icmp_seq=82 ttl=55 time=46.753 ms
64 bytes from 88.99.241.206: icmp_seq=83 ttl=55 time=45.577 ms
64 bytes from 88.99.241.206: icmp_seq=84 ttl=55 time=46.529 ms
64 bytes from 88.99.241.206: icmp_seq=85 ttl=55 time=44.717 ms
64 bytes from 88.99.241.206: icmp_seq=86 ttl=55 time=45.067 ms
64 bytes from 88.99.241.206: icmp_seq=87 ttl=55 time=44.938 ms
64 bytes from 88.99.241.206: icmp_seq=88 ttl=55 time=44.932 ms
64 bytes from 88.99.241.206: icmp_seq=89 ttl=55 time=45.331 ms
64 bytes from 88.99.241.206: icmp_seq=90 ttl=55 time=45.168 ms
64 bytes from 88.99.241.206: icmp_seq=91 ttl=55 time=46.594 ms
64 bytes from 88.99.241.206: icmp_seq=92 ttl=55 time=47.015 ms
64 bytes from 88.99.241.206: icmp_seq=93 ttl=55 time=47.031 ms
64 bytes from 88.99.241.206: icmp_seq=94 ttl=55 time=47.475 ms
64 bytes from 88.99.241.206: icmp_seq=95 ttl=55 time=45.807 ms
64 bytes from 88.99.241.206: icmp_seq=96 ttl=55 time=45.601 ms
64 bytes from 88.99.241.206: icmp_seq=97 ttl=55 time=46.078 ms
64 bytes from 88.99.241.206: icmp_seq=98 ttl=55 time=46.032 ms
64 bytes from 88.99.241.206: icmp_seq=99 ttl=55 time=45.965 ms
64 bytes from 88.99.241.206: icmp_seq=100 ttl=55 time=46.035 ms
64 bytes from 88.99.241.206: icmp_seq=101 ttl=55 time=45.183 ms
64 bytes from 88.99.241.206: icmp_seq=102 ttl=55 time=46.464 ms
64 bytes from 88.99.241.206: icmp_seq=103 ttl=55 time=46.076 ms
64 bytes from 88.99.241.206: icmp_seq=104 ttl=55 time=46.496 ms
64 bytes from 88.99.241.206: icmp_seq=105 ttl=55 time=46.993 ms
64 bytes from 88.99.241.206: icmp_seq=106 ttl=55 time=46.251 ms
64 bytes from 88.99.241.206: icmp_seq=107 ttl=55 time=46.955 ms
64 bytes from 88.99.241.206: icmp_seq=108 ttl=55 time=46.566 ms
64 bytes from 88.99.241.206: icmp_seq=109 ttl=55 time=46.680 ms
64 bytes from 88.99.241.206: icmp_seq=110 ttl=55 time=46.890 ms
64 bytes from 88.99.241.206: icmp_seq=111 ttl=55 time=45.787 ms
64 bytes from 88.99.241.206: icmp_seq=112 ttl=55 time=46.374 ms
64 bytes from 88.99.241.206: icmp_seq=113 ttl=55 time=45.604 ms
64 bytes from 88.99.241.206: icmp_seq=114 ttl=55 time=45.444 ms
64 bytes from 88.99.241.206: icmp_seq=115 ttl=55 time=45.116 ms
64 bytes from 88.99.241.206: icmp_seq=116 ttl=55 time=45.618 ms
64 bytes from 88.99.241.206: icmp_seq=117 ttl=55 time=46.267 ms
64 bytes from 88.99.241.206: icmp_seq=118 ttl=55 time=46.039 ms
64 bytes from 88.99.241.206: icmp_seq=119 ttl=55 time=46.656 ms
64 bytes from 88.99.241.206: icmp_seq=120 ttl=55 time=45.919 ms
64 bytes from 88.99.241.206: icmp_seq=121 ttl=55 time=45.889 ms
64 bytes from 88.99.241.206: icmp_seq=122 ttl=55 time=46.541 ms
64 bytes from 88.99.241.206: icmp_seq=123 ttl=55 time=46.059 ms
64 bytes from 88.99.241.206: icmp_seq=124 ttl=55 time=45.991 ms
64 bytes from 88.99.241.206: icmp_seq=125 ttl=55 time=46.451 ms
64 bytes from 88.99.241.206: icmp_seq=126 ttl=55 time=44.930 ms
64 bytes from 88.99.241.206: icmp_seq=127 ttl=55 time=45.986 ms
64 bytes from 88.99.241.206: icmp_seq=128 ttl=55 time=45.633 ms
64 bytes from 88.99.241.206: icmp_seq=129 ttl=55 time=46.507 ms
64 bytes from 88.99.241.206: icmp_seq=130 ttl=55 time=46.349 ms
64 bytes from 88.99.241.206: icmp_seq=131 ttl=55 time=45.807 ms
64 bytes from 88.99.241.206: icmp_seq=132 ttl=55 time=46.571 ms
64 bytes from 88.99.241.206: icmp_seq=133 ttl=55 time=46.191 ms
64 bytes from 88.99.241.206: icmp_seq=134 ttl=55 time=45.093 ms
64 bytes from 88.99.241.206: icmp_seq=135 ttl=55 time=46.188 ms
64 bytes from 88.99.241.206: icmp_seq=136 ttl=55 time=45.535 ms
64 bytes from 88.99.241.206: icmp_seq=137 ttl=55 time=46.024 ms
64 bytes from 88.99.241.206: icmp_seq=138 ttl=55 time=45.335 ms
64 bytes from 88.99.241.206: icmp_seq=139 ttl=55 time=44.727 ms
64 bytes from 88.99.241.206: icmp_seq=140 ttl=55 time=46.489 ms
64 bytes from 88.99.241.206: icmp_seq=141 ttl=55 time=45.633 ms
64 bytes from 88.99.241.206: icmp_seq=142 ttl=55 time=46.580 ms
64 bytes from 88.99.241.206: icmp_seq=143 ttl=55 time=46.170 ms
64 bytes from 88.99.241.206: icmp_seq=144 ttl=55 time=45.899 ms
64 bytes from 88.99.241.206: icmp_seq=145 ttl=55 time=46.859 ms
64 bytes from 88.99.241.206: icmp_seq=146 ttl=55 time=45.567 ms
64 bytes from 88.99.241.206: icmp_seq=147 ttl=55 time=45.589 ms
64 bytes from 88.99.241.206: icmp_seq=148 ttl=55 time=46.157 ms
64 bytes from 88.99.241.206: icmp_seq=149 ttl=55 time=45.145 ms
64 bytes from 88.99.241.206: icmp_seq=150 ttl=55 time=44.922 ms
64 bytes from 88.99.241.206: icmp_seq=151 ttl=55 time=45.059 ms
64 bytes from 88.99.241.206: icmp_seq=152 ttl=55 time=45.442 ms
64 bytes from 88.99.241.206: icmp_seq=153 ttl=55 time=47.194 ms
64 bytes from 88.99.241.206: icmp_seq=154 ttl=55 time=46.545 ms
64 bytes from 88.99.241.206: icmp_seq=155 ttl=55 time=45.226 ms
64 bytes from 88.99.241.206: icmp_seq=156 ttl=55 time=45.728 ms
64 bytes from 88.99.241.206: icmp_seq=157 ttl=55 time=45.417 ms
64 bytes from 88.99.241.206: icmp_seq=158 ttl=55 time=46.284 ms
64 bytes from 88.99.241.206: icmp_seq=159 ttl=55 time=45.912 ms
64 bytes from 88.99.241.206: icmp_seq=160 ttl=55 time=46.338 ms
64 bytes from 88.99.241.206: icmp_seq=161 ttl=55 time=44.947 ms
64 bytes from 88.99.241.206: icmp_seq=162 ttl=55 time=46.464 ms
64 bytes from 88.99.241.206: icmp_seq=163 ttl=55 time=45.866 ms
64 bytes from 88.99.241.206: icmp_seq=164 ttl=55 time=46.486 ms
64 bytes from 88.99.241.206: icmp_seq=165 ttl=55 time=45.752 ms
64 bytes from 88.99.241.206: icmp_seq=166 ttl=55 time=45.433 ms
64 bytes from 88.99.241.206: icmp_seq=167 ttl=55 time=45.755 ms
64 bytes from 88.99.241.206: icmp_seq=168 ttl=55 time=46.497 ms
64 bytes from 88.99.241.206: icmp_seq=169 ttl=55 time=46.360 ms
64 bytes from 88.99.241.206: icmp_seq=170 ttl=55 time=46.156 ms
64 bytes from 88.99.241.206: icmp_seq=171 ttl=55 time=45.665 ms
64 bytes from 88.99.241.206: icmp_seq=172 ttl=55 time=46.275 ms
64 bytes from 88.99.241.206: icmp_seq=173 ttl=55 time=45.897 ms
64 bytes from 88.99.241.206: icmp_seq=174 ttl=55 time=47.117 ms
64 bytes from 88.99.241.206: icmp_seq=175 ttl=55 time=46.394 ms
64 bytes from 88.99.241.206: icmp_seq=176 ttl=55 time=45.571 ms
64 bytes from 88.99.241.206: icmp_seq=177 ttl=55 time=45.955 ms
64 bytes from 88.99.241.206: icmp_seq=178 ttl=55 time=45.820 ms
64 bytes from 88.99.241.206: icmp_seq=179 ttl=55 time=45.018 ms
64 bytes from 88.99.241.206: icmp_seq=180 ttl=55 time=45.953 ms
64 bytes from 88.99.241.206: icmp_seq=181 ttl=55 time=45.532 ms
64 bytes from 88.99.241.206: icmp_seq=182 ttl=55 time=46.409 ms
64 bytes from 88.99.241.206: icmp_seq=183 ttl=55 time=46.474 ms
64 bytes from 88.99.241.206: icmp_seq=184 ttl=55 time=45.907 ms
64 bytes from 88.99.241.206: icmp_seq=185 ttl=55 time=46.503 ms
64 bytes from 88.99.241.206: icmp_seq=186 ttl=55 time=45.061 ms
64 bytes from 88.99.241.206: icmp_seq=187 ttl=55 time=45.937 ms
64 bytes from 88.99.241.206: icmp_seq=188 ttl=55 time=46.518 ms
64 bytes from 88.99.241.206: icmp_seq=189 ttl=55 time=45.986 ms
Request timeout for icmp_seq 190 - and it lost all outside connectivity while I found the container network configuration.
Request timeout for icmp_seq 191
Request timeout for icmp_seq 192
Request timeout for icmp_seq 193
Request timeout for icmp_seq 194
Request timeout for icmp_seq 195
Request timeout for icmp_seq 196
Click to expand...

EDIT:
I can add to this and say I tried to delete all my containers and just keep the machine on with the provided host configuration above... It went down after somewhere between 5-10 minutes, idling. :/
 
Last edited:
Bit old, however I have just experienced the same problem. 10 containers and 5 VMS lost their network connection
 
kylea said:
Bit old, however I have just experienced the same problem. 10 containers and 5 VMS lost their network connection
Click to expand...
Sorry to Necro as well, but same problem. 1 out of 4 VMs with Debian 9.4, installed at the same time, same Repo. Installed with virtio network card, changed to e1000 or realtek for testing, same problem. VM is randomly not able to ping host/gateway but the other VMs are.

Host with and without iptables rules. No difference.

HDD of VMs are local storage as well.

Traffic on all interfaces routed via vmbr0 interface to same subnet and gateway. HP Procurve Switch can't be the problem, because host can't be pinged as well but was replaced by different for testing. Same problem.

No log entrys in syslog nor dmesg - it just went offline and is unable to ping or provide any other tcp/udp service. Seconds/minutes later

Anything I can do it find the reason and fix it?
 
One time I've seen this behavior it was duplicate IPs on the network. Check the ARP table on your router/firewall and see if the MAC address for host IP is correct. Or shut your hosts down and then ping their IP's from another machine, if you get a response then something else is using the same IP.
 
Oh, yeah, I forgot to mention, I already checked that as well. -> Set it from random automatic to manually ( BE:EF:00:00: DB:01 )
I also exchanged the NIC from the internal to an PCI-E one.
Sadly just the same Brand/Chip - (Broadcom Limited NetXtreme II BCM5706 Gigabit Ethernet (rev 02)) - Possible culprit here?
 
Not MAC address, IP address. From the symptoms, it sounds like you have 2 devices on the network configured with the same IP address, ie 192.168.1.x
 
Yeah, it sounds like it, but it certainly is not the case. It's a newly created vlan without dhcp and former use, so, nope.
 
I've never done the NAT'ing for a guest, but since there isn't a GUI way of configuring it, I looked back at your host config and looked at the wiki entry on how to configure it. There is one thing that may be off and possibly contributing.

https://pve.proxmox.com/wiki/Network_Model#Masquerading_.28NAT.29

The wiki has the NAT bridge not actually bridged to a physical interface, but you do.

Code: 
auto lo
iface lo inet loopback

auto eth0
#real IP adress
iface eth0 inet static
        address  192.168.10.2
        netmask  255.255.255.0
        gateway  192.168.10.1

auto vmbr0
#private sub network
iface vmbr0 inet static
        address  10.10.10.1
        netmask  255.255.255.0
        bridge_ports none
        bridge_stp off
        bridge_fd 0

        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o eth0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o eth0 -j MASQUERADE

bridge_ports none

Code: 
auto lo
iface lo inet loopback

iface lo inet6 loopback

auto eth0
iface eth0 inet static
address 88.99.241.206
netmask 255.255.255.192
gateway 88.99.241.193
pointopoint 88.99.241.193
up route add -net 88.99.241.192 netmask 255.255.255.192 gw 88.99.241.193 dev eth0
# route 88.99.241.192/26 via 88.99.241.193

iface eth0 inet6 static
address 2a01:4f8:10b:215::2
netmask 64
gateway fe80::1

auto vmbr0
iface vmbr0 inet static
address 10.10.10.1
netmask 255.255.255.0
bridge_ports eth0
bridge_stp off
bridge_fd 0

post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o eth0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o eth0 -j MASQUERADE

bridge_ports eth0

I'd say try creating a dedicated NAT bridge that isn't also bridged to a physical interface and see if that makes a difference.
 
Not really cool workaround but I will try it in a couple days and will report back - But why would it make a difference for one VM over the others which currently do not have that problem at all? I mean it's still using the same Hardware/Software/Settings....
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back