proxmox containers - multiple NICs and strange network issue

[Joby Mathai]

Hi,

My proxmox host machine is having three network bridges 10.0.27.x/24 , 10.0.28.x/24 and 10.0.30.x series which are connected to three different networks and my guest containers are having multiple NICs which will be using these network bridges according to the network requirement. Since multiple NICs are used in containers, we are adding routes in guest machines to route the traffic to specific network device based on IP.

Issue::

Eg: Suppose my proxmox guest container is having IPs 10.0.30.160(eth0) and 10.0.27.160(eth1) . 30.x is the default route of the container. When we try to ping the 10.0.27.160 from another remote physical server having IP 10.0.27.30, it's not able to connect to 10.0.27.160 but default network 30.x is connecting fine. But when we do a ping from guest machine to IP 10.0.27.1 ( gateway IP ) , suddenly the remote server starts pinging to 10.0.27.160. ie, it is getting connection to guest machine in the 27.x network. Again after sometime, connectivity gets lost. Repeating the ping exercise again brings back the connectivity. What exactly am I doing wrong here ?? or any specific network settings needed to be applied or checked?? Anybody experienced similar situation?

 

[Richard]

Hi,

My proxmox host machine is having three network bridges 10.0.27.x/24 , 10.0.28.x/24 and 10.0.30.x series which are connected to three different networks and my guest containers are having multiple NICs which will be using these network bridges according to the network requirement. Since multiple NICs are used in containers, we are adding routes in guest machines to route the traffic to specific network device based on IP.

Issue::

Eg: Suppose my proxmox guest container is having IPs 10.0.30.160(eth0) and 10.0.27.160(eth1) . 30.x is the default route of the container. When we try to ping the 10.0.27.160 from another remote physical server having IP 10.0.27.30, it's not able to connect to 10.0.27.160 but default network 30.x is connecting fine. But when we do a ping from guest machine to IP 10.0.27.1 ( gateway IP ) , suddenly the remote server starts pinging to 10.0.27.160. ie, it is getting connection to guest machine in the 27.x network. Again after sometime, connectivity gets lost. Repeating the ping exercise again brings back the connectivity. What exactly am I doing wrong here ?? or any specific network settings needed to be applied or checked?? Anybody experienced similar situation?

First of all check your LAN cabling and firewall settings In any case use different switches for different subnets a deactivate (for a test) all firewalls.

If the problem persists track the IP packets via tcpdump (i.e. in container, Proxmox host, "another remote physical server") in order to figure out where they get lost.

 

[Joby Mathai]

But the strange thing is, issue get solved with a ping and the problem repeats only after, say once in two months or so. Any way I will check with tcpdump. Thanks for your suggestions

 

[Richard]

But the strange thing is, issue get solved with a ping and the problem repeats only after, say once in two months or so. Any way I will check with tcpdump. Thanks for your suggestions

It can happen since firewalls as well as intelligent switches may react on packets changing rules for blocking/allowing packets. Therefore the hint: remove all firewalls and use simple unmanaged switches (at least temporarily for a test).