Hello all!
I'm confused, because I don't know in which way should I go to look for solutions. Let me describe my problem as simple as I can. It's about network issues. This is schema:
Description:
In upper picture we have two routers, which are working on one PVE and they are separated by added NICs which belong to different VLANs. The routers sees their NICs as normal ethernet (eth0, eth1). They don't sees/makes VLANs. Traffic from LAN is going through Router2 and then Router1, which have a connection to the Internet. All addresses here are public (I have a 1.1.1.0/24 public IPs). Router1 have trace to the 1.1.1.64/30 via 1.1.1.2 on eth0. Generally routing is fine - checked hundreds times. Routers are working on CentOS. When I'm on Host1 I have a Internet connection, sites are working well, no problems at first look.
The problem:
Ping to Host1 from Router1 and from the world doesn't back. It will start to be more curious when I tell You that ping request are actually get into Host1! I have no idea why host just doesn't give an answers to it. It has been checked on different computers and routers. It's not a firewall on Host1 problem, because pings from Routers2 or from network of Host1 get answers. Let me show You some screenshots (213.216.77.125 is like Host1 here; 213.216.77.1 is like Router1; 213.216.77.121 is like Router2):
I didn't find differences between ICMP request packets from the first Router2 and from other hosts behind Router2 :/
Until now I was working with NAT on Router2 and it was fine. Now, I have to give a real public IPs at some hosts in my LAN.
And now...
When I do a VM on this ProxmoxVE and when I set it up exactly like Host1 then pings works fine! So, the problem is on PVE or on VLANs or on hardware.
Hardware:
I'm using a Cisco SLM224G switch. There is no so much options to screw up, also the newest firmware is there.
Hardware for PVE is Dell PowerEdge R410 on 2xXeon 5500 series.
ProxmoxVE & VLANs:
It's the newest version 1.9, but on 1.7 was the same problem. Both routers are running as KVMs on CentOS (checked on 5 an 6). All VLANs are made by Proxmox and given as NICs to KVMs. I'm using VIRTIO, but tried also Realtek and e1000 emulation (no changes at all). Network configure:
/etc/network/interfaces
ip link show
I give up! It's killing me from weeks. I can't get a properly working internet connection on Host1 because of this. I think that the problem is somewhere on OSI Layer 2, but results are in Layer 3 somehow. Or maybe something is going wrong on Layer 3 only, it's hard to say for me. It starts between VM and Host1, so we have PVE or switch, but more people would say that issue is somewhere in PVE, maybe in method of packets conversions and outgoing.
All in all, now I even don't know where should I dig - is it a issue with PVE, VLANs, switch, hardware or something more? Please, somebody help me! I will be appreciate for any ideas.
Best Regards,
syd
I'm confused, because I don't know in which way should I go to look for solutions. Let me describe my problem as simple as I can. It's about network issues. This is schema:
Code:
_____________________________________________________________
| |
| _______________________ ______________________ |
| |eth0(11) eth1(33)| |eth0(33) eth1(22)| |
| |1.1.1.65/30 1.1.1.2/30| ------ |1.1.1.1/30 1.0.0.2/30| |
| |________Router2________| |________Router1_______| |
| |
| |
|______eth0_______________ProxmoxVE________________eth1_______|
| |
| |
| |
| |
___|___________________________________________|____
| e1 | e2 | e3 | e4 | e5 | e6 |
| id:11 | id:11 | | | id:22 | id:22 |
| tagged |untagged| | |untagged| tagged |
| | | | | |
|________|___|____|_____Switch_____|___|____|________|
| |
| |
| |
LAN WAN
______|______ |
| eth0 | INTERNET
| 1.1.1.66/30 | 1.0.0.1/30
|____Host1____| |Description:
In upper picture we have two routers, which are working on one PVE and they are separated by added NICs which belong to different VLANs. The routers sees their NICs as normal ethernet (eth0, eth1). They don't sees/makes VLANs. Traffic from LAN is going through Router2 and then Router1, which have a connection to the Internet. All addresses here are public (I have a 1.1.1.0/24 public IPs). Router1 have trace to the 1.1.1.64/30 via 1.1.1.2 on eth0. Generally routing is fine - checked hundreds times. Routers are working on CentOS. When I'm on Host1 I have a Internet connection, sites are working well, no problems at first look.
The problem:
Ping to Host1 from Router1 and from the world doesn't back. It will start to be more curious when I tell You that ping request are actually get into Host1! I have no idea why host just doesn't give an answers to it. It has been checked on different computers and routers. It's not a firewall on Host1 problem, because pings from Routers2 or from network of Host1 get answers. Let me show You some screenshots (213.216.77.125 is like Host1 here; 213.216.77.1 is like Router1; 213.216.77.121 is like Router2):
I didn't find differences between ICMP request packets from the first Router2 and from other hosts behind Router2 :/
Until now I was working with NAT on Router2 and it was fine. Now, I have to give a real public IPs at some hosts in my LAN.
And now...
When I do a VM on this ProxmoxVE and when I set it up exactly like Host1 then pings works fine! So, the problem is on PVE or on VLANs or on hardware.
Hardware:
I'm using a Cisco SLM224G switch. There is no so much options to screw up, also the newest firmware is there.
Hardware for PVE is Dell PowerEdge R410 on 2xXeon 5500 series.
ProxmoxVE & VLANs:
It's the newest version 1.9, but on 1.7 was the same problem. Both routers are running as KVMs on CentOS (checked on 5 an 6). All VLANs are made by Proxmox and given as NICs to KVMs. I'm using VIRTIO, but tried also Realtek and e1000 emulation (no changes at all). Network configure:
/etc/network/interfaces
Code:
# network interface settings
auto lo
iface lo inet loopback
iface eth0 inet manual
iface eth1 inet manual
auto vmbr0
iface vmbr0 inet static
address 172.16.1.10
netmask 255.255.255.0
bridge_ports eth0
bridge_stp off
bridge_fd 0
auto vmbr1
iface vmbr1 inet manual
bridge_ports eth1
bridge_stp off
bridge_fd 0
auto vmbr11
iface vmbr11 inet static
address 172.21.1.10
netmask 255.255.0.0
gateway 172.21.1.1
bridge_ports eth0.11
bridge_stp off
bridge_fd 0
auto vmbr22
iface vmbr22 inet manual
bridge_ports eth1.22
bridge_stp off
bridge_fd 0
auto vmbr33
iface vmbr33 inet manual
bridge_ports eth0.33
bridge_stp off
bridge_fd 0ip link show
Code:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:26:b9:3f:47:81 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:26:b9:3f:47:82 brd ff:ff:ff:ff:ff:ff
4: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether 00:26:b9:3f:47:81 brd ff:ff:ff:ff:ff:ff
5: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether 00:26:b9:3f:47:82 brd ff:ff:ff:ff:ff:ff
6: vmbr11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether 00:26:b9:3f:47:81 brd ff:ff:ff:ff:ff:ff
7: eth0.11@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 00:26:b9:3f:47:81 brd ff:ff:ff:ff:ff:ff
8: vmbr22: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether 00:26:b9:3f:47:82 brd ff:ff:ff:ff:ff:ff
9: eth1.22@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 00:26:b9:3f:47:82 brd ff:ff:ff:ff:ff:ff
10: vmbr33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether 00:26:b9:3f:47:81 brd ff:ff:ff:ff:ff:ff
11: eth0.33@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 00:26:b9:3f:47:81 brd ff:ff:ff:ff:ff:ff
12: venet0: <BROADCAST,POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/void
13: tap102i11d0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
link/ether 82:8b:f6:d8:47:71 brd ff:ff:ff:ff:ff:ff
14: tap102i33d0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
link/ether 42:2d:38:ff:4d:b5 brd ff:ff:ff:ff:ff:ff
15: tap101i22d0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
link/ether 3a:e6:a0:0a:91:39 brd ff:ff:ff:ff:ff:ff
16: tap101i33d0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
link/ether 7e:44:ab:24:6d:ba brd ff:ff:ff:ff:ff:ff
21: tap106i11d0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
link/ether 96:ff:5f:f1:0a:43 brd ff:ff:ff:ff:ff:ff
22: tap106i33d0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
link/ether 22:2c:a8:43:cb:41 brd ff:ff:ff:ff:ff:ff
25: tap105i11d0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
link/ether 2e:fe:29:49:34:e6 brd ff:ff:ff:ff:ff:ffI give up! It's killing me from weeks. I can't get a properly working internet connection on Host1 because of this. I think that the problem is somewhere on OSI Layer 2, but results are in Layer 3 somehow. Or maybe something is going wrong on Layer 3 only, it's hard to say for me. It starts between VM and Host1, so we have PVE or switch, but more people would say that issue is somewhere in PVE, maybe in method of packets conversions and outgoing.
All in all, now I even don't know where should I dig - is it a issue with PVE, VLANs, switch, hardware or something more? Please, somebody help me! I will be appreciate for any ideas.
Best Regards,
syd
Last edited:
