cannot trace bridge from virtual machines

B

behnam

Member
Jul 29, 2017
19
0
6
37
Hi, I have a proxmox which have 4 VMs. I defined 3 vmbr(0,2,3). vmbr0 is connected to the Internet and vmbr2 and vmbr3 are connected to two separate physical interfaces. Each VM has 3 interfaces which are connected to the virtual bridges. I have latency in VMs when I ping other VMs and two dedicated servers which are connected to the vmbr2 and vmbr3.
(192.168.3.2, 192.168.3.5 are my VMs and 192.168.3.254 is vmbr3)

ping 192.168.3.254
PING 192.168.3.254 (192.168.3.254) 56(84) bytes of data.
64 bytes from 192.168.3.254: icmp_seq=1 ttl=64 time=0.162 ms
64 bytes from 192.168.3.254: icmp_seq=2 ttl=64 time=0.205 ms
64 bytes from 192.168.3.254: icmp_seq=3 ttl=64 time=0.196 ms
64 bytes from 192.168.3.254: icmp_seq=4 ttl=64 time=0.228 ms
64 bytes from 192.168.3.254: icmp_seq=5 ttl=64 time=0.234 ms
64 bytes from 192.168.3.254: icmp_seq=6 ttl=64 time=0.160 ms

ping 192.168.3.5
PING 192.168.3.5 (192.168.3.5) 56(84) bytes of data.
64 bytes from 192.168.3.5: icmp_seq=1 ttl=64 time=0.372 ms
64 bytes from 192.168.3.5: icmp_seq=2 ttl=64 time=0.341 ms
64 bytes from 192.168.3.5: icmp_seq=3 ttl=64 time=0.445 ms
64 bytes from 192.168.3.5: icmp_seq=4 ttl=64 time=0.505 ms
64 bytes from 192.168.3.5: icmp_seq=5 ttl=64 time=0.439 ms
64 bytes from 192.168.3.5: icmp_seq=6 ttl=64 time=0.388 ms
64 bytes from 192.168.3.5: icmp_seq=7 ttl=64 time=0.465 ms

Another issue is that when I trace the VMs and dedicated servers from proxmox, the result is OK:

traceroute 192.168.3.2
traceroute to 192.168.3.2 (192.168.3.2), 30 hops max, 60 byte packets
1 192.168.3.2 (192.168.3.2) 0.646 ms 0.593 ms 196.995 ms

But when I trace the vmbr2 from any VMs or dedicated servers the results is as follows:

traceroute 192.168.3.254
traceroute to 192.168.3.254 (192.168.3.254), 30 hops max, 60 byte packets
1 * * *
2 * * *
3 * * *
4 * * *
5 * * *
6 * * *
7 * * *
8 * * *
9 * * *
10 * * *
11 * * *
12 * * *
13 * * *
14 * * *
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *
I disabled and stopped all firewalls in proxmox and VMs. I also checked the packets with tcpdump.
tcpdump output when trace from proxmox:
tcpdump -i eth1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
14:10:57.993188 IP 192.168.3.254.37751 > Web1.server.com.traceroute: UDP, length 32
14:10:57.993289 IP Web1.server.com > 192.168.3.254: ICMP Web1.server.com udp port traceroute unreachable, length 68
14:10:57.993308 IP 192.168.3.254.43559 > Web1.server.com.33435: UDP, length 32
14:10:57.993359 IP Web1.server.com > 192.168.3.254: ICMP Web1.server.com udp port 33435 unreachable, length 68
14:10:57.993367 IP 192.168.3.254.45694 > Web1.server.com.33436: UDP, length 32
14:10:57.993753 IP Web1.server.com > 192.168.3.254: ICMP Web1.server.com udp port 33436 unreachable, length 68
14:10:57.993768 IP 192.168.3.254.54165 > Web1.server.com.33437: UDP, length 32
14:10:57.993872 IP Web1.server.com > 192.168.3.254: ICMP Web1.server.com udp port 33437 unreachable, length 68
14:10:57.993885 IP 192.168.3.254.45581 > Web1.server.com.33438: UDP, length 32
14:10:57.993969 IP Web1.server.com > 192.168.3.254: ICMP Web1.server.com udp port 33438 unreachable, length 68
14:10:57.993981 IP 192.168.3.254.57234 > Web1.server.com.33439: UDP, length 32
14:10:57.994449 IP Web1.server.com > 192.168.3.254: ICMP Web1.server.com udp port 33439 unreachable, length 68
14:10:57.994462 IP 192.168.3.254.51722 > Web1.server.com.33440: UDP, length 32
14:10:57.994532 IP 192.168.3.254.35600 > Web1.server.com.33441: UDP, length 32
14:10:57.994572 IP 192.168.3.254.33030 > Web1.server.com.33442: UDP, length 32
14:10:57.994619 IP 192.168.3.254.42225 > Web1.server.com.33443: UDP, length 32
14:10:57.994666 IP 192.168.3.254.52295 > Web1.server.com.33444: UDP, length 32
14:10:57.994714 IP 192.168.3.254.35283 > Web1.server.com.33445: UDP, length 32
14:10:57.994757 IP 192.168.3.254.41124 > Web1.server.com.33446: UDP, length 32
14:10:57.994793 IP 192.168.3.254.56475 > Web1.server.com.33447: UDP, length 32
14:10:57.994840 IP 192.168.3.254.51230 > Web1.server.com.33448: UDP, length 32
14:10:57.994886 IP 192.168.3.254.42721 > Web1.server.com.33449: UDP, length 32
14:11:03.009524 ARP, Request who-has 192.168.3.254 tell Web1.server.com, length 28
14:11:03.009720 ARP, Reply 192.168.3.254 is-at 44:1e:a1:55:1c:7a (oui Unknown), length 28
14:11:03.101853 ARP, Request who-has Web1.server.com tell 192.168.3.254, length 28
14:11:03.101869 ARP, Reply Web1.server.com is-at d2:a1:a1:60:6d:c6 (oui Unknown), length 28
------------------------------
tcpdump output when trace from a VM:
tcpdump -i vmbr3 udp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmbr3, link-type EN10MB (Ethernet), capture size 262144 bytes
14:12:58.922032 IP 192.168.3.2.54906 > 192.168.3.254.33434: UDP, length 32
14:12:58.922089 IP 192.168.3.2.32991 > 192.168.3.254.33435: UDP, length 32
14:12:58.922221 IP 192.168.3.2.37432 > 192.168.3.254.33436: UDP, length 32
14:12:58.922310 IP 192.168.3.2.42272 > 192.168.3.254.33437: UDP, length 32
14:12:58.922392 IP 192.168.3.2.45519 > 192.168.3.254.33438: UDP, length 32
14:12:58.922477 IP 192.168.3.2.60021 > 192.168.3.254.33439: UDP, length 32
14:12:58.922565 IP 192.168.3.2.59216 > 192.168.3.254.33440: UDP, length 32
14:13:03.933719 IP 192.168.3.2.58922 > 192.168.3.254.33450: UDP, length 32
14:13:03.933797 IP 192.168.3.2.41332 > 192.168.3.254.33451: UDP, length 32
14:13:03.933868 IP 192.168.3.2.44173 > 192.168.3.254.33452: UDP, length 32
14:13:03.933937 IP 192.168.3.2.33406 > 192.168.3.254.33453: UDP, length 32
14:13:03.933995 IP 192.168.3.2.50590 > 192.168.3.254.33454: UDP, length 32
14:13:03.934073 IP 192.168.3.2.54211 > 192.168.3.254.33455: UDP, length 32
14:13:03.934176 IP 192.168.3.2.44997 > 192.168.3.254.33456: UDP, length 32
14:13:03.934245 IP 192.168.3.2.33904 > 192.168.3.254.33457: UDP, length 32
14:13:03.934287 IP 192.168.3.2.54948 > 192.168.3.254.33458: UDP, length 32
14:13:03.934389 IP 192.168.3.2.42724 > 192.168.3.254.33459: UDP, length 32
14:13:03.934470 IP 192.168.3.2.34022 > 192.168.3.254.33460: UDP, length 32
14:13:03.934545 IP 192.168.3.2.47004 > 192.168.3.254.33461: UDP, length 32
14:13:03.934624 IP 192.168.3.2.40366 > 192.168.3.254.33462: UDP, length 32
14:13:03.934706 IP 192.168.3.2.36526 > 192.168.3.254.33463: UDP, length 32
14:13:03.934786 IP 192.168.3.2.49021 > 192.168.3.254.33464: UDP, length 32
14:13:03.934854 IP 192.168.3.2.49145 > 192.168.3.254.33465: UDP, length 32
-------------------

here's the network configuration in proxmox:
auto lo
iface lo inet loopback

iface enp3s0f0 inet manual

iface enp3s0f1 inet manual

iface enp4s0f0 inet manual

iface enp4s0f1 inet manual

auto vmbr0
iface vmbr0 inet static
address X.X.X.X
netmask 255.255.255.240
gateway X.X.X.Y
bridge_ports enp3s0f0
bridge_stp off
bridge_fd 0

auto vmbr2
iface vmbr2 inet static
address 192.168.2.254
netmask 255.255.255.0
bridge_ports enp4s0f0
bridge_stp off
bridge_fd 0

auto vmbr3
iface vmbr3 inet static
address 192.168.3.254
netmask 255.255.255.0
bridge_ports enp4s0f1
bridge_stp off
bridge_fd 0
--------------
Any help is appreciated.
 
behnam said:
Hi, I have a proxmox which have 4 VMs. I defined 3 vmbr(0,2,3). vmbr0 is connected to the Internet and vmbr2 and vmbr3 are connected to two separate physical interfaces. Each VM has 3 interfaces which are connected to the virtual bridges. I have latency in VMs when I ping other VMs and two dedicated servers which are connected to the vmbr2 and vmbr3.
(192.168.3.2, 192.168.3.5 are my VMs and 192.168.3.254 is vmbr3)

ping 192.168.3.254
PING 192.168.3.254 (192.168.3.254) 56(84) bytes of data.
64 bytes from 192.168.3.254: icmp_seq=1 ttl=64 time=0.162 ms
64 bytes from 192.168.3.254: icmp_seq=2 ttl=64 time=0.205 ms
64 bytes from 192.168.3.254: icmp_seq=3 ttl=64 time=0.196 ms
64 bytes from 192.168.3.254: icmp_seq=4 ttl=64 time=0.228 ms
64 bytes from 192.168.3.254: icmp_seq=5 ttl=64 time=0.234 ms
64 bytes from 192.168.3.254: icmp_seq=6 ttl=64 time=0.160 ms

ping 192.168.3.5
PING 192.168.3.5 (192.168.3.5) 56(84) bytes of data.
64 bytes from 192.168.3.5: icmp_seq=1 ttl=64 time=0.372 ms
64 bytes from 192.168.3.5: icmp_seq=2 ttl=64 time=0.341 ms
64 bytes from 192.168.3.5: icmp_seq=3 ttl=64 time=0.445 ms
64 bytes from 192.168.3.5: icmp_seq=4 ttl=64 time=0.505 ms
64 bytes from 192.168.3.5: icmp_seq=5 ttl=64 time=0.439 ms
64 bytes from 192.168.3.5: icmp_seq=6 ttl=64 time=0.388 ms
64 bytes from 192.168.3.5: icmp_seq=7 ttl=64 time=0.465 ms
Click to expand...

Ping reply < 1ms is normal in a physical network, in a virtual one it's usually < 0.2 ms but depends on actual load.

behnam said:
Another issue is that when I trace the VMs and dedicated servers from proxmox, the result is OK:

traceroute 192.168.3.2
traceroute to 192.168.3.2 (192.168.3.2), 30 hops max, 60 byte packets
1 192.168.3.2 (192.168.3.2) 0.646 ms 0.593 ms 196.995 ms

But when I trace the vmbr2 from any VMs or dedicated servers the results is as follows:

traceroute 192.168.3.254
traceroute to 192.168.3.254 (192.168.3.254), 30 hops max, 60 byte packets
1 * * *
2 * * *
3 * * *
4 * * *
5 * * *
6 * * *
7 * * *
8 * * *
9 * * *
10 * * *
11 * * *
12 * * *
13 * * *
14 * * *
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *
I disabled and stopped all firewalls in proxmox and VMs. I also checked the packets with tcpdump.
tcpdump output when trace from proxmox:
tcpdump -i eth1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
14:10:57.993188 IP 192.168.3.254.37751 > Web1.server.com.traceroute: UDP, length 32
14:10:57.993289 IP Web1.server.com > 192.168.3.254: ICMP Web1.server.com udp port traceroute unreachable, length 68
14:10:57.993308 IP 192.168.3.254.43559 > Web1.server.com.33435: UDP, length 32
14:10:57.993359 IP Web1.server.com > 192.168.3.254: ICMP Web1.server.com udp port 33435 unreachable, length 68
14:10:57.993367 IP 192.168.3.254.45694 > Web1.server.com.33436: UDP, length 32
14:10:57.993753 IP Web1.server.com > 192.168.3.254: ICMP Web1.server.com udp port 33436 unreachable, length 68
14:10:57.993768 IP 192.168.3.254.54165 > Web1.server.com.33437: UDP, length 32
14:10:57.993872 IP Web1.server.com > 192.168.3.254: ICMP Web1.server.com udp port 33437 unreachable, length 68
14:10:57.993885 IP 192.168.3.254.45581 > Web1.server.com.33438: UDP, length 32
14:10:57.993969 IP Web1.server.com > 192.168.3.254: ICMP Web1.server.com udp port 33438 unreachable, length 68
14:10:57.993981 IP 192.168.3.254.57234 > Web1.server.com.33439: UDP, length 32
14:10:57.994449 IP Web1.server.com > 192.168.3.254: ICMP Web1.server.com udp port 33439 unreachable, length 68
14:10:57.994462 IP 192.168.3.254.51722 > Web1.server.com.33440: UDP, length 32
14:10:57.994532 IP 192.168.3.254.35600 > Web1.server.com.33441: UDP, length 32
14:10:57.994572 IP 192.168.3.254.33030 > Web1.server.com.33442: UDP, length 32
14:10:57.994619 IP 192.168.3.254.42225 > Web1.server.com.33443: UDP, length 32
14:10:57.994666 IP 192.168.3.254.52295 > Web1.server.com.33444: UDP, length 32
14:10:57.994714 IP 192.168.3.254.35283 > Web1.server.com.33445: UDP, length 32
14:10:57.994757 IP 192.168.3.254.41124 > Web1.server.com.33446: UDP, length 32
14:10:57.994793 IP 192.168.3.254.56475 > Web1.server.com.33447: UDP, length 32
14:10:57.994840 IP 192.168.3.254.51230 > Web1.server.com.33448: UDP, length 32
14:10:57.994886 IP 192.168.3.254.42721 > Web1.server.com.33449: UDP, length 32
14:11:03.009524 ARP, Request who-has 192.168.3.254 tell Web1.server.com, length 28
14:11:03.009720 ARP, Reply 192.168.3.254 is-at 44:1e:a1:55:1c:7a (oui Unknown), length 28
14:11:03.101853 ARP, Request who-has Web1.server.com tell 192.168.3.254, length 28
14:11:03.101869 ARP, Reply Web1.server.com is-at d2:a1:a1:60:6d:c6 (oui Unknown), length 28
------------------------------
tcpdump output when trace from a VM:
tcpdump -i vmbr3 udp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmbr3, link-type EN10MB (Ethernet), capture size 262144 bytes
Click to expand...

If you trace just udp messages you will not see "ICMP ... udp port 33439 unreachable".
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back