I have two severs (px1 and px2) which connect to same switch (Juniper QFX) with LACP. each server have 2 physical ports and those bonded to bond0 using LACP. I used default Linux bond (not OVS).
After create the bond I created VLAN interface on that bond like below
first server (px1)
auto bond0
iface bond0 inet manual
bond-slaves ens1f0 ens1f1
bond-miimon 100
bond-mode 802.3ad
auto bond0.40
iface bond0.40 inet static
address 192.168.40.10
netmask 24
second server (px2)
auto bond0
iface bond0 inet manual
bond-slaves ens1f0 ens1f1
bond-miimon 100
bond-mode 802.3ad
auto bond0.40
iface bond0.40 inet static
address 192.168.40.11
netmask 24
After this when I try to ping px2 to px1 I cannot ping. Then I do the tcpdump on px1 while pinging from px2.
tcpdump -nneqt -i any arp | grep 192.168.40.10
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
(I did the tcpdump on px2 while pinging from px1 and results are same)
I can see ARP request coming from server px2 but no reply to those messages from server px1.
Then I tried adding MACs manually using arp -s command on both servers and I able to ping after that.
I don't have any visibility from the switch end what are the configurations. But since both servers can see incoming ARP requests why there is no ARP reply for those messages ?
To ensure this setup working without bond, I setup 3rd physical connection to the both servers and tag vlan 50 using OVSintport. I can access the VLAN 50 and I can see both ARP request and reply coming through that connection. and I can access all the hosts on that network (192.168.50.0/24)
Proxmox v6.0
update
----------------------------
I added MTU 9000 on bond0 and bond.40 as found in some posts. But still not working.
update 2
----------------------------
When I remove one interface from bond I can reach all hosts in 192.168.40.0/24 VLAN. and ARP reply is sending properly.
After create the bond I created VLAN interface on that bond like below
first server (px1)
auto bond0
iface bond0 inet manual
bond-slaves ens1f0 ens1f1
bond-miimon 100
bond-mode 802.3ad
auto bond0.40
iface bond0.40 inet static
address 192.168.40.10
netmask 24
second server (px2)
auto bond0
iface bond0 inet manual
bond-slaves ens1f0 ens1f1
bond-miimon 100
bond-mode 802.3ad
auto bond0.40
iface bond0.40 inet static
address 192.168.40.11
netmask 24
After this when I try to ping px2 to px1 I cannot ping. Then I do the tcpdump on px1 while pinging from px2.
tcpdump -nneqt -i any arp | grep 192.168.40.10
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
B 48:xxxxxxxxxxxxx vlan 40, p 0, Request who-has 192.168.40.10 tell 192.168.40.11, length 46
(I did the tcpdump on px2 while pinging from px1 and results are same)
I can see ARP request coming from server px2 but no reply to those messages from server px1.
Then I tried adding MACs manually using arp -s command on both servers and I able to ping after that.
I don't have any visibility from the switch end what are the configurations. But since both servers can see incoming ARP requests why there is no ARP reply for those messages ?
To ensure this setup working without bond, I setup 3rd physical connection to the both servers and tag vlan 50 using OVSintport. I can access the VLAN 50 and I can see both ARP request and reply coming through that connection. and I can access all the hosts on that network (192.168.50.0/24)
Proxmox v6.0
update
----------------------------
I added MTU 9000 on bond0 and bond.40 as found in some posts. But still not working.
update 2
----------------------------
When I remove one interface from bond I can reach all hosts in 192.168.40.0/24 VLAN. and ARP reply is sending properly.
Last edited:
