Have been playing with EVPN in Lab setup. Have 3 proxmox hosts, each eBGP peered to a TOR switches.
Initially I was trying to do exit node on each host, however I was running into a weird problem that the /32 for each vm was being advertised to the TOR that host wasn't peered to.
I couldn't get this working, so i moved to setting up a VM on each host to be the eVPN peer for each host, and that vyos vm also then doing the eBGP for ipv4 to the TORs. ( Using vyos 1.4 for these).
After a bit of configuring I've got the vyos vms working as exit nodes for the SDN all ok, with the proxmox successfully host peered to these, and successfully have outbound traffic to rest of network from the vms, as well as east west between vms on different hosts.
However.. running iperf between vms on different hosts shows v low through put (3-4mbits) and really high tcp retries. But if i run any of these out to the rest of the network, the results are ok. Iperf between the hosts is totally fine.
I've also tried connecting a vm on each host directly into the vxlan via bridge interfaces on the vyos exit routers, and this works ok too.
physical MTU is set to 2000 on the host interfaces and vyos vm interfaces.
mtu on vxlan is set to 1500, and vms are set to 1500.
Update 1.
Mystery continues. iperf performance is only bad in one direction. vm1 -> vm2: good, vm2->vm1 bad. However, if i migrate each vm to the other host, it flips - ie the "bad" direction is always from vms on host A to host B. However, iperf directly between hosts is fine in either direction..
ping between vms in either direction is solid.
Initially I was trying to do exit node on each host, however I was running into a weird problem that the /32 for each vm was being advertised to the TOR that host wasn't peered to.
I couldn't get this working, so i moved to setting up a VM on each host to be the eVPN peer for each host, and that vyos vm also then doing the eBGP for ipv4 to the TORs. ( Using vyos 1.4 for these).
After a bit of configuring I've got the vyos vms working as exit nodes for the SDN all ok, with the proxmox successfully host peered to these, and successfully have outbound traffic to rest of network from the vms, as well as east west between vms on different hosts.
However.. running iperf between vms on different hosts shows v low through put (3-4mbits) and really high tcp retries. But if i run any of these out to the rest of the network, the results are ok. Iperf between the hosts is totally fine.
I've also tried connecting a vm on each host directly into the vxlan via bridge interfaces on the vyos exit routers, and this works ok too.
physical MTU is set to 2000 on the host interfaces and vyos vm interfaces.
mtu on vxlan is set to 1500, and vms are set to 1500.
Update 1.
Mystery continues. iperf performance is only bad in one direction. vm1 -> vm2: good, vm2->vm1 bad. However, if i migrate each vm to the other host, it flips - ie the "bad" direction is always from vms on host A to host B. However, iperf directly between hosts is fine in either direction..
ping between vms in either direction is solid.
Last edited: