Internal network latency

[dmarlow]

Hello. Are there any recommendations to help improve network latency? I'm using the latest version of PVE to test out OpenWRT in LXC to replace an aging router. Using Smokeping, I see my current router is around 200 microseconds and all network interfaces of my proxmox server is around 575 microseconds. I'm going to try a completely separate distro using a live image to see if latency is just what it is for this machine or if it's different. Anything I should try or look at?
Thanks!

 

[Richard]

Hello. Are there any recommendations to help improve network latency? I'm using the latest version of PVE to test out OpenWRT in LXC to replace an aging router. Using Smokeping, I see my current router is around 200 microseconds and all network interfaces of my proxmox server is around 575 microseconds. I'm going to try a completely separate distro using a live image to see if latency is just what it is for this machine or if it's different. Anything I should try or look at?
Thanks!

What exactly did you check: VM(Container) to host, host to external or VM(container) to external? What is the result when you compare the figures of these 3 scenarios?

 

[dmarlow]

What exactly did you check: VM(Container) to host, host to external or VM(container) to external? What is the result when you compare the figures of these 3 scenarios?

Host has 3 nics, one is used only for proxmox, the other 2 are used for testing things like OPNSense/OpenWRT (WAN/LAN).

External host (Smokeping on Synology NAS) to Proxmox: 0.540 ms avg
External host (Smokeping on Synology NAS) to OpenWRT LXC: 0.562 ms avg

External (using Ubuntu via gigabit Eth) host to Proxmox: rtt min/avg/max/mdev = 0.443/0.787/1.038/0.145 ms
External (using Ubuntu via gigabit Eth) host to OpenWRT LXC: rtt min/avg/max/mdev = 0.579/0.931/1.289/0.179 ms

Proxmox to OpenWRT LXC container: rtt min/avg/max/mdev = 0.188/0.292/0.659/0.041 ms
Proxmox to Synology NAS (what runs Smokeping in a container): rtt min/avg/max/mdev = 0.188/0.220/0.254/0.015 ms

OpenWRT container to Synology NAS: round-trip min/avg/max = 0.218/0.306/0.333 ms
OpenWRT container to Proxmox: round-trip min/avg/max = 0.190/0.327/0.355 ms

Conclusion is that outbound from Proxmox is more stable than inbound. If there's a setting or something I can try, I will give it a shot.

 

[Richard]

Conclusion is that outbound from Proxmox is more stable than inbound. If there's a setting or something I can try, I will give it a shot.

In principle yes (I would say "faster" not "more stable"), but comparision shows also that is an issue of the physical NIC(s ?) not of virtualization. Since between external machines and Proxmox host there are probably 1 or more switches it is not necessarily caused by the NIC in Proxmox host or it's software.
As a next step for narrowing the problem I would:
- (temporarily) connect the Proxmox Host NIC directly with a machine (i.e. without switch between) and check whether in this case the difference between outbound and inbound is aslo given
- if yes (if no it's definitely caused by switch(es)) try it by loading a different OS (e.g. Ubuntu Live) into your Proxmox Host and do the same test again. Only if the results are different it's caused by firmware/software used for Proxmox.

Moreover there are relatively big differences between min and max latency, looks like some traffic load is on your network and in such a case also differences between inbound and outbound are quite usual.