Increasing the nic card speed to 10 gb.

[sherani1987]

Hi everyone
Is it possible to increase the speed of nic card of proxmox or vm to 10 gb. We have setup of 10 gb fiber optic.

 

[ph0x]

Proxmox does not restrict the NIC speed unless you set it in the VM's options.

 

[sherani1987]

Proxmox does not restrict the NIC speed unless you set it in the VM's options.

we have 10 gib pipe connect to proxmox machine but whenever we copy files from proxmox ve to other machine it speed is only 1 gib.

 

[Dunuin]

Are you copying files from inside a VM or from the host itself? Without passthrough it is possible that the virtio NICs can't handle 10G and the transfer speed is limited by the speed of the CPU virtualizing the NICs and bridges.
And are both storages SSDs and fast enough to support writes/reads with more 1Gbit?

And did you verified that all NICs and Swich ports are working in 10G SFP+ and not in 1G SFP mode?

 

[sherani1987]

Are you copying files from inside a VM or from the host itself? Without passthrough it is possible that the virtio NICs can't handle 10G and the transfer speed is limited by the speed of the CPU virtualizing the NICs and bridges.
And are both storages SSDs and fast enough to support writes/reads with more 1Gbit?

And did you verified that all NICs and Swich ports are working in 10G SFP+ and not in 1G SFP mode?

I am copying from proxmox host and also from vm. But the speed is same 1 gib. Both storages are ssd. How to verify the nics and switchport are working in 10gb.

 

[Dunuin]

I am copying from proxmox host and also from vm. But the speed is same 1 gib. Both storages are ssd. How to verify the nics and switchport are working in 10gb.

Are you using a switch in between? What NIC model are you using?

Switching to jumbo frames also increased my speed. Without it my CPU wasn't able to handle the amount of packets and I only got 6 and not 10 Gbit.
iperf3 is a great tool if you want to benchmark the throughput because it isn't limited by the storage.

 

[sherani1987]

Yes there is switch between of 10gb. Nics on proxmox machine is intel 10gigabyte

 

[Ramalama]

ethtool enpxxxxx
Shows your linkspeed. What's the output?

 

[sherani1987]

ethtool enpxxxxx
Shows your linkspeed. What's the output?

this is the snapshot for both nic cards

 

[Ramalama]

Well, this looks good.

So switch+server are indeed with 10gb connected.
Then it's either, that something on your way has 1gbs, if you copy from server to your pc, probably your pc?

Or what is possible either, there are a ton of fake x550/x540 intel cards out (pcie cards).
But even this fake cards are working and do around 3-4gb/s.

 

[spirit]

we have 10 gib pipe connect to proxmox machine but whenever we copy files from proxmox ve to other machine it speed is only 1 gib.

how do you copy between proxmox and vm ?
because if it's through ssh/scp, you'll be limited by cpu, 1 scp transfert can't use more than 1 core.

 

[sherani1987]

Well, this looks good.

So switch+server are indeed with 10gb connected.
Then it's either, that something on your way has 1gbs, if you copy from server to your pc, probably your pc?

Or what is possible either, there are a ton of fake x550/x540 intel cards out (pcie cards).
But even this fake cards are working and do around 3-4gb/s.

Yup..... eventhough if it is fake. It should work atleast 2-3 gb.

 

[sherani1987]

how do you copy between proxmox and vm ?
because if it's through ssh/scp, you'll be limited by cpu, 1 scp transfert can't use more than 1 core.

I am copying from proxmox to another linux server

 

[Ramalama]

Check your way, there must be something that is connected through 1gb/s.

Ethtool doesn't lies, and virtio/linux bridge doesn't bottlenecks for sure.

Do you probably have an shared 1gb/s bmc/ipmi nic on that board?
Can you unplug it?

 

[sherani1987]

Check your way, there must be something that is connected through 1gb/s.

Ethtool doesn't lies, and virtio/linux bridge doesn't bottlenecks for sure.

Do you probably have an shared 1gb/s bmc/ipmi nic on that board?
Can you unplug it?

Should i unplug it on another linux machine or proxmox machine

 

[Ramalama]

If both have an shared ipmi/bmc nic, both.
I don't know where you have the misconfiguration, probably on the other linux machine if it takes the route over that 1gb/s nic, instead of over the 10gb/s

 

[sherani1987]

If both have an shared ipmi/bmc nic, both.
I don't know where you have the misconfiguration, probably on the other linux machine if it takes the route over that 1gb/s nic, instead of over the 10gb/s

Ok... i wil check it and wil let you.... thanks

 

[Ramalama]

I need to get some sleep.
And test probably to copy a file from proxmox /tmp to linix-server /tmp, to exclude slow file storage probably (if your storage comes from network for example either) xD

Good night

 

[Dunuin]

As I already said iperf3 is nice to benchmark the connection. You are not limited by the storage and you can tell it what interface, protocol and port to use. Run it on one host in server mode and on one host in client mode and verify that the 10G NICs are selected as interface.And look what speed you get.
https://iperf.fr/iperf-doc.php