Hello everyone.
I'm making some changes to my cluster. Does this make sense, and is this the right approach overall?
I have three PVE 9.0.11 servers and shared storage with NFS (TrueNAS Scale 25.04.2.4) for VMs.
Each server and storage has one Intel XXV710-DA2 network card with a 25 Gbps connection.
Everything is connected via a switch; each server and storage is connected to the switch with aggregation of both ports.
Everything is on the same subnet, with a standard MTU.
What I want to change:
Add an Intel E810-XXVDA2 network card to each server and storage, redirect the management interface to it on each server, and also aggregate both channels, and connect the servers and storage separately through the same switch.
This channel will only be used to connect the servers and storage. Everything that's currently available remains and will be used for everything else, connecting to user VMs, etc., via a separate vmbr1, which will be bound to both ports of the Intel XXV710-DA2.
Can any interface and its assigned IP address be used for management and connections on port 8006?
If there are multiple interfaces, can I connect via any of them?
So there's no hard and fast rule that only this interface is for management and not others?
Will it be sufficient to set the MTU to 9000 only on the management interfaces of the vmbr0 servers and the TrueNAS storage, or do I need to configure something else in the network card settings via the console?
How is it determined which interface connects the storage to all cluster hosts if the hosts have multiple interfaces? I don't see an interface option in the storage connection settings.
I'll also add that a PBS is connected to this cluster.
But it's not connected to the same switch, but to another one higher up.
I understand that it will also need to be set to MTU 9000.
And in general, the question of whether this MTU is even worth it is a separate one. I see a lot of problems with it here on the forum.
Maybe I should do everything as I described, but leave the MTU as default?
I'm making some changes to my cluster. Does this make sense, and is this the right approach overall?
I have three PVE 9.0.11 servers and shared storage with NFS (TrueNAS Scale 25.04.2.4) for VMs.
Each server and storage has one Intel XXV710-DA2 network card with a 25 Gbps connection.
Everything is connected via a switch; each server and storage is connected to the switch with aggregation of both ports.
Everything is on the same subnet, with a standard MTU.
What I want to change:
Add an Intel E810-XXVDA2 network card to each server and storage, redirect the management interface to it on each server, and also aggregate both channels, and connect the servers and storage separately through the same switch.
This channel will only be used to connect the servers and storage. Everything that's currently available remains and will be used for everything else, connecting to user VMs, etc., via a separate vmbr1, which will be bound to both ports of the Intel XXV710-DA2.
Can any interface and its assigned IP address be used for management and connections on port 8006?
If there are multiple interfaces, can I connect via any of them?
So there's no hard and fast rule that only this interface is for management and not others?
Will it be sufficient to set the MTU to 9000 only on the management interfaces of the vmbr0 servers and the TrueNAS storage, or do I need to configure something else in the network card settings via the console?
How is it determined which interface connects the storage to all cluster hosts if the hosts have multiple interfaces? I don't see an interface option in the storage connection settings.
I'll also add that a PBS is connected to this cluster.
But it's not connected to the same switch, but to another one higher up.
I understand that it will also need to be set to MTU 9000.
And in general, the question of whether this MTU is even worth it is a separate one. I see a lot of problems with it here on the forum.
Maybe I should do everything as I described, but leave the MTU as default?
Last edited: