Hi guys,
I need help configuring my environment's network, I'm new to Proxmox and we are setting up a test environment (Proxmox) with 3 hosts that must meet all the requirements of a production environment with redundancy and performance from a network point of view . The goal is to disassemble our vmware cluster.
Important informations:
- We have two 10/25G Switches (Stack)
- Each Host has only one 10GbE card with two ports, each port on a different Switch with LACP configured and on each port we configure 50 different vlans
- iSCSI Storage with two different Network/vLAN
- Proxmox 8.2.4
- open-iscsi (2.1.8-1.pve1)
- multipath-tools (0.9.4-5~bpo11+1)
I understand that the best scenario would be to use bond for network fault-tolerant and increase the performance, mainly due to the performance of the iscsi network, but as I am not an expert I accept other scenarios as long as the performance of the iscsi network is at least equal to the current environment with vmware that has the same settings.
I tested some combinations using bridge and the disk access performance (iscsi) is much lower than the vmware environment, tests with iometer.
So now I'm trying to configure the environment with bond + linux vlan and I'm having problems, maybe because I don't have knowledge/experience.
- Management Network (vLAN1)
- iSCSI-10 Network (vLAN10)
- iSCSI-20 Network (vLAN20)
Just for information:
root@pve-01:~# multipath -l
MP-01 (36000d3100092d40000000000000006b8) dm-7 COMPELNT,Compellent Vol
size=1.0T features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=0 status=active
|- 9:0:0:5 sdf 8:80 active undef running
`- 7:0:0:5 sdb 8:16 active undef running
MP-02 (36000d3100092d40000000000000006b9) dm-6 COMPELNT,Compellent Vol
size=10T features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=0 status=active
|- 10:0:0:6 sde 8:64 active undef running
`- 8:0:0:6 sdc 8:32 active undef running
root@pve-01:~#
Follow the /etc/network/interfaces settings
auto it
iface lo inet loopback
self enp4s0f0
iface enp4s0f0 inet manual
self enp4s0f1
iface enp4s0f1 inet manual
auto bond0
iface bond0 inet manual
bond-slaves enp4s0f0 enp4s0f1
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2+3
iface bond0.10 inet manual
iface bond0.20 inet manual
iface bond0.30 inet manual
auto vmbr0v10
iface vmbr0v10 inet static
address 10.10.1.11/24
gateway 10.10.1.254
bridge-ports bond0.10
bridge-stp off
bridge-fd 0
#Management Network
auto vmbr0v20
iface vmbr0v20 inet static
address 10.20.1.11/24
bridge-ports bond0.20
bridge-stp off
bridge-fd 0
#iSCSI - 20
auto vmbr0v30
iface vmbr0v30 inet static
address 10.30.1.11/24
bridge-ports bond0.30
bridge-stp off
bridge-fd 0
#iSCSI - 30
auto vmbr0
iface vmbr0 inet manual
bridge-ports bond0
bridge-stp off
bridge-fd 0
auto vmbr1
iface vmbr1 inet manual
bridge-ports bond0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
source /etc/network/interfaces.d/*
________________________________________
When I run the ifreload -a command the following error appears.
root@pve-01:~# ifreload -a
error: misconfig..? bond0 bridge port is enslaved to multiple interfaces ['bond0.10', 'bond0.20', 'bond0.30', 'vmbr0', 'vmbr1']
root@pve-01:~#
- I don't know if this error is normal.
- I did the test with iometer and the performance was the same as the environment configured with bridge and so I believe something is wrong.
I need to understand the best possible scenario in this case.
Thanks
I need help configuring my environment's network, I'm new to Proxmox and we are setting up a test environment (Proxmox) with 3 hosts that must meet all the requirements of a production environment with redundancy and performance from a network point of view . The goal is to disassemble our vmware cluster.
Important informations:
- We have two 10/25G Switches (Stack)
- Each Host has only one 10GbE card with two ports, each port on a different Switch with LACP configured and on each port we configure 50 different vlans
- iSCSI Storage with two different Network/vLAN
- Proxmox 8.2.4
- open-iscsi (2.1.8-1.pve1)
- multipath-tools (0.9.4-5~bpo11+1)
I understand that the best scenario would be to use bond for network fault-tolerant and increase the performance, mainly due to the performance of the iscsi network, but as I am not an expert I accept other scenarios as long as the performance of the iscsi network is at least equal to the current environment with vmware that has the same settings.
I tested some combinations using bridge and the disk access performance (iscsi) is much lower than the vmware environment, tests with iometer.
So now I'm trying to configure the environment with bond + linux vlan and I'm having problems, maybe because I don't have knowledge/experience.
- Management Network (vLAN1)
- iSCSI-10 Network (vLAN10)
- iSCSI-20 Network (vLAN20)
Just for information:
root@pve-01:~# multipath -l
MP-01 (36000d3100092d40000000000000006b8) dm-7 COMPELNT,Compellent Vol
size=1.0T features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=0 status=active
|- 9:0:0:5 sdf 8:80 active undef running
`- 7:0:0:5 sdb 8:16 active undef running
MP-02 (36000d3100092d40000000000000006b9) dm-6 COMPELNT,Compellent Vol
size=10T features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=0 status=active
|- 10:0:0:6 sde 8:64 active undef running
`- 8:0:0:6 sdc 8:32 active undef running
root@pve-01:~#
Follow the /etc/network/interfaces settings
auto it
iface lo inet loopback
self enp4s0f0
iface enp4s0f0 inet manual
self enp4s0f1
iface enp4s0f1 inet manual
auto bond0
iface bond0 inet manual
bond-slaves enp4s0f0 enp4s0f1
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2+3
iface bond0.10 inet manual
iface bond0.20 inet manual
iface bond0.30 inet manual
auto vmbr0v10
iface vmbr0v10 inet static
address 10.10.1.11/24
gateway 10.10.1.254
bridge-ports bond0.10
bridge-stp off
bridge-fd 0
#Management Network
auto vmbr0v20
iface vmbr0v20 inet static
address 10.20.1.11/24
bridge-ports bond0.20
bridge-stp off
bridge-fd 0
#iSCSI - 20
auto vmbr0v30
iface vmbr0v30 inet static
address 10.30.1.11/24
bridge-ports bond0.30
bridge-stp off
bridge-fd 0
#iSCSI - 30
auto vmbr0
iface vmbr0 inet manual
bridge-ports bond0
bridge-stp off
bridge-fd 0
auto vmbr1
iface vmbr1 inet manual
bridge-ports bond0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
source /etc/network/interfaces.d/*
________________________________________
When I run the ifreload -a command the following error appears.
root@pve-01:~# ifreload -a
error: misconfig..? bond0 bridge port is enslaved to multiple interfaces ['bond0.10', 'bond0.20', 'bond0.30', 'vmbr0', 'vmbr1']
root@pve-01:~#
- I don't know if this error is normal.
- I did the test with iometer and the performance was the same as the environment configured with bridge and so I believe something is wrong.
I need to understand the best possible scenario in this case.
Thanks