Hello!
I'm new to Proxmox and I'm testing environment settings for possible adoption of the tool in production. I have some servers with 4 network cards (1Gbps) and I tried to configure all of them in Bond LACP layer 2 + 3. Objective is to have fault tolerance and better performance in all functions of each node. I plan to configure Ceph for storage.
I have two switches HP that support LACP but cannot be stacked. So, I decided to connect three ports of each server on one of the switches and only one port on the other switch, with Spannig tree and interconnecting the switches also with a LACP trunk.
Enable jumbo frames.
I got the result I would like, because it worked all right, I had an increase in performance with the three ports connected and I also had fault tolerance in case one of the switches should fail.
Apparently overcoming this physical topology, I set out to configure VLANS.
Enable jumbo frames on all switch VLANS for better performance.
The problem started when I tried to configure VLAN's to separate the networks into three VLAN's for better management and security.
So, I created a VLAN 8 (Tagged) for use by Ceph and another VLAN 9 (Tagged) for different uses of the Cluster. I still maintained the default VLAN (unttaged) for managing Proxmox and also for use with VMs. I have future plans to use other VLANs (Taggeds) to use VMs.
I configured IPv4 addresses of different subnets in each VLAN.
When I made these configurations, only 1 server worked as it should, and the others work on the standard VLAN (unttaged) but cannot ping addresses on VLANS 8 and 9 (Tagged).
On 1 of the servers, I can ping addresses on multiple devices from all subnets with Bond and everything goes well.
In the others, Bond only works on default VLAN. Or with tagged VLAN directly on the physical interfaces, without Bond. With Bond, i can't even ping the VLAN interfaces of any of the switches.
The settings maintained on the servers are apparently the same, obviously only changing the addresses of the nodes.
Please, would anyone know what could be happening?
I'm new to Proxmox and I'm testing environment settings for possible adoption of the tool in production. I have some servers with 4 network cards (1Gbps) and I tried to configure all of them in Bond LACP layer 2 + 3. Objective is to have fault tolerance and better performance in all functions of each node. I plan to configure Ceph for storage.
I have two switches HP that support LACP but cannot be stacked. So, I decided to connect three ports of each server on one of the switches and only one port on the other switch, with Spannig tree and interconnecting the switches also with a LACP trunk.
Enable jumbo frames.
I got the result I would like, because it worked all right, I had an increase in performance with the three ports connected and I also had fault tolerance in case one of the switches should fail.
Apparently overcoming this physical topology, I set out to configure VLANS.
Enable jumbo frames on all switch VLANS for better performance.
The problem started when I tried to configure VLAN's to separate the networks into three VLAN's for better management and security.
So, I created a VLAN 8 (Tagged) for use by Ceph and another VLAN 9 (Tagged) for different uses of the Cluster. I still maintained the default VLAN (unttaged) for managing Proxmox and also for use with VMs. I have future plans to use other VLANs (Taggeds) to use VMs.
I configured IPv4 addresses of different subnets in each VLAN.
When I made these configurations, only 1 server worked as it should, and the others work on the standard VLAN (unttaged) but cannot ping addresses on VLANS 8 and 9 (Tagged).
On 1 of the servers, I can ping addresses on multiple devices from all subnets with Bond and everything goes well.
In the others, Bond only works on default VLAN. Or with tagged VLAN directly on the physical interfaces, without Bond. With Bond, i can't even ping the VLAN interfaces of any of the switches.
The settings maintained on the servers are apparently the same, obviously only changing the addresses of the nodes.
Please, would anyone know what could be happening?
Code:
auto lo
iface lo inet loopback
auto enp3s0f0
iface enp3s0f0 inet manual
mtu 9000
#Default Physical Interface 1
auto enp3s0f1
iface enp3s0f1 inet manual
mtu 9000
#Default Physical Interface 2
auto enp4s0f0
iface enp4s0f0 inet manual
mtu 9000
#Default Physical Interface 3
auto enp4s0f1
iface enp4s0f1 inet manual
#Default Physical Interface 4
auto bond0
iface bond0 inet manual
bond-slaves enp3s0f0 enp3s0f1 enp4s0f0 enp4s0f1
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2+3
mtu 9000
auto vmbr0
iface vmbr0 inet static
address 192.168.0.37/24
gateway 192.168.0.1
bridge-ports bond0
bridge-stp off
bridge-fd 0
mtu 9000
auto vlan8
iface vlan8 inet static
address 10.0.8.12/24
mtu 9000
vlan-raw-device bond0
vlan-id 8
network 10.0.8.0
#ceph
auto vlan9
iface vlan9 inet static
address 10.0.9.12/24
mtu 9000
vlan-raw-device bond0
vlan-id 9
network 10.0.9.0
#Cluster