For my home network I´ve had a single node running for several years. I´m now preparing to set up two additional nodes in a cluster. Mainly for ease of management as well as some flexibility on migrating guests between nodes etc. No HA planned as of yet.
My older server has IPMI but for the new ones I´d like to use Intel AMT to manage them. I´ve set a static IP for AMT.
I have these VLANs:
3 for Management, i.e. Intel AMT and Proxmox management interface
4 for corosync traffic
10 for VMs/guests
I am planning to use the built-in copper NIC for corosync and SFP+ NIC for VM traffic, migration etc. as well as corosync traffic redundancy.
Built-in NIC I plan to connect to separate/dedicated switch. Since AMT cannot easily be configured to a specific VLAN the switch port is set native to vlan 3 and allow tagged traffic for the other VLANs.
I then set a random mac address for the interface in Proxmox, and did the same for the other interfaces as they could be distinguished on network side (Unifi). This is what /etc/network/interfaces look like for one of the nodes:
Issue is that there is some instability. Some of the IPs become unreachable, Intel AMT dropped off the network etc. This has happened for both the nodes. I´ve experimented a bit with vlan_mgmt on both NICs, on which interface the gateway is defined etc. Latest is as above, with just one bridge both for guests and for management interface.
Is there any issues to define multiple interfaces on same VLAN, or any other things I should change with this setup?
I´ve not decided yet if the "corosync" switch will be connected to the rest of the network or not, but now during setup/testing it is connected.
Appreciate any help, thanks!
My older server has IPMI but for the new ones I´d like to use Intel AMT to manage them. I´ve set a static IP for AMT.
I have these VLANs:
3 for Management, i.e. Intel AMT and Proxmox management interface
4 for corosync traffic
10 for VMs/guests
I am planning to use the built-in copper NIC for corosync and SFP+ NIC for VM traffic, migration etc. as well as corosync traffic redundancy.
Built-in NIC I plan to connect to separate/dedicated switch. Since AMT cannot easily be configured to a specific VLAN the switch port is set native to vlan 3 and allow tagged traffic for the other VLANs.
I then set a random mac address for the interface in Proxmox, and did the same for the other interfaces as they could be distinguished on network side (Unifi). This is what /etc/network/interfaces look like for one of the nodes:
Code:
auto lo
iface lo inet loopback
iface enp0s31f6 inet manual
iface enp3s0f1np1 inet manual
iface enp3s0f0np0 inet manual
auto vmbr0
iface vmbr0 inet static
address 10.10.3.12/27
gateway 10.10.3.1
bridge-ports enp3s0f1np1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 3 10 11
hwaddress ether 00:49:5c:2b:92:25
#VM Interface
#auto vlan_mgmt
#iface vlan_mgmt inet static
# address 10.10.3.12/27
# gateway 10.10.3.1
# vlan-id 3
# vlan-raw-device enp3s0f1np1
# hwaddress ether 00:49:5c:2b:92:22
#For Proxmox management interface
auto vlan_corosync
iface vlan_corosync inet static
address 10.10.4.12/27
vlan-id 4
vlan-raw-device enp0s31f6
hwaddress ether 00:49:5c:2b:92:23
#Corosync traffic
auto vlan_corosync2
iface vlan_corosync2 inet static
address 10.10.4.22/27
vlan-id 4
vlan-raw-device enp3s0f1np1
hwaddress ether 00:49:5c:2b:92:24
#Corosync traffic 2
source /etc/network/interfaces.d/*Issue is that there is some instability. Some of the IPs become unreachable, Intel AMT dropped off the network etc. This has happened for both the nodes. I´ve experimented a bit with vlan_mgmt on both NICs, on which interface the gateway is defined etc. Latest is as above, with just one bridge both for guests and for management interface.
Is there any issues to define multiple interfaces on same VLAN, or any other things I should change with this setup?
I´ve not decided yet if the "corosync" switch will be connected to the rest of the network or not, but now during setup/testing it is connected.
Appreciate any help, thanks!