I fully understand that this is to massive but maybe someone have any information for any of the 5-6 questions I have and I would be very grateful for any knowledge you have to share!
I'm working on doing my first 3 node cluster. "Proxmox Cluster", "3x Proxmox Backup Server", "3x k3s worker and Longhorn" will use a back-end network without switches.
The k3s/Longhorn is using Ubuntu cloud-init and I have learned a lot so far already! But I need some help with "Proxmox VE" networking and a few more things. I had to disable cloud-init network config because it forces settings on "eth1" that won't work and change nameserver to hosts settings (a diffrent VLAN).
I have 3 Proxmox Nodes: "Neo", "Morpheus", "Trinity".
"eth1" (MTU 9000) on each VM is connected to "vmbr1" (MTU 9000), Ports for "vmbr1" is "enxg0" (SFP+ #1) AND "enxg1" (SFP+ #2).
They are just directly connected to the other two nodes. Orange cables in the next picture.
VM's are Pink color in the next picture.
1. Is Open vSwitch needed because of RSTP Loop instead of STP? Proxmox wiki "Routed Setup (Simple)"
2. I also want to be able to backup "Proxmox VE" with "Proxmox Backup Server VM's" over "vmbr1" because of 10g and Jumbo frames. Would that require that I add a "vmbr2" and only have "enxg0" on "vmbr1" and "enxg1" on "vmbr2" (see two pictures down) and would it require that I set IP's for the "Linux Bridge's" for the host to reach "Proxmox Backup Server's? Is a "/30 Link-network" required then?
or
3. My Proxmox hosts "Cluster" is using my LAN on "VLAN 101" and I would like to use the newer "10g back-end, jumbo frame, ring/mesh network". Can the nodes url's just be changed? Or dismantle and rebuild? (no CEPH or so just for a overview of nodes)
4. The k3s VM's and PBS VM's cannot reach each other. Would I need to have the two "Linux Bridge's" with only one SFP+ (enxg0, enxg1) using a "/30 Link-network" for each node for this to work? But that can't work if i understand this correctly because the VM's will be on a diffrent subnet. I think I need static routing. What am I missing? Would it be better to do like Proxmox wiki "Routed Setup (Simple)" and use "Open vSwitch" and create "bond0" and more.. I do not understand all of this yet. Goal is to use this back-end network for "Proxmox Cluster", "3x Proxmox Backup Server", "3x k3s worker and Longhorn".
I hope i covered everything.
I'm working on doing my first 3 node cluster. "Proxmox Cluster", "3x Proxmox Backup Server", "3x k3s worker and Longhorn" will use a back-end network without switches.
The k3s/Longhorn is using Ubuntu cloud-init and I have learned a lot so far already! But I need some help with "Proxmox VE" networking and a few more things. I had to disable cloud-init network config because it forces settings on "eth1" that won't work and change nameserver to hosts settings (a diffrent VLAN).
I have 3 Proxmox Nodes: "Neo", "Morpheus", "Trinity".
"eth1" (MTU 9000) on each VM is connected to "vmbr1" (MTU 9000), Ports for "vmbr1" is "enxg0" (SFP+ #1) AND "enxg1" (SFP+ #2).
They are just directly connected to the other two nodes. Orange cables in the next picture.
VM's are Pink color in the next picture.
1. Is Open vSwitch needed because of RSTP Loop instead of STP? Proxmox wiki "Routed Setup (Simple)"
2. I also want to be able to backup "Proxmox VE" with "Proxmox Backup Server VM's" over "vmbr1" because of 10g and Jumbo frames. Would that require that I add a "vmbr2" and only have "enxg0" on "vmbr1" and "enxg1" on "vmbr2" (see two pictures down) and would it require that I set IP's for the "Linux Bridge's" for the host to reach "Proxmox Backup Server's? Is a "/30 Link-network" required then?
or
3. My Proxmox hosts "Cluster" is using my LAN on "VLAN 101" and I would like to use the newer "10g back-end, jumbo frame, ring/mesh network". Can the nodes url's just be changed? Or dismantle and rebuild? (no CEPH or so just for a overview of nodes)
4. The k3s VM's and PBS VM's cannot reach each other. Would I need to have the two "Linux Bridge's" with only one SFP+ (enxg0, enxg1) using a "/30 Link-network" for each node for this to work? But that can't work if i understand this correctly because the VM's will be on a diffrent subnet. I think I need static routing. What am I missing? Would it be better to do like Proxmox wiki "Routed Setup (Simple)" and use "Open vSwitch" and create "bond0" and more.. I do not understand all of this yet. Goal is to use this back-end network for "Proxmox Cluster", "3x Proxmox Backup Server", "3x k3s worker and Longhorn".
I hope i covered everything.
Last edited:
