Hi all,
I have to admit, this is a weird question. I need to setup a new Proxmox cluster that, using ZFS, will be replicating VMs and CTs disks through different nodes.
I know that quite a lot of bandwith might be required so I was checking out Mellanox with a 10Gbps SFP+. The problem is that purchasing a switch with 20 SFP+ ports becomes really expensive. Therefore, I was tinkering of cheaper solutions.
I realized i could simply interconnect the servers, since every one has two SFP+ ports, and then use one as uplink. Very straightforward:
switch (SFP+) -> (SFP+ port 1) node 1 (SFP+ port 2) -> (SFP+ port 1) node 2 (SFP+ port 2) -> (SFP+ port 1) node 3 etc...
Correct me if I'm wrong but bridging with a virtual bridge the SFP+ ports on the servers should do the trick here (Q: does the traffic going e.g. from the switch to node 2 need to pass from the CPU of node 1?).
Anyway, this idea has the problem that I'm deploying this solution to enhance fault rendurance and here, if one of the nodes, or worse, the first node with the uplink, fails, all other nodes in cascade would go offline. I need an idea to get this solution fault rendurant. I was thinking of "closing the chain" so connect the last node back to the switch (or better, to another switch... also switches can fail) but I can't figure out how should this be configured via software: If i set it as a virtual bridge, traffic will collide while going to the switch from both the first and the last node whereas configuring it as a bond would not allow other nodes to benefit of the connection he's uplinking from the switch in case the other one fails.
Is this even technically possible? And is it possible in Proxmox? Should I configure the switch(es) in a specific manner? (e.g. link aggregation?)
Thanks in advance,
Best,
Marco
I have to admit, this is a weird question. I need to setup a new Proxmox cluster that, using ZFS, will be replicating VMs and CTs disks through different nodes.
I know that quite a lot of bandwith might be required so I was checking out Mellanox with a 10Gbps SFP+. The problem is that purchasing a switch with 20 SFP+ ports becomes really expensive. Therefore, I was tinkering of cheaper solutions.
I realized i could simply interconnect the servers, since every one has two SFP+ ports, and then use one as uplink. Very straightforward:
switch (SFP+) -> (SFP+ port 1) node 1 (SFP+ port 2) -> (SFP+ port 1) node 2 (SFP+ port 2) -> (SFP+ port 1) node 3 etc...
Correct me if I'm wrong but bridging with a virtual bridge the SFP+ ports on the servers should do the trick here (Q: does the traffic going e.g. from the switch to node 2 need to pass from the CPU of node 1?).
Anyway, this idea has the problem that I'm deploying this solution to enhance fault rendurance and here, if one of the nodes, or worse, the first node with the uplink, fails, all other nodes in cascade would go offline. I need an idea to get this solution fault rendurant. I was thinking of "closing the chain" so connect the last node back to the switch (or better, to another switch... also switches can fail) but I can't figure out how should this be configured via software: If i set it as a virtual bridge, traffic will collide while going to the switch from both the first and the last node whereas configuring it as a bond would not allow other nodes to benefit of the connection he's uplinking from the switch in case the other one fails.
Is this even technically possible? And is it possible in Proxmox? Should I configure the switch(es) in a specific manner? (e.g. link aggregation?)
Thanks in advance,
Best,
Marco