Network Config

[VC-Mike]

Hello,

I have some questions regarding the networking setup. Our current hardware configuration is as follows;

Management Server - 4/6 Ethernet Ports (Will be used as backup server as well (Separate RAID Array)
4x Hypervisor Servers - 4 Ethernet Ports

With our current Cloud Platform (OnApp) all 4 ports are being used for their different internal networkings (Waste of NIC's if you ask me...) but since we're moving to Proxmox this now provides us with additional ports which we're not entirely sure how to use, whether it be that we bond them all as the storage network or if we bond 2 as the "Internet" interfaces for VM's to gain internet access and such and bond the other 2 for storage etc.

What do you suggest? What would you do?

Thanks so much in advance!

 

[danielmash]

What do you suggest? What would you do?

Pair them: first bond for cluster and second bond for storage.

 

[VC-Mike]

Are there any network diagrams available that I can review? Any specific network instructions, as you say pair 2 for the cluster and 2 for storage?

With cluster, do you mean the internal networking or the VM internet access?

 

[danielmash]

Are there any network diagrams available that I can review? Any specific network instructions, as you say pair 2 for the cluster and 2 for storage?

With cluster, do you mean the internal networking or the VM internet access?

Might be not a perfect design but here is my plan:

[ switch0 (cluster talks) ]
vmbr0 | | | | | | | | (bonded links)
[ mox1 ] [ mox2 ] [ mox3 ] [ mox4 ]
vmbr1 | | | | | | | | (bonded links. 803.1ad if switch supports LACP)
[ switch1 (storage) ]

vmbr0 will share cluster talks, migration, vm internet access.
vmbr1 will share storage replication, backups and vm internal networks

You can extend that and separate cluster talks from the internet for more security.
Also might be two switches for each netowk for redundancy and failover.

 

[VC-Mike]

I'm a bit stuck here. We have 6 Hypervisors and 2 SANs. Each HV has 4 ports and the SANs have around 6 ports. I'm trying to work out how I can get all of these to bond so we have the cluster bond and the storage bond. I've created 2 VLANs on our switch (Dell PC 6248), one for the cluster and one for the storage. I think our switch only supports 4 ports in a single LACP channel.

Can anyone provide some suggestions on how we could proceed?

Thanks in advance!

 

[danielmash]

2 SANs.

What is the model number?

I'm trying to work out how I can get all of these to bond so we have the cluster bond and the storage bond.

Dedicated switch/switches for storage area network might be better. What if your 6248 die?

I've created 2 VLANs on our switch (Dell PC 6248), one for the cluster and one for the storage.

I am not sure if VLAN somehow related to LACP. You might need to make sure that switch configuration is right before proceed any further.
CLI will show you how switch aggregated channels.

[I]console#show lacp [/I]

Honestly I would prefer to have separate devices rather then having VLAN challenges and single point of failure.

I think our switch only supports 4 ports in a single LACP channel.

We only need two ports per hypervisor and two ports per SAN (I presume the third is for management only)

The idea is to walk from simple to advanced. First setup switch with LACP and without VLANS and test cluster communications. Generate some load and check interfaces. (I use dstat -N eth0,eth1,... ) Then backup switch configuration and add VLANS and another LACP for storage. Document what you are doing.