more then one network on one network card?

[felipe]

hi,

i have just 2 network cards ion all old servers. so i have one network for internal traffic (backup, isos etc) and on for the vms traffic.
now i want also to use the ceph from the new servers. internal traffic should be separated from the ceph monitors i think? (ceph osds have their own network)
so is it possible somethink like i normally do like eth0:1 eth0:2 etc? to have 2 networks for one of the cards. i think in the proxmox gui its not pissible. and the bridges dont support this?

if its not possilble the only way would be to put the external ceph traffic in the same network as the backup network...


thank you

 

[mo_]

The only reason why the Ceph guys recommend putting the cluster communication into a separate, private, non-routed network is for security purposes: so that it's impossible for anybody to disrupt the cluster communication by flooding the network.

That said, its not actually a necessity, ceph can have ceph<->KVM and ceph<->ceph communication in one network just fine. You may want to make sure that network has enough bandwidth then though.

Using ethX:1 to split off the Ceph cluster communication should be possible if you prefer that and your network hardware permits this. The same limitation applies though: make sure you've got enough bandwidth. 1GBit for Ceph AND client I/O might be somewhat limiting in performance. 2x1 or 4x1 GBit for ceph alone should be perfectly fine though (even for Ceph clusters with SSDs for Journals or Caches).

 

[udo]

Using ethX:1 to split off the Ceph cluster communication should be possible if you prefer that and your network hardware permits this...

Hi,
but you wil get problems to use an ethX:X interface for bridges. OK, for ceph there are no bridges nessesary but for pve it's important.

What's about vlan-tagging? This is the clean way to use multible networks on one nic.
But NICs are not to expensive - why you don't use an seperate NIC - than you can use different MTUs (you need 1500 for an bridge and perhaps want 9000 for ceph?).

Udo

 

[mo_]

jumbo frames are useful for ceph, just be aware that your switches need to support them.