[SOLVED] How to design proxmox + ceph network

able

New Member
Feb 21, 2024
11
1
3
Hi, I have read proxmox's documentation on ceph,and then planned to deploy 6 nodes ceph cluter.
But I was confused when designing the network, my plan:
ceph-network.png
My XGE Switch configuration:
switch.png
This is something that puzzles me, the main thing is that I don't understand linux bridge well:
1. My vm as a client does not seem to have access to the ceph public network.
2. Gateway cannot be configured on vmbr1-3. How can ceph Clients of other nodes communicate with each other.
...
Prior to ceph, vmbr0 was used by my management network and business network for communication, the solutions I can think of for these problems have to do with set a route for my nodes, but I don't have a clear mind, Please give me some advice. I really appreciate it.
 
ceph_network.png
In proxmox ceph, proxmox node server(not vm server), acts as the client to communicate with the ceph public network.
In the end, my design looks like this pic.
The router as gateway of the business servers.
The switch VLAN interface as gateway of ceph network.
The ceph public/cluster network use a separate 10 GB NIC and the vm traffic is separate from it.
But vlan3 doesn't seem to need to communicate with vlan4.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!