In the doc there is just information about separating storage and public Ceph networks. But it doesn't seem to be that easy, eg. here https://forum.proxmox.com/threads/vm-storage-traffic-on-ceph.117137/
VLAN setup:
Seems that the flow is: read OSD (Ceph storage VLAN) -> Monitors -> Proxmox -> VM -> Other VM -> Proxmox -> Monitor -> save to another OSD
What should be the setup to be able to utilize full 100Gb/s bandwidth? Proxmox should be in the same VLAN as Monitors and OSDs? Traffic in VM VLAN is independent and that is probably correct.
VLAN setup:
- Proxmox MGMT
- Ceph storage
- Ceph monitors
- VMs
Seems that the flow is: read OSD (Ceph storage VLAN) -> Monitors -> Proxmox -> VM -> Other VM -> Proxmox -> Monitor -> save to another OSD
What should be the setup to be able to utilize full 100Gb/s bandwidth? Proxmox should be in the same VLAN as Monitors and OSDs? Traffic in VM VLAN is independent and that is probably correct.