[SOLVED] After change Ceph cluster network, OSD is listening on both public & cluster network

lazypaul

Member
Aug 20, 2020
47
1
8
35
I've change the /etc/pve/ceph.conf , in the begining the are in the same network 10.0.141.x/24, now I change ceph cluster to 10.98.141.0/24



Code:
[global]
     auth_client_required = cephx
     auth_cluster_required = cephx
     auth_service_required = cephx
     cluster_network = 10.98.141.0/24
     fsid = 2c475bdb-82b2-4614-ad15-becbd73c6c68
     mon_allow_pool_delete = true
     mon_host = 10.0.141.1 10.0.141.4 10.0.141.3 10.0.141.2 10.0.141.5 10.0.141.6 10.0.141.7
     osd_pool_default_min_size = 2
     osd_pool_default_size = 2
     public_network = 10.0.141.0/16

Then I run this command to restart ceph, to make it work.

Code:
systemctl restart ceph.target


Then I found that osd still listening both public & cluster network, is this ok? If they work together, I'm afraid the net flow still in the old public network.

Code:
root@g8kvm01:~# netstat -nap | grep ceph-osd | more
tcp        0      0 10.98.141.1:6821        0.0.0.0:*               LISTEN      3923612/ceph-osd   
tcp        0      0 10.0.141.1:6821         0.0.0.0:*               LISTEN      3923619/ceph-osd   
tcp        0      0 10.98.141.1:6822        0.0.0.0:*               LISTEN      3923613/ceph-osd   
tcp        0      0 10.0.141.1:6822         0.0.0.0:*               LISTEN      3923613/ceph-osd   
tcp        0      0 10.98.141.1:6823        0.0.0.0:*               LISTEN      3923619/ceph-osd   
tcp        0      0 10.0.141.1:6823         0.0.0.0:*               LISTEN      3923612/ceph-osd   
tcp        0      0 10.98.141.1:6824        0.0.0.0:*               LISTEN      3923613/ceph-osd   
tcp        0      0 10.0.141.1:6824         0.0.0.0:*               LISTEN      3923613/ceph-osd   
tcp        0      0 10.98.141.1:6825        0.0.0.0:*               LISTEN      3923619/ceph-osd   
tcp        0      0 10.0.141.1:6825         0.0.0.0:*               LISTEN      3923619/ceph-osd   
tcp        0      0 10.98.141.1:6826        0.0.0.0:*               LISTEN      3923612/ceph-osd   
tcp        0      0 10.0.141.1:6826         0.0.0.0:*               LISTEN      3923613/ceph-osd   
tcp        0      0 10.98.141.1:6827        0.0.0.0:*               LISTEN      3923612/ceph-osd   
tcp        0      0 10.0.141.1:6827         0.0.0.0:*               LISTEN      3923612/ceph-osd   
tcp        0      0 10.0.141.1:6828         0.0.0.0:*               LISTEN      3923619/ceph-osd   
tcp        0      0 10.0.141.1:6829         0.0.0.0:*               LISTEN      3923612/ceph-osd   
tcp        0      0 10.98.141.1:6800        0.0.0.0:*               LISTEN      3923602/ceph-osd   
tcp        0      0 10.98.141.1:6801        0.0.0.0:*               LISTEN      3923602/ceph-osd   
tcp        0      0 10.98.141.1:6802        0.0.0.0:*               LISTEN      3923602/ceph-osd   
tcp        0      0 10.0.141.1:6802         0.0.0.0:*               LISTEN      3923602/ceph-osd   
tcp        0      0 10.98.141.1:6803        0.0.0.0:*               LISTEN      3923602/ceph-osd   
tcp        0      0 10.0.141.1:6803         0.0.0.0:*               LISTEN      3923602/ceph-osd   
tcp        0      0 10.98.141.1:6804        0.0.0.0:*               LISTEN      3923611/ceph-osd   
tcp        0      0 10.0.141.1:6804         0.0.0.0:*               LISTEN      3923602/ceph-osd   
tcp        0      0 10.98.141.1:6805        0.0.0.0:*               LISTEN      3923611/ceph-osd   
tcp        0      0 10.0.141.1:6805         0.0.0.0:*               LISTEN      3923602/ceph-osd   
tcp        0      0 10.98.141.1:6806        0.0.0.0:*               LISTEN      3923611/ceph-osd   
tcp        0      0 10.0.141.1:6806         0.0.0.0:*               LISTEN      3923611/ceph-osd   
tcp        0      0 10.98.141.1:6807        0.0.0.0:*               LISTEN      3923611/ceph-osd   
tcp        0      0 10.0.141.1:6807         0.0.0.0:*               LISTEN      3923611/ceph-osd   
tcp        0      0 10.98.141.1:6808        0.0.0.0:*               LISTEN      3923621/ceph-osd   
tcp        0      0 10.0.141.1:6808         0.0.0.0:*               LISTEN      3923611/ceph-osd   
tcp        0      0 10.98.141.1:6809        0.0.0.0:*               LISTEN      3923621/ceph-osd   
tcp        0      0 10.0.141.1:6809         0.0.0.0:*               LISTEN      3923611/ceph-osd   
tcp        0      0 10.98.141.1:6810        0.0.0.0:*               LISTEN      3923621/ceph-osd
 

wolfgang

Proxmox Staff Member
Staff member
Oct 1, 2014
6,206
424
103
Hi,
The OSD must listen to both networks. Only the replication traffic uses the private network.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!