[SOLVED] After change Ceph cluster network, OSD is listening on both public & cluster network

[lazypaul]

I've change the /etc/pve/ceph.conf , in the begining the are in the same network 10.0.141.x/24, now I change ceph cluster to 10.98.141.0/24

[global]
     auth_client_required = cephx
     auth_cluster_required = cephx
     auth_service_required = cephx
     cluster_network = 10.98.141.0/24
     fsid = 2c475bdb-82b2-4614-ad15-becbd73c6c68
     mon_allow_pool_delete = true
     mon_host = 10.0.141.1 10.0.141.4 10.0.141.3 10.0.141.2 10.0.141.5 10.0.141.6 10.0.141.7
     osd_pool_default_min_size = 2
     osd_pool_default_size = 2
     public_network = 10.0.141.0/16

Then I run this command to restart ceph, to make it work.

systemctl restart ceph.target

Then I found that osd still listening both public & cluster network, is this ok? If they work together, I'm afraid the net flow still in the old public network.

root@g8kvm01:~# netstat -nap | grep ceph-osd | more
tcp        0      0 10.98.141.1:6821        0.0.0.0:*               LISTEN      3923612/ceph-osd   
tcp        0      0 10.0.141.1:6821         0.0.0.0:*               LISTEN      3923619/ceph-osd   
tcp        0      0 10.98.141.1:6822        0.0.0.0:*               LISTEN      3923613/ceph-osd   
tcp        0      0 10.0.141.1:6822         0.0.0.0:*               LISTEN      3923613/ceph-osd   
tcp        0      0 10.98.141.1:6823        0.0.0.0:*               LISTEN      3923619/ceph-osd   
tcp        0      0 10.0.141.1:6823         0.0.0.0:*               LISTEN      3923612/ceph-osd   
tcp        0      0 10.98.141.1:6824        0.0.0.0:*               LISTEN      3923613/ceph-osd   
tcp        0      0 10.0.141.1:6824         0.0.0.0:*               LISTEN      3923613/ceph-osd   
tcp        0      0 10.98.141.1:6825        0.0.0.0:*               LISTEN      3923619/ceph-osd   
tcp        0      0 10.0.141.1:6825         0.0.0.0:*               LISTEN      3923619/ceph-osd   
tcp        0      0 10.98.141.1:6826        0.0.0.0:*               LISTEN      3923612/ceph-osd   
tcp        0      0 10.0.141.1:6826         0.0.0.0:*               LISTEN      3923613/ceph-osd   
tcp        0      0 10.98.141.1:6827        0.0.0.0:*               LISTEN      3923612/ceph-osd   
tcp        0      0 10.0.141.1:6827         0.0.0.0:*               LISTEN      3923612/ceph-osd   
tcp        0      0 10.0.141.1:6828         0.0.0.0:*               LISTEN      3923619/ceph-osd   
tcp        0      0 10.0.141.1:6829         0.0.0.0:*               LISTEN      3923612/ceph-osd   
tcp        0      0 10.98.141.1:6800        0.0.0.0:*               LISTEN      3923602/ceph-osd   
tcp        0      0 10.98.141.1:6801        0.0.0.0:*               LISTEN      3923602/ceph-osd   
tcp        0      0 10.98.141.1:6802        0.0.0.0:*               LISTEN      3923602/ceph-osd   
tcp        0      0 10.0.141.1:6802         0.0.0.0:*               LISTEN      3923602/ceph-osd   
tcp        0      0 10.98.141.1:6803        0.0.0.0:*               LISTEN      3923602/ceph-osd   
tcp        0      0 10.0.141.1:6803         0.0.0.0:*               LISTEN      3923602/ceph-osd   
tcp        0      0 10.98.141.1:6804        0.0.0.0:*               LISTEN      3923611/ceph-osd   
tcp        0      0 10.0.141.1:6804         0.0.0.0:*               LISTEN      3923602/ceph-osd   
tcp        0      0 10.98.141.1:6805        0.0.0.0:*               LISTEN      3923611/ceph-osd   
tcp        0      0 10.0.141.1:6805         0.0.0.0:*               LISTEN      3923602/ceph-osd   
tcp        0      0 10.98.141.1:6806        0.0.0.0:*               LISTEN      3923611/ceph-osd   
tcp        0      0 10.0.141.1:6806         0.0.0.0:*               LISTEN      3923611/ceph-osd   
tcp        0      0 10.98.141.1:6807        0.0.0.0:*               LISTEN      3923611/ceph-osd   
tcp        0      0 10.0.141.1:6807         0.0.0.0:*               LISTEN      3923611/ceph-osd   
tcp        0      0 10.98.141.1:6808        0.0.0.0:*               LISTEN      3923621/ceph-osd   
tcp        0      0 10.0.141.1:6808         0.0.0.0:*               LISTEN      3923611/ceph-osd   
tcp        0      0 10.98.141.1:6809        0.0.0.0:*               LISTEN      3923621/ceph-osd   
tcp        0      0 10.0.141.1:6809         0.0.0.0:*               LISTEN      3923611/ceph-osd   
tcp        0      0 10.98.141.1:6810        0.0.0.0:*               LISTEN      3923621/ceph-osd

 

[wolfgang]

Hi,
The OSD must listen to both networks. Only the replication traffic uses the private network.

 

[lazypaul]

Hi,
The OSD must listen to both networks. Only the replication traffic uses the private network.

Thank you ! Resolved.