ceph-osd often slow ops after upgrading to quincy from pacific

[rozaq]

Hi everyone,
Has anyone experienced that after upgrading Ceph from Pacific to Quincy, they often experience slow operations and OSD flapping?

I experienced that after half of the host osd was upgraded, the warning "DAEMON_OLD_VERSION" appeared. And at first there were frequent slow ops and OSD flapping. I checked there were no NIC and disk failures, and it goes away by itself, but still appears frequently.
I have sent a sample log.

 

[alexskysilk]

Has anyone experienced that after upgrading Ceph from Pacific to Quincy, they often experience slow operations and OSD flapping?

Not precisely, but they did start taking a LONG time to come up to operation after reboot. this behavior remained through all quincy point updates and not resolved until the cluster was upgraded to reef.