[SOLVED] Ceph Object Gateway (RadosGW), multi-part uploads and ETag

[herzkerl]

We'd been having issues with a particular (Synology) Hyper Backup quite some time.
Synology support analyzed the log files and told us that there's no ETag returned after a multi-part upload:

2023-11-21T13:20:55+01:00 NAS img_worker[20149]: (20149) [err] transfer_s3.cpp:355 ETag is empty
2023-11-21T13:20:55+01:00 NAS img_worker[20149]: (20149) [err] transfer_s3.cpp:496 convert response [{"ETag":"","RequestId":null,"success":true}] to finfo failed
2023-11-21T13:20:55+01:00 NAS img_worker[20149]: (20149) file_transfer.cpp:798 failed to sendfile to [NAS_Data.hbk/Config/@Share/homes/83.db.35] from [/volume1/@img_bkp_cache/aws_s3_customername_accessid.jARAYS/NAS_Data.hbk/Config/@Share/homes/83.db], data_hot=[1], error = [8]

I found the following issue in Ceph tracker: reef: S3 CompleteMultipartUploadResult has empty ETag element (target version v18.2.1)
Which according to the linked PR on GitHub should have been merged on Oct 23, 2023.

The current version on our Proxmox hosts is ceph version 18.2.1 (850293cdaae6621945e1191aa8c28ea2918269c3) reef (stable)

Since we're still seeing the same error, I guess it might not have made it into the specific version that Proxmox uses?
Can someone confirm, or at least put me in the right direction? Thanks in advance!

 

[rjaklic]

We do not use Proxmox, but we are experiencing same problem with ceph version 17.2.7

 

[herzkerl]

It's been a while, but after the recent update to ceph version 18.2.4 this issue seems to have been fixed. Finally!

 

[rjaklic]

The fix is there and included in newer versions of ceph, but not for quincy (17.x) for some reason.

Developer said it should be included in next release for quincy also.

https://github.com/ceph/ceph/pull/51446