Backup and Garbage Collection Failed

[kev1904]

Hello,

Backups of some Server failes with I/O error, other vms backups fine. Garbe Collection fails as well

Spoiler: Backup

()
Task viewer: Datastore Backup Backup vm/243
OutputStatus
Stop
2021-02-18T19:21:48+01:00: starting new backup on datastore 'Backup': "vm/243/2021-02-18T18:21:48Z"
2021-02-18T19:21:48+01:00: GET /previous: 400 Bad Request: no valid previous backup
2021-02-18T19:21:48+01:00: created new fixed index 1 ("vm/243/2021-02-18T18:21:48Z/drive-virtio0.img.fidx")
2021-02-18T19:21:52+01:00: add blob "/mnt/Datastore1/vm/243/2021-02-18T18:21:48Z/qemu-server.conf.blob" (330 bytes, comp: 330)
2021-02-18T19:31:53+01:00: POST /fixed_chunk: 400 Bad Request: Input/output error (os error 5)
2021-02-18T19:31:53+01:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection: protocol error: stream no longer needed
2021-02-18T19:31:53+01:00: backup ended and finish failed: backup ended but finished flag is not set.
2021-02-18T19:31:53+01:00: removing unfinished backup
2021-02-18T19:31:53+01:00: TASK ERROR: removing backup snapshot "/mnt/Datastore1/vm/243/2021-02-18T18:21:48Z" failed - Directory not empty (os error 39)

Spoiler: Garbage Collect

()


Task viewer: Datastore Backup - Garbage collect

OutputStatus

Stop
2021-02-19T08:35:57+01:00: starting garbage collection on store Backup
2021-02-19T08:35:57+01:00: Start GC phase1 (mark used chunks)
2021-02-19T08:36:24+01:00: marked 1% (3 of 226 index files)
2021-02-19T08:36:28+01:00: marked 2% (5 of 226 index files)
2021-02-19T08:36:31+01:00: marked 3% (7 of 226 index files)
2021-02-19T08:36:38+01:00: marked 4% (10 of 226 index files)
2021-02-19T08:36:41+01:00: marked 5% (12 of 226 index files)
2021-02-19T08:36:44+01:00: marked 6% (14 of 226 index files)
2021-02-19T08:37:30+01:00: marked 7% (16 of 226 index files)
2021-02-19T08:37:41+01:00: marked 8% (19 of 226 index files)
2021-02-19T08:37:47+01:00: marked 9% (21 of 226 index files)
2021-02-19T08:37:56+01:00: marked 10% (23 of 226 index files)
2021-02-19T08:38:03+01:00: marked 11% (25 of 226 index files)
2021-02-19T08:38:17+01:00: marked 12% (28 of 226 index files)
2021-02-19T08:38:28+01:00: TASK ERROR: update atime failed for chunk/file "/mnt/Datastore1/.chunks/89de/89deb011df600a127c7906bf3204c43e4af90fb5c2d02ec39da3f23e0e83b38c" - EIO: I/O error

 

[fabian]

sounds like your storage has problems?

 

[kev1904]

Storage is an mounted NFS. I alrdy checked the Premissions. If i create a second Datastore on the same NFS Mount, the Backups from failed vms works.

 

[fabian]

well, you get an I/O error when uploading a chunk, and when updating a chunks atime (the logs don't indicate whether they are the same chunk).. maybe check on the NFS server side as well whether you see anything in the logs?