Hi,
From time to time my entire system stops responding and the only way to get it back is to do a hard reset.
Sometimes it goes a while before it happens again just as now. But this time I was accessing servers on my Proxmox when it happens so I knew the time and everything.
So I fetched the logs from that time (see below). I'm a novice guy and don't know anything regarding this. Anyone that can help me understand what is happening.
As I see it it's regarding the HDDs. But it is actually the HHDs or the system itself? What can I do to fix this? Any suggestions?
Sep 20 22:01:43 proxmox kernel: ahci 0000:16:00.1: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xfebb1000 flags=0x0000]
Sep 20 22:01:44 proxmox kernel: ata2.00: exception Emask 0x10 SAct 0x60008000 SErr 0x40000 action 0x6 frozen
Sep 20 22:01:44 proxmox kernel: ata2.00: irq_stat 0x08000000, interface fatal error
Sep 20 22:01:44 proxmox kernel: ata2: SError: { CommWake }
Sep 20 22:01:44 proxmox kernel: ata2.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:01:44 proxmox kernel: ata2.00: cmd 61/28:78:10:de:98/00:00:0c:00:00/40 tag 15 ncq dma 20480 out res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x10 (ATA bus error)
Sep 20 22:01:44 proxmox kernel: ata2.00: status: { DRDY }
Sep 20 22:01:44 proxmox kernel: ata2.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:01:44 proxmox kernel: ata2.00: cmd 61/60:e8:00:e4:49/00:00:0c:00:00/40 tag 29 ncq dma 49152 out res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x10 (ATA bus error)
Sep 20 22:01:44 proxmox kernel: ata2.00: status: { DRDY }
Sep 20 22:01:44 proxmox kernel: ata2.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:01:44 proxmox kernel: ata2.00: cmd 61/08:f0:b8:ce:45/00:00:0c:00:00/40 tag 30 ncq dma 4096 out res 40/00:01:06:4f:c2/00:00:00:00:00/00 Emask 0x10 (ATA bus error)
Sep 20 22:01:44 proxmox kernel: ata2.00: status: { DRDY }
Sep 20 22:01:44 proxmox kernel: ata2: hard resetting link
Sep 20 22:01:44 proxmox kernel: ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Sep 20 22:01:44 proxmox kernel: ata2.00: supports DRM functions and may not be fully accessible
Sep 20 22:01:59 proxmox kernel: ata2.00: qc timeout after 15000 msecs (cmd 0x2f)
Sep 20 22:03:36 proxmox kernel: ata2.00: Read log 0x00 page 0x00 failed, Emask 0x4
Sep 20 22:03:36 proxmox kernel: ata2.00: NCQ Send/Recv Log not supported
Sep 20 22:03:36 proxmox kernel: ata2.00: Read log 0x00 page 0x00 failed, Emask 0x40
Sep 20 22:03:36 proxmox kernel: ata2.00: ATA Identify Device Log not supported
Sep 20 22:03:36 proxmox kernel: ata2.00: Security Log not supported
Sep 20 22:03:36 proxmox kernel: ata2.00: Read log 0x00 page 0x00 failed, Emask 0x40
Sep 20 22:03:36 proxmox kernel: ata2.00: Read log 0x00 page 0x00 failed, Emask 0x40
Sep 20 22:03:36 proxmox kernel: ata2.00: failed to set xfermode (err_mask=0x40)
Sep 20 22:03:36 proxmox kernel: ata2: hard resetting link
Sep 20 22:03:36 proxmox kernel: ata2: softreset failed (1st FIS failed)
Sep 20 22:03:36 proxmox kernel: ata2: hard resetting link
Sep 20 22:03:36 proxmox kernel: ata2: softreset failed (1st FIS failed)
Sep 20 22:03:36 proxmox kernel: ata2: hard resetting link
Sep 20 22:03:36 proxmox kernel: ata1.00: exception Emask 0x0 SAct 0x4001fed8 SErr 0x50000 action 0x6 frozen
Sep 20 22:03:36 proxmox kernel: ata1: SError: { PHYRdyChg CommWake }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:18:c0:75:ed/00:00:01:00:00/40 tag 3 ncq dma 4096 out res 40/00:01:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:20:00:e6:87/00:00:02:00:00/40 tag 4 ncq dma 4096 out res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/48:30:88:90:2e/00:00:07:00:00/40 tag 6 ncq dma 36864 out res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:38:00:e7:87/00:00:02:00:00/40 tag 7 ncq dma 4096 out res 40/00:01:06:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:48:00:17:e0/00:00:07:00:00/40 tag 9 ncq dma 4096 out res 40/00:01:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:50:a8:50:cd/00:00:08:00:00/40 tag 10 ncq dma 4096 out res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:58:38:ea:87/00:00:02:00:00/40 tag 11 ncq dma 4096 out res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:60:a8:ec:87/00:00:02:00:00/40 tag 12 ncq dma 4096 out res 40/00:01:06:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/20:68:90:e4:87/00:00:02:00:00/40 tag 13 ncq dma 16384 out res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/10:70:58:da:86/00:00:02:00:00/40 tag 14 ncq dma 8192 out res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:78:d8:da:86/00:00:02:00:00/40 tag 15 ncq dma 4096 out res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:80:38:dd:86/00:00:02:00:00/40 tag 16 ncq dma 4096 out res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:f0:b8:18:02/00:00:04:00:00/40 tag 30 ncq dma 4096 out res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1: hard resetting link
Sep 20 22:03:36 proxmox kernel: ata1: softreset failed (1st FIS failed)
Sep 20 22:03:36 proxmox kernel: ata1: hard resetting link
Sep 20 22:03:36 proxmox kernel: ata2: softreset failed (1st FIS failed)
Sep 20 22:03:36 proxmox kernel: ata2: limiting SATA link speed to 3.0 Gbps
Sep 20 22:03:36 proxmox kernel: ata2: hard resetting link
Sep 20 22:03:36 proxmox kernel: ata1: softreset failed (1st FIS failed)
Sep 20 22:03:36 proxmox kernel: ata1: hard resetting link
Sep 20 22:03:36 proxmox kernel: ata2: softreset failed (1st FIS failed)
Sep 20 22:03:36 proxmox kernel: ata2: softreset failed
Sep 20 22:03:36 proxmox kernel: ata2: reset failed, giving up
Sep 20 22:03:36 proxmox kernel: ata2.00: disable device
Sep 20 22:03:36 proxmox kernel: ahci 0000:16:00.1: port does not support device sleep
Sep 20 22:03:36 proxmox kernel: ata2: EH complete
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#1 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#1 CDB: Synchronize Cache(10) 35 00 00 00 00 00 00 00 00 00
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 137312384 op 0x1
WRITE) flags 0x800 phys_seg 0 prio class 0
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#2 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#2 CDB: Write(10) 2a 00 0b c1 eb c8 00 00 08 00
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 197258184 op 0x1
WRITE) flags 0x800 phys_seg 1 prio class 0
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on dev dm-15, logical block 153929, lost async page write
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#3 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#3 CDB: Write(10) 2a 00 0c 44 fe b8 00 00 08 00
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 205848248 op 0x1
WRITE) flags 0x3800 phys_seg 1 prio class 0
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on dev dm-6, logical block 9255, lost sync page write
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#9 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#9 CDB: Write(10) 2a 00 0c 45 c1 68 00 00 18 00
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 205898088 op 0x1
WRITE) flags 0x0 phys_seg 1 prio class 0
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#4 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=75s
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-6): kmmpd:185: comm kmmpd-dm-6: Error writing to MMP block
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#4 CDB: Write(10) 2a 00 0c 98 de 10 00 00 28 00
Sep 20 22:03:36 proxmox kernel: EXT4-fs warning (device dm-8): ext4_end_bio:342: I/O error 10 writing to inode 1763 starting block 82093)
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 211344912 op 0x1
WRITE) flags 0x9800 phys_seg 5 prio class 2
Sep 20 22:03:36 proxmox kernel: Aborting journal on device dm-6-8.
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on device dm-8, logical block 82093
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on device dm-8, logical block 82094
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on device dm-8, logical block 82095
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#10 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#10 CDB: Write(10) 2a 00 0c 45 c1 f0 00 00 08 00
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#5 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=76s
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 205898224 op 0x1
WRITE) flags 0x0 phys_seg 1 prio class 0
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#5 CDB: Write(10) 2a 00 0c 49 e4 00 00 00 60 00
Sep 20 22:03:36 proxmox kernel: Aborting journal on device dm-22-8.
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 206169088 op 0x1
WRITE) flags 0x9800 phys_seg 12 prio class 2
Sep 20 22:03:36 proxmox kernel: EXT4-fs warning (device dm-8): ext4_end_bio:342: I/O error 10 writing to inode 1763 starting block 82126)
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on device dm-8, logical block 82126
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#11 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-22): ext4_journal_check_start:84: comm node: Detected aborted journal
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#11 CDB: Write(10) 2a 00 0c 45 c4 78 00 00 08 00
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 205898872 op 0x1
WRITE) flags 0x0 phys_seg 1 prio class 0
Sep 20 22:03:36 proxmox kernel: EXT4-fs warning (device dm-8): ext4_end_bio:342: I/O error 10 writing to inode 1763 starting block 82415)
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on device dm-8, logical block 82415
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#6 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=75s
Sep 20 22:03:36 proxmox kernel: EXT4-fs warning (device dm-6): ext4_convert_unwritten_extents:4879: inode #4814: block 2541: len 2: ext4_ext_map_blocks returned -30
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#6 CDB: Write(10) 2a 00 0c 45 ce b8 00 00 08 00
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#12 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 205901496 op 0x1
WRITE) flags 0x3800 phys_seg 1 prio class 0
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#12 CDB: Write(10) 2a 00 0c 45 c4 b0 00 00 08 00
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-6) in ext4_reserve_inode_write:5880: Journal has aborted
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on dev dm-20, logical block 9255, lost sync page write
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 205898928 op 0x1
WRITE) flags 0x0 phys_seg 1 prio class 0
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on dev dm-9, logical block 9255, lost sync page write
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-20): kmmpd:185: comm kmmpd-dm-20: Error writing to MMP block
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-6): ext4_convert_unwritten_extents:4884: inode #4814: comm kworker/u48:15: mark_inode_dirty error
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-9): kmmpd:185: comm kmmpd-dm-9: Error writing to MMP block
Sep 20 22:03:36 proxmox kernel: EXT4-fs warning (device dm-8): ext4_end_bio:342: I/O error 10 writing to inode 1763 starting block 82422)
Sep 20 22:03:36 proxmox kernel: Aborting journal on device dm-20-8.
Sep 20 22:03:36 proxmox kernel: Aborting journal on device dm-9-8.
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-6) in ext4_convert_unwritten_io_end_vec:4923: Journal has aborted
Sep 20 22:03:36 proxmox kernel: EXT4-fs (dm-6): failed to convert unwritten extents to written extents -- potential data loss! (inode 4814, error -30)
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on device dm-8, logical block 82422
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-6) in ext4_reserve_inode_write:5880: Journal has aborted
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on dev dm-22, logical block 9255, lost sync page write
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-6): ext4_dirty_inode:6084: inode #1329: comm tracer: mark_inode_dirty error
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-22): kmmpd:185: comm kmmpd-dm-22: Error writing to MMP block
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-6) in ext4_dirty_inode:6085: Journal has aborted
Sep 20 22:03:36 proxmox kernel: Aborting journal on device dm-23-8.
From time to time my entire system stops responding and the only way to get it back is to do a hard reset.
Sometimes it goes a while before it happens again just as now. But this time I was accessing servers on my Proxmox when it happens so I knew the time and everything.
So I fetched the logs from that time (see below). I'm a novice guy and don't know anything regarding this. Anyone that can help me understand what is happening.
As I see it it's regarding the HDDs. But it is actually the HHDs or the system itself? What can I do to fix this? Any suggestions?
Sep 20 22:01:43 proxmox kernel: ahci 0000:16:00.1: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000a address=0xfebb1000 flags=0x0000]
Sep 20 22:01:44 proxmox kernel: ata2.00: exception Emask 0x10 SAct 0x60008000 SErr 0x40000 action 0x6 frozen
Sep 20 22:01:44 proxmox kernel: ata2.00: irq_stat 0x08000000, interface fatal error
Sep 20 22:01:44 proxmox kernel: ata2: SError: { CommWake }
Sep 20 22:01:44 proxmox kernel: ata2.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:01:44 proxmox kernel: ata2.00: cmd 61/28:78:10:de:98/00:00:0c:00:00/40 tag 15 ncq dma 20480 out res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x10 (ATA bus error)
Sep 20 22:01:44 proxmox kernel: ata2.00: status: { DRDY }
Sep 20 22:01:44 proxmox kernel: ata2.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:01:44 proxmox kernel: ata2.00: cmd 61/60:e8:00:e4:49/00:00:0c:00:00/40 tag 29 ncq dma 49152 out res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x10 (ATA bus error)
Sep 20 22:01:44 proxmox kernel: ata2.00: status: { DRDY }
Sep 20 22:01:44 proxmox kernel: ata2.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:01:44 proxmox kernel: ata2.00: cmd 61/08:f0:b8:ce:45/00:00:0c:00:00/40 tag 30 ncq dma 4096 out res 40/00:01:06:4f:c2/00:00:00:00:00/00 Emask 0x10 (ATA bus error)
Sep 20 22:01:44 proxmox kernel: ata2.00: status: { DRDY }
Sep 20 22:01:44 proxmox kernel: ata2: hard resetting link
Sep 20 22:01:44 proxmox kernel: ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Sep 20 22:01:44 proxmox kernel: ata2.00: supports DRM functions and may not be fully accessible
Sep 20 22:01:59 proxmox kernel: ata2.00: qc timeout after 15000 msecs (cmd 0x2f)
Sep 20 22:03:36 proxmox kernel: ata2.00: Read log 0x00 page 0x00 failed, Emask 0x4
Sep 20 22:03:36 proxmox kernel: ata2.00: NCQ Send/Recv Log not supported
Sep 20 22:03:36 proxmox kernel: ata2.00: Read log 0x00 page 0x00 failed, Emask 0x40
Sep 20 22:03:36 proxmox kernel: ata2.00: ATA Identify Device Log not supported
Sep 20 22:03:36 proxmox kernel: ata2.00: Security Log not supported
Sep 20 22:03:36 proxmox kernel: ata2.00: Read log 0x00 page 0x00 failed, Emask 0x40
Sep 20 22:03:36 proxmox kernel: ata2.00: Read log 0x00 page 0x00 failed, Emask 0x40
Sep 20 22:03:36 proxmox kernel: ata2.00: failed to set xfermode (err_mask=0x40)
Sep 20 22:03:36 proxmox kernel: ata2: hard resetting link
Sep 20 22:03:36 proxmox kernel: ata2: softreset failed (1st FIS failed)
Sep 20 22:03:36 proxmox kernel: ata2: hard resetting link
Sep 20 22:03:36 proxmox kernel: ata2: softreset failed (1st FIS failed)
Sep 20 22:03:36 proxmox kernel: ata2: hard resetting link
Sep 20 22:03:36 proxmox kernel: ata1.00: exception Emask 0x0 SAct 0x4001fed8 SErr 0x50000 action 0x6 frozen
Sep 20 22:03:36 proxmox kernel: ata1: SError: { PHYRdyChg CommWake }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:18:c0:75:ed/00:00:01:00:00/40 tag 3 ncq dma 4096 out res 40/00:01:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:20:00:e6:87/00:00:02:00:00/40 tag 4 ncq dma 4096 out res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/48:30:88:90:2e/00:00:07:00:00/40 tag 6 ncq dma 36864 out res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:38:00:e7:87/00:00:02:00:00/40 tag 7 ncq dma 4096 out res 40/00:01:06:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:48:00:17:e0/00:00:07:00:00/40 tag 9 ncq dma 4096 out res 40/00:01:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:50:a8:50:cd/00:00:08:00:00/40 tag 10 ncq dma 4096 out res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:58:38:ea:87/00:00:02:00:00/40 tag 11 ncq dma 4096 out res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:60:a8:ec:87/00:00:02:00:00/40 tag 12 ncq dma 4096 out res 40/00:01:06:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/20:68:90:e4:87/00:00:02:00:00/40 tag 13 ncq dma 16384 out res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/10:70:58:da:86/00:00:02:00:00/40 tag 14 ncq dma 8192 out res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:78:d8:da:86/00:00:02:00:00/40 tag 15 ncq dma 4096 out res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:80:38:dd:86/00:00:02:00:00/40 tag 16 ncq dma 4096 out res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1.00: failed command: WRITE FPDMA QUEUED
Sep 20 22:03:36 proxmox kernel: ata1.00: cmd 61/08:f0:b8:18:02/00:00:04:00:00/40 tag 30 ncq dma 4096 out res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Sep 20 22:03:36 proxmox kernel: ata1.00: status: { DRDY }
Sep 20 22:03:36 proxmox kernel: ata1: hard resetting link
Sep 20 22:03:36 proxmox kernel: ata1: softreset failed (1st FIS failed)
Sep 20 22:03:36 proxmox kernel: ata1: hard resetting link
Sep 20 22:03:36 proxmox kernel: ata2: softreset failed (1st FIS failed)
Sep 20 22:03:36 proxmox kernel: ata2: limiting SATA link speed to 3.0 Gbps
Sep 20 22:03:36 proxmox kernel: ata2: hard resetting link
Sep 20 22:03:36 proxmox kernel: ata1: softreset failed (1st FIS failed)
Sep 20 22:03:36 proxmox kernel: ata1: hard resetting link
Sep 20 22:03:36 proxmox kernel: ata2: softreset failed (1st FIS failed)
Sep 20 22:03:36 proxmox kernel: ata2: softreset failed
Sep 20 22:03:36 proxmox kernel: ata2: reset failed, giving up
Sep 20 22:03:36 proxmox kernel: ata2.00: disable device
Sep 20 22:03:36 proxmox kernel: ahci 0000:16:00.1: port does not support device sleep
Sep 20 22:03:36 proxmox kernel: ata2: EH complete
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#1 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#1 CDB: Synchronize Cache(10) 35 00 00 00 00 00 00 00 00 00
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 137312384 op 0x1

Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#2 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#2 CDB: Write(10) 2a 00 0b c1 eb c8 00 00 08 00
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 197258184 op 0x1

Sep 20 22:03:36 proxmox kernel: Buffer I/O error on dev dm-15, logical block 153929, lost async page write
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#3 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#3 CDB: Write(10) 2a 00 0c 44 fe b8 00 00 08 00
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 205848248 op 0x1

Sep 20 22:03:36 proxmox kernel: Buffer I/O error on dev dm-6, logical block 9255, lost sync page write
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#9 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#9 CDB: Write(10) 2a 00 0c 45 c1 68 00 00 18 00
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 205898088 op 0x1

Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#4 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=75s
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-6): kmmpd:185: comm kmmpd-dm-6: Error writing to MMP block
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#4 CDB: Write(10) 2a 00 0c 98 de 10 00 00 28 00
Sep 20 22:03:36 proxmox kernel: EXT4-fs warning (device dm-8): ext4_end_bio:342: I/O error 10 writing to inode 1763 starting block 82093)
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 211344912 op 0x1

Sep 20 22:03:36 proxmox kernel: Aborting journal on device dm-6-8.
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on device dm-8, logical block 82093
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on device dm-8, logical block 82094
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on device dm-8, logical block 82095
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#10 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#10 CDB: Write(10) 2a 00 0c 45 c1 f0 00 00 08 00
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#5 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=76s
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 205898224 op 0x1

Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#5 CDB: Write(10) 2a 00 0c 49 e4 00 00 00 60 00
Sep 20 22:03:36 proxmox kernel: Aborting journal on device dm-22-8.
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 206169088 op 0x1

Sep 20 22:03:36 proxmox kernel: EXT4-fs warning (device dm-8): ext4_end_bio:342: I/O error 10 writing to inode 1763 starting block 82126)
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on device dm-8, logical block 82126
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#11 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-22): ext4_journal_check_start:84: comm node: Detected aborted journal
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#11 CDB: Write(10) 2a 00 0c 45 c4 78 00 00 08 00
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 205898872 op 0x1

Sep 20 22:03:36 proxmox kernel: EXT4-fs warning (device dm-8): ext4_end_bio:342: I/O error 10 writing to inode 1763 starting block 82415)
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on device dm-8, logical block 82415
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#6 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=75s
Sep 20 22:03:36 proxmox kernel: EXT4-fs warning (device dm-6): ext4_convert_unwritten_extents:4879: inode #4814: block 2541: len 2: ext4_ext_map_blocks returned -30
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#6 CDB: Write(10) 2a 00 0c 45 ce b8 00 00 08 00
Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#12 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 205901496 op 0x1

Sep 20 22:03:36 proxmox kernel: sd 1:0:0:0: [sdb] tag#12 CDB: Write(10) 2a 00 0c 45 c4 b0 00 00 08 00
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-6) in ext4_reserve_inode_write:5880: Journal has aborted
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on dev dm-20, logical block 9255, lost sync page write
Sep 20 22:03:36 proxmox kernel: I/O error, dev sdb, sector 205898928 op 0x1

Sep 20 22:03:36 proxmox kernel: Buffer I/O error on dev dm-9, logical block 9255, lost sync page write
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-20): kmmpd:185: comm kmmpd-dm-20: Error writing to MMP block
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-6): ext4_convert_unwritten_extents:4884: inode #4814: comm kworker/u48:15: mark_inode_dirty error
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-9): kmmpd:185: comm kmmpd-dm-9: Error writing to MMP block
Sep 20 22:03:36 proxmox kernel: EXT4-fs warning (device dm-8): ext4_end_bio:342: I/O error 10 writing to inode 1763 starting block 82422)
Sep 20 22:03:36 proxmox kernel: Aborting journal on device dm-20-8.
Sep 20 22:03:36 proxmox kernel: Aborting journal on device dm-9-8.
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-6) in ext4_convert_unwritten_io_end_vec:4923: Journal has aborted
Sep 20 22:03:36 proxmox kernel: EXT4-fs (dm-6): failed to convert unwritten extents to written extents -- potential data loss! (inode 4814, error -30)
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on device dm-8, logical block 82422
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-6) in ext4_reserve_inode_write:5880: Journal has aborted
Sep 20 22:03:36 proxmox kernel: Buffer I/O error on dev dm-22, logical block 9255, lost sync page write
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-6): ext4_dirty_inode:6084: inode #1329: comm tracer: mark_inode_dirty error
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-22): kmmpd:185: comm kmmpd-dm-22: Error writing to MMP block
Sep 20 22:03:36 proxmox kernel: EXT4-fs error (device dm-6) in ext4_dirty_inode:6085: Journal has aborted
Sep 20 22:03:36 proxmox kernel: Aborting journal on device dm-23-8.