[SOLVED] Big problem : vms don't start et errors messages :(

[Bristow]

Hello to all,

This weekend, I found my server without any free disk space on my ProxMox 6.0 server.

I fixed the problem by deleting the syslog, message and kern.log file.

Since then, I updated my proxmox to 6.4.6 but the problem persists. I have the impression to have problems on my disks, can you confirm me and tell me if there is something to do?

/var/log/message extract :

May 31 14:46:00 proxmox kernel: [ 125.576697] EXT4-fs warning (device dm-16): ext4_clear_journal_err:5191: Filesystem error recorded from previous mount: IO failure
May 31 14:46:00 proxmox kernel: [ 125.576701] EXT4-fs warning (device dm-16): ext4_clear_journal_err:5192: Marking fs in need of filesystem check.
May 31 14:46:00 proxmox kernel: [ 125.621179] EXT4-fs (dm-16): warning: mounting fs with errors, running e2fsck is recommended
May 31 14:46:00 proxmox kernel: [ 125.661083] EXT4-fs (dm-16): recovery complete
May 31 14:46:00 proxmox kernel: [ 125.672863] EXT4-fs (dm-16): mounted filesystem with ordered data mode. Opts: (null)
May 31 14:46:01 proxmox kernel: [ 126.046555] EXT4-fs warning (device dm-16): ext4_end_bio:315: I/O error 3 writing to inode 131101 (offset 0 size 4096 starting block 557066)
May 31 14:46:01 proxmox kernel: [ 126.314811] EXT4-fs warning (device dm-16): ext4_end_bio:315: I/O error 3 writing to inode 525786 (offset 0 size 4096 starting block 2129951)
May 31 14:46:01 proxmox kernel: [ 126.349855] EXT4-fs warning (device dm-16): ext4_end_bio:315: I/O error 3 writing to inode 525787 (offset 0 size 4096 starting block 2130014)
May 31 14:46:01 proxmox kernel: [ 126.355683] EXT4-fs warning (device dm-16): ext4_end_bio:315: I/O error 3 writing to inode 525788 (offset 0 size 4096 starting block 2130265)
May 31 14:46:01 proxmox kernel: [ 126.364955] EXT4-fs warning (device dm-16): ext4_end_bio:315: I/O error 3 writing to inode 525335 (offset 0 size 4096 starting block 2130377)
May 31 14:46:03 proxmox kernel: [ 127.819756] audit: type=1400 audit(1622465163.112:21): apparmor="STATUS" operation="profile_load" profile="/usr/bin/lxc-start" name="lxc-140_</var/lib/lxc>" pid=2257 comm="apparmor_parser"
May 31 14:46:04 proxmox kernel: [ 129.372370] fwbr140i0: port 1(fwln140i0) entered blocking state
May 31 14:46:04 proxmox kernel: [ 129.372376] fwbr140i0: port 1(fwln140i0) entered disabled state
May 31 14:46:04 proxmox kernel: [ 129.374546] device fwln140i0 entered promiscuous mode
May 31 14:46:04 proxmox kernel: [ 129.374711] fwbr140i0: port 1(fwln140i0) entered blocking state
May 31 14:46:04 proxmox kernel: [ 129.374715] fwbr140i0: port 1(fwln140i0) entered forwarding state
May 31 14:46:04 proxmox kernel: [ 129.396969] vmbr0: port 5(fwpr140p0) entered blocking state
May 31 14:46:04 proxmox kernel: [ 129.396975] vmbr0: port 5(fwpr140p0) entered disabled state
May 31 14:46:04 proxmox kernel: [ 129.398074] device fwpr140p0 entered promiscuous mode
May 31 14:46:04 proxmox kernel: [ 129.398166] vmbr0: port 5(fwpr140p0) entered blocking state
May 31 14:46:04 proxmox kernel: [ 129.398170] vmbr0: port 5(fwpr140p0) entered forwarding state
May 31 14:46:04 proxmox kernel: [ 129.410068] fwbr140i0: port 2(veth140i0) entered blocking state
May 31 14:46:04 proxmox kernel: [ 129.410073] fwbr140i0: port 2(veth140i0) entered disabled state
May 31 14:46:04 proxmox kernel: [ 129.410991] device veth140i0 entered promiscuous mode
May 31 14:46:04 proxmox kernel: [ 129.485465] eth0: renamed from vethHzxLSU
May 31 14:46:06 proxmox kernel: [ 130.814739] JBD2: Detected IO errors while flushing file data on dm-16-8
May 31 14:46:07 proxmox kernel: [ 132.406467] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
May 31 14:46:07 proxmox kernel: [ 132.406553] fwbr140i0: port 2(veth140i0) entered blocking state
May 31 14:46:07 proxmox kernel: [ 132.406557] fwbr140i0: port 2(veth140i0) entered forwarding state
May 31 14:46:08 proxmox kernel: [ 133.480875] buffer_io_error: 15 callbacks suppressed
May 31 14:46:09 proxmox kernel: [ 133.997018] JBD2: recovery failed
May 31 14:46:12 proxmox kernel: [ 136.725973] JBD2: recovery failed
May 31 14:46:16 proxmox kernel: [ 140.982055] perf: interrupt took too long (4002 > 3993), lowering kernel.perf_event_max_sample_rate to 49750
May 31 14:46:23 proxmox kernel: [ 148.109800] buffer_io_error: 5 callbacks suppressed
May 31 14:46:23 proxmox kernel: [ 148.235138] JBD2: recovery failed
May 31 14:46:26 proxmox kernel: [ 151.289257] JBD2: recovery failed
May 31 14:46:29 proxmox kernel: [ 153.706787] buffer_io_error: 72 callbacks suppressed
May 31 14:46:29 proxmox kernel: [ 154.122204] JBD2: recovery failed
May 31 14:47:18 proxmox kernel: [ 202.718441] perf: interrupt took too long (5005 > 5002), lowering kernel.perf_event_max_sample_rate to 39750
May 31 14:49:44 proxmox kernel: [ 349.478249] EXT4-fs (dm-9): error count since last fsck: 9
May 31 14:49:44 proxmox kernel: [ 349.478274] EXT4-fs (dm-9): initial error at time 1622333274: ext4_journal_check_start:61
May 31 14:49:44 proxmox kernel: [ 349.478283] EXT4-fs (dm-9): last error at time 1622465134: ext4_journal_check_start:61
May 31 14:50:00 proxmox kernel: [ 365.174028] perf: interrupt took too long (6258 > 6256), lowering kernel.perf_event_max_sample_rate to 31750
May 31 14:51:06 proxmox kernel: [ 431.402238] EXT4-fs (dm-16): error count since last fsck: 8
May 31 14:51:06 proxmox kernel: [ 431.402242] EXT4-fs (dm-16): initial error at time 1622336480: ext4_journal_check_start:61
May 31 14:51:06 proxmox kernel: [ 431.402245] EXT4-fs (dm-16): last error at time 1622465166: ext4_journal_check_start:61

What can I do? For your information, some of my VMs turn on but Apache doesn't work

Thanks to all !

 

[Bristow]

I found the problem.