Corrupt ext4 inside a KVM guest after Backuping

chris121212

Active Member
May 26, 2016
24
0
41
38
Ingolstadt, Bayern
Hey guys,

i'm running pve 4.4.79 on a dedicated machine with 4 disks. 2* mirror zfs and 2*mirror zfs.
There are a few of lxc containers and one kvm virtual machine.

The kvm guest has 3 (raw) partitons from both pools. I'm using ext4 (sda, vda, vdb) inside the guest.
The performance is ok and the system is running smoothly.

I do a daily snapshot backup (via proxmox) of the kvm machine in reasons of desaster recovery.
Unfortunately, there is a problem on around every third backup for the kvm guest.
The ext4 on sda gots corrupted when backup starts and goes in ro mode, sometimes the whole vm hangs. It's needed to restart the system and do fsck on the sda.
sda is mounted on /

How can i fix that problem?

chris
 
What does the task log tell? Is there enough free space left on all volumes (virtual disks, volumes they reside on, backup storage)?
 
Snapshot is not backup. Moreover, PVE can not do proper backup of running VM, so filesystem corruption can be expected...
 
What does the task log tell? Is there enough free space left on all volumes (virtual disks, volumes they reside on, backup storage)?

Task Log for the job from Sunday - which the problem happened:
Code:
INFO: trying to get global lock - waiting...
INFO: got global lock
INFO: starting new backup job: vzdump 111 --mailto xx@xx.com --mode snapshot --mailnotification always --compress lzo --node XXX --quiet 1 --storage backup1
INFO: Starting Backup of VM 111 (qemu)
INFO: status = running
INFO: update VM 111: -lock backup
INFO: VM Name: xxx.de
INFO: include disk 'virtio3' 'ktzfs:vm-111-disk-3' 200G
INFO: exclude disk 'virtio4' 'ktzfs:vm-111-disk-1' (backup=no)
INFO: include disk 'sata0' 'ssd:vm-111-disk-1' 200G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating archive '/ktpool/backup1/dump/vzdump-qemu-111-2017_09_16-02_49_17.vma.lzo'
INFO: started backup task '0939c628-792b-4489-a03c-cb60cd76e945'
INFO: status: 0% (634781696/429496729600), sparse 0% (110776320), duration 3, 211/174 MB/s
INFO: status: 1% (4424466432/429496729600), sparse 0% (349581312), duration 25, 172/161 MB/s
INFO: status: 2% (8661368832/429496729600), sparse 0% (530669568), duration 88, 67/64 MB/s 
....
INFO: status: 100% (429496729600/429496729600), sparse 11% (48801419264), duration 7466, 1065/0 MB/s
INFO: transferred 429496 MB in 7466 seconds (57 MB/s)
INFO: archive file size: 300.72GB
INFO: delete old backup '/ktpool/backup1/dump/vzdump-qemu-111-2017_09_15-02_38_49.vma.lzo'
INFO: Finished Backup of VM 111 (02:04:43)
INFO: Backup job finished successfully
TASK OK

df -h inside the container:
Code:
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1       197G   95G   92G  51% /
udev             10M     0   10M   0% /dev
tmpfs           1.6G  121M  1.5G   8% /run
tmpfs           4.0G     0  4.0G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           4.0G     0  4.0G   0% /sys/fs/cgroup
/dev/vda        197G  129G   58G  69% /var/qmail
/dev/vdb        187G   20G  158G  12% /var/lib/psa/dumps

Backupstorage is 457 GB available
"volumes they reside on" I'm not sure what u mean with that. The ZFS Volumes are raw and reserved in the system.
 
The ZFS Volumes are raw and reserved in the system.
Yes, but it's possible to over-commission. You can create a 32G zvol in a pool with 8G free space. Basically I'd just like to be sure that all parties have enough free space. You need to be able to store one more backup than the max backups you configured for the backup storage. So if you have a VM that can be 25G at max and want to store 3 backups, you need 100G backup storage space because the new backup is written before the oldest one is deleted. You probably want some overhead, but you get the point. But 457G should be enough.
Is this the log of the corrupted backup?
 
Yes thats right. It's from that moment.
I'm not sure if the backup is corrupted - its more the running filesystem is broken.

Here is the syslog from inside the kvm, maybe that gives you more information:
Code:
Sep 17 02:34:51 #hostname# qemu-ga: info: guest-fsfreeze called
Sep 17 02:35:01 #hostname# CRON[15848]: (root) CMD (/opt/psa/admin/bin/php -dauto_prepend_file=sdk.php '/opt/psa/admin/plib/modules/magicspam/scripts/ms_clean_queue.php')
Sep 17 02:35:01 #hostname# CRON[15847]: (root) CMD (/opt/psa/admin/bin/php -dauto_prepend_file=sdk.php '/opt/psa/admin/plib/modules/dropbox-backup/scripts/plesk-task-manager.php')
Sep 17 02:35:05 #hostname# spamd[3402]: bayes: cannot write to /var/qmail/mailnames/xxxx/info/.spamassassin/bayes_journal, bayes db update ignored: Permission denied
Sep 17 02:35:20 #hostname# plesk_saslauthd[15844]: select timeout, exiting
Sep 17 02:35:27 #hostname# kernel: [201235.768128] INFO: rcu_sched detected stalls on CPUs/tasks: { 0} (detected by 3, t=5252 jiffies, g=6908836, c=6908835, q=745)
Sep 17 02:35:27 #hostname# kernel: [201235.772094] sending NMI to all CPUs:
Sep 17 02:35:27 #hostname# kernel: [201236.673904] NMI backtrace for cpu 0
Sep 17 02:35:27 #hostname# kernel: [201236.673912] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.16.0-4-amd64 #1 Debian 3.16.43-2+deb8u3
Sep 17 02:35:27 #hostname# kernel: [201236.673914] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.9.3-0-ge2fc41e-prebuilt.qemu-project.org 04/01/2014
Sep 17 02:35:27 #hostname# kernel: [201236.673915] task: ffffffff8181a460 ti: ffffffff81800000 task.ti: ffffffff81800000
Sep 17 02:35:27 #hostname# kernel: [201236.673918] RIP: 0010:[<ffffffff81519f99>]  [<ffffffff81519f99>] _raw_spin_lock_irqsave+0x39/0x50
Sep 17 02:35:27 #hostname# kernel: [201236.673936] RSP: 0018:ffff88023fc03d58  EFLAGS: 00000002
Sep 17 02:35:27 #hostname# kernel: [201236.673944] RAX: 000000000000ca17 RBX: ffff88002c577800 RCX: 000000000000ca17
Sep 17 02:35:27 #hostname# kernel: [201236.673945] RDX: 000000000000ca16 RSI: 0000000000000286 RDI: ffff880036c56718
Sep 17 02:35:27 #hostname# kernel: [201236.673946] RBP: ffff880036f3e800 R08: 0000000000000000 R09: ffff880199e3a288
Sep 17 02:35:27 #hostname# kernel: [201236.673946] R10: ffff8800776ae400 R11: 0000000000000000 R12: ffff880036f20000
Sep 17 02:35:27 #hostname# kernel: [201236.673947] R13: ffff88002c577800 R14: ffff880036f3e848 R15: ffff880199e3a180
Sep 17 02:35:27 #hostname# kernel: [201236.673949] FS:  0000000000000000(0000) GS:ffff88023fc00000(0000) knlGS:0000000000000000
Sep 17 02:35:27 #hostname# kernel: [201236.673950] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Sep 17 02:35:27 #hostname# kernel: [201236.673952] CR2: 0000000009b11000 CR3: 0000000098dce000 CR4: 00000000000006f0
Sep 17 02:35:27 #hostname# kernel: [201236.673956] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 17 02:35:27 #hostname# kernel: [201236.673957] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Sep 17 02:35:27 #hostname# kernel: [201236.673958] Stack:
Sep 17 02:35:27 #hostname# kernel: [201236.673959]  ffffffffa01261ba 0000000000000000 ffff88002c577800 ffff8800ba7c2800
Sep 17 02:35:27 #hostname# kernel: [201236.673963]  ffff8800ba7c2800 ffff88002c577800 ffff880036f3e848 ffff880199e3a180
Sep 17 02:35:27 #hostname# kernel: [201236.673964]  ffffffffa00303e4 ffff880036f3e800 ffff8800ba7c2800 ffff880036c94008
Sep 17 02:35:27 #hostname# kernel: [201236.673966] Call Trace:
Sep 17 02:35:27 #hostname# kernel: [201236.673967]  <IRQ>
Sep 17 02:35:27 #hostname# kernel: [201236.674266]  [<ffffffffa01261ba>] ? ata_scsi_queuecmd+0x2a/0x410 [libata]
Sep 17 02:35:27 #hostname# kernel: [201236.674394]  [<ffffffffa00303e4>] ? scsi_dispatch_cmd+0xb4/0x2d0 [scsi_mod]
Sep 17 02:35:27 #hostname# kernel: [201236.674401]  [<ffffffffa0038c1d>] ? scsi_request_fn+0x2fd/0x500 [scsi_mod]
Sep 17 02:35:27 #hostname# kernel: [201236.674532]  [<ffffffff81280dbf>] ? __blk_run_queue+0x2f/0x40
Sep 17 02:35:27 #hostname# kernel: [201236.674534]  [<ffffffff81280e31>] ? blk_run_queue+0x21/0x40
Sep 17 02:35:27 #hostname# kernel: [201236.674539]  [<ffffffffa00386c4>] ? scsi_run_queue+0x134/0x280 [scsi_mod]
Sep 17 02:35:27 #hostname# kernel: [201236.674544]  [<ffffffffa003917b>] ? scsi_next_command+0x1b/0x30 [scsi_mod]
Sep 17 02:35:27 #hostname# kernel: [201236.674548]  [<ffffffffa00393b0>] ? scsi_io_completion+0x1d0/0x790 [scsi_mod]
Sep 17 02:35:27 #hostname# kernel: [201236.674551]  [<ffffffff8128a43b>] ? blk_done_softirq+0x7b/0x90
Sep 17 02:35:27 #hostname# kernel: [201236.674558]  [<ffffffff8106d941>] ? __do_softirq+0xf1/0x2d0
Sep 17 02:35:27 #hostname# kernel: [201236.674560]  [<ffffffff8106dd55>] ? irq_exit+0x95/0xa0
Sep 17 02:35:27 #hostname# kernel: [201236.674564]  [<ffffffff8151d2a2>] ? do_IRQ+0x52/0xe0
Sep 17 02:35:27 #hostname# kernel: [201236.674567]  [<ffffffff8151b0ed>] ? common_interrupt+0x6d/0x6d
Sep 17 02:35:27 #hostname# kernel: [201236.674567]  <EOI>
Sep 17 02:35:27 #hostname# kernel: [201236.674568]  [<ffffffff8101da90>] ? mwait_idle+0xa0/0xa0
Sep 17 02:35:27 #hostname# kernel: [201236.674695]  [<ffffffff81052c22>] ? native_safe_halt+0x2/0x10
Sep 17 02:35:27 #hostname# kernel: [201236.674698]  [<ffffffff8101daa9>] ? default_idle+0x19/0xd0
Sep 17 02:35:27 #hostname# kernel: [201236.674704]  [<ffffffff810a9bc4>] ? cpu_startup_entry+0x374/0x470
Sep 17 02:35:27 #hostname# kernel: [201236.674706]  [<ffffffff81904076>] ? start_kernel+0x497/0x4a2
Sep 17 02:35:27 #hostname# kernel: [201236.674708]  [<ffffffff81903a04>] ? set_init_arg+0x4e/0x4e
Sep 17 02:35:27 #hostname# kernel: [201236.674710]  [<ffffffff81903120>] ? early_idt_handler_array+0x120/0x120
Sep 17 02:35:27 #hostname# kernel: [201236.674712]  [<ffffffff8190371f>] ? x86_64_start_kernel+0x14d/0x15c
Sep 17 02:35:27 #hostname# kernel: [201236.674713] Code: c6 fa 66 0f 1f 44 00 00 ba 00 00 01 00 f0 0f c1 17 89 d1 c1 e9 10 66 39 d1 89 c8 75 04 48 89 f0 c3 0f b7 17 66 39 d1 74 f4 f3 90 <0f> b7 17 66 39 d0 75 f6 eb e8 66 66 66 66 2e 0f 1f 84 00 00 00
Sep 17 02:35:27 #hostname# kernel: [201235.772094] NMI backtrace for cpu 3
Sep 17 02:35:27 #hostname# kernel: [201235.772094] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 3.16.0-4-amd64 #1 Debian 3.16.43-2+deb8u3
Sep 17 02:35:27 #hostname# kernel: [201235.772094] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.9.3-0-ge2fc41e-prebuilt.qemu-project.org 04/01/2014
Sep 17 02:35:27 #hostname# kernel: [201235.772094] task: ffff880236da73b0 ti: ffff880236db0000 task.ti: ffff880236db0000
Sep 17 02:35:27 #hostname# kernel: [201235.772094] RIP: 0010:[<ffffffff81052ae6>]  [<ffffffff81052ae6>] native_write_msr_safe+0x6/0x10
Sep 17 02:35:27 #hostname# kernel: [201235.772094] RSP: 0018:ffff88023fd83e08  EFLAGS: 00000046
Sep 17 02:35:27 #hostname# kernel: [201235.772094] RAX: 0000000000000400 RBX: 0000000000000003 RCX: 0000000000000830
Sep 17 02:35:27 #hostname# kernel: [201235.772094] RDX: 0000000000000003 RSI: 0000000000000400 RDI: 0000000000000830
Sep 17 02:35:27 #hostname# kernel: [201235.772094] RBP: ffffffff818e2b40 R08: ffffffff818e2b40 R09: 00000000000002cc
Sep 17 02:35:27 #hostname# kernel: [201235.772094] R10: 61206f7420494d4e R11: 3a73555043206c6c R12: 0000000000000003
Sep 17 02:35:27 #hostname# kernel: [201235.772094] R13: 000000000000a0ea R14: 0000000000000002 R15: 0000000000080000
Sep 17 02:35:27 #hostname# kernel: [201235.772094] FS:  0000000000000000(0000) GS:ffff88023fd80000(0000) knlGS:0000000000000000
Sep 17 02:35:27 #hostname# kernel: [201235.772094] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Sep 17 02:35:27 #hostname# kernel: [201235.772094] CR2: 00007fc26adb7000 CR3: 0000000235672000 CR4: 00000000000006e0
Sep 17 02:35:27 #hostname# kernel: [201235.772094] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 17 02:35:27 #hostname# kernel: [201235.772094] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Sep 17 02:35:27 #hostname# kernel: [201235.772094] Stack:
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  ffffffff8104b34d 0000000000000002 0000000000000082 ffff88023fd8d6a0
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  ffffffff81853840 0000000000000003 ffffffff818e30a0 00000000000002e9
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  ffffffff81853840 ffffffff81047da3 ffff88023fd8d6a0 ffffffff810c770d
Sep 17 02:35:27 #hostname# kernel: [201235.772094] Call Trace:
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  <IRQ>
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff8104b34d>] ? __x2apic_send_IPI_mask+0xad/0xe0
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff81047da3>] ? arch_trigger_all_cpu_backtrace+0xc3/0x140
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff810c770d>] ? rcu_check_callbacks+0x64d/0x670
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff810d1ef0>] ? tick_sched_handle.isra.16+0x60/0x60
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff81076000>] ? update_process_times+0x40/0x70
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff810d1eb0>] ? tick_sched_handle.isra.16+0x20/0x60
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff810d1f2c>] ? tick_sched_timer+0x3c/0x60
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff8108c6f7>] ? __run_hrtimer+0x67/0x210
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff8108caf9>] ? hrtimer_interrupt+0xe9/0x220
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff8151d36b>] ? smp_apic_timer_interrupt+0x3b/0x50
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff8151b3fd>] ? apic_timer_interrupt+0x6d/0x80
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  <EOI>
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff8109b335>] ? sched_clock_local+0x15/0x80
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff8101da90>] ? mwait_idle+0xa0/0xa0
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff81052c22>] ? native_safe_halt+0x2/0x10
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff8101daa9>] ? default_idle+0x19/0xd0
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff810a9bc4>] ? cpu_startup_entry+0x374/0x470
Sep 17 02:35:27 #hostname# kernel: [201235.772094]  [<ffffffff8104401f>] ? start_secondary+0x20f/0x2d0
Sep 17 02:35:27 #hostname# kernel: [201235.772094] Code: c2 48 89 d0 c3 89 f9 0f 32 31 c9 48 c1 e2 20 89 c0 89 0e 48 09 c2 48 89 d0 c3 66 66 2e 0f 1f 84 00 00 00 00 00 89 f0 89 f9 0f 30 <31> c0 c3 0f 1f 80 00 00 00 00 89 f9 0f 33 48 c1 e2 20 89 c0 48
Sep 17 02:35:27 #hostname# kernel: [201236.675099] NMI backtrace for cpu 2
Sep 17 02:35:27 #hostname# kernel: [201236.675102] CPU: 2 PID: 14047 Comm: imap Not tainted 3.16.0-4-amd64 #1 Debian 3.16.43-2+deb8u3
Sep 17 02:35:27 #hostname# kernel: [201236.675104] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.9.3-0-ge2fc41e-prebuilt.qemu-project.org 04/01/2014
Sep 17 02:35:27 #hostname# kernel: [201236.675105] task: ffff880236937670 ti: ffff88005f050000 task.ti: ffff88005f050000
Sep 17 02:35:27 #hostname# kernel: [201236.675107] RIP: 0033:[<00007f3b3e522fb0>]  [<00007f3b3e522fb0>] 0x7f3b3e522fb0
Sep 17 02:35:27 #hostname# kernel: [201236.675123] RSP: 002b:00007ffc6092ff08  EFLAGS: 00010206
Sep 17 02:35:27 #hostname# kernel: [201236.675124] RAX: 00000000025dce50 RBX: 00007f3b3e8359b1 RCX: 00000000025dce50
Sep 17 02:35:27 #hostname# kernel: [201236.675125] RDX: 00007f3b3e49a628 RSI: 00007ffc6092ff10 RDI: 00007f3b3e8359b1
Sep 17 02:35:27 #hostname# kernel: [201236.675126] RBP: 00000000025ec130 R08: 0000000000000000 R09: 00000000025fb4b0
Sep 17 02:35:27 #hostname# kernel: [201236.675127] R10: 0000000000000003 R11: 0000000000000000 R12: 00007ffc60930030
Sep 17 02:35:27 #hostname# kernel: [201236.675128] R13: 00007ffc60930020 R14: 000000000000000c R15: 0000000000000000
Sep 17 02:35:27 #hostname# kernel: [201236.675129] FS:  00007f3b3ec79700(0000) GS:ffff88023fd00000(0000) knlGS:0000000000000000
Sep 17 02:35:27 #hostname# kernel: [201236.675130] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Sep 17 02:35:27 #hostname# kernel: [201236.675131] CR2: 00000000099a6002 CR3: 0000000199fa9000 CR4: 00000000000006e0
Sep 17 02:35:27 #hostname# kernel: [201236.675134] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 17 02:35:27 #hostname# kernel: [201236.675135] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Sep 17 02:35:27 #hostname# kernel: [201236.675135]
Sep 17 02:35:27 #hostname# kernel: [201236.675285] NMI backtrace for cpu 1
Sep 17 02:35:27 #hostname# kernel: [201236.675289] CPU: 1 PID: 1162 Comm: mysqld Not tainted 3.16.0-4-amd64 #1 Debian 3.16.43-2+deb8u3
Sep 17 02:35:27 #hostname# kernel: [201236.675291] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.9.3-0-ge2fc41e-prebuilt.qemu-project.org 04/01/2014
Sep 17 02:35:27 #hostname# kernel: [201236.675292] task: ffff8800ba49f5b0 ti: ffff8802358cc000 task.ti: ffff8802358cc000
Sep 17 02:35:27 #hostname# kernel: [201236.675293] RIP: 0010:[<ffffffffa0165a3a>]  [<ffffffffa0165a3a>] ahci_qc_issue+0x8a/0x110 [libahci]
Sep 17 02:35:27 #hostname# kernel: [201236.675312] RSP: 0018:ffff8802358cfbc8  EFLAGS: 00000006
Sep 17 02:35:27 #hostname# kernel: [201236.675313] RAX: 0000000000400000 RBX: ffff880036f20000 RCX: 0000000000000016
Sep 17 02:35:27 #hostname# kernel: [201236.675314] RDX: ffffc90000e2e100 RSI: 0000000000400000 RDI: ffff880036f21548
Sep 17 02:35:27 #hostname# kernel: [201236.675315] RBP: 0000000000000000 R08: 0000000000000000 R09: 00000000000000b0
Sep 17 02:35:27 #hostname# kernel: [201236.675323] R10: ffff880098f64400 R11: 0000000000000000 R12: ffff880036f21e70
Sep 17 02:35:27 #hostname# kernel: [201236.675324] R13: ffff8802368d2098 R14: 0000000000000004 R15: ffff880036f21548
Sep 17 02:35:27 #hostname# kernel: [201236.675326] FS:  00007fb945ff8700(0000) GS:ffff88023fc80000(0000) knlGS:0000000000000000
Sep 17 02:35:27 #hostname# kernel: [201236.675327] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Sep 17 02:35:27 #hostname# kernel: [201236.675328] CR2: 00007f2bf344a000 CR3: 0000000233fb4000 CR4: 00000000000006e0
Sep 17 02:35:27 #hostname# kernel: [201236.675330] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 17 02:35:27 #hostname# kernel: [201236.675331] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Sep 17 02:35:27 #hostname# kernel: [201236.675332] Stack:
Sep 17 02:35:27 #hostname# kernel: [201236.675333]  ffff880036f20000 ffffffffa011d73c 0000000136f21548 ffff88019e9c9700
Sep 17 02:35:27 #hostname# kernel: [201236.675335]  ffffffff8182c960 0000000000000000 ffff880036f21548 ffff880036f20000
Sep 17 02:35:27 #hostname# kernel: [201236.675336]  0000000000000296 ffffffffa01232f0 ffff880036f222a0 ffffffffa01262bd
Sep 17 02:35:27 #hostname# kernel: [201236.675338] Call Trace:
Sep 17 02:35:27 #hostname# kernel: [201236.675345]  [<ffffffffa011d73c>] ? ata_qc_issue+0x16c/0x390 [libata]
Sep 17 02:35:27 #hostname# kernel: [201236.675351]  [<ffffffffa01232f0>] ? ata_scsi_invalid_field+0x40/0x40 [libata]
Sep 17 02:35:27 #hostname# kernel: [201236.675356]  [<ffffffffa01262bd>] ? ata_scsi_queuecmd+0x12d/0x410 [libata]
Sep 17 02:35:27 #hostname# kernel: [201236.675361]  [<ffffffffa00303e4>] ? scsi_dispatch_cmd+0xb4/0x2d0 [scsi_mod]
Sep 17 02:35:27 #hostname# kernel: [201236.675366]  [<ffffffffa0038c1d>] ? scsi_request_fn+0x2fd/0x500 [scsi_mod]
Sep 17 02:35:27 #hostname# kernel: [201236.675369]  [<ffffffff81280dbf>] ? __blk_run_queue+0x2f/0x40
Sep 17 02:35:27 #hostname# kernel: [201236.675371]  [<ffffffff81280e79>] ? queue_unplugged+0x29/0xc0
Sep 17 02:35:27 #hostname# kernel: [201236.675375]  [<ffffffff812851b7>] ? blk_flush_plug_list+0x1f7/0x230
Sep 17 02:35:27 #hostname# kernel: [201236.675377]  [<ffffffff812855a0>] ? blk_finish_plug+0x10/0x40
Sep 17 02:35:27 #hostname# kernel: [201236.675803]  [<ffffffffa01bb10e>] ? ext4_writepages+0x4ce/0xd30 [ext4]
Sep 17 02:35:27 #hostname# kernel: [201236.675828]  [<ffffffff81140b4e>] ? __filemap_fdatawrite_range+0x4e/0x60
Sep 17 02:35:27 #hostname# kernel: [201236.675830]  [<ffffffff81140c25>] ? filemap_write_and_wait_range+0x25/0x60
Sep 17 02:35:27 #hostname# kernel: [201236.675838]  [<ffffffffa01b25d1>] ? ext4_sync_file+0xb1/0x350 [ext4]
Sep 17 02:35:27 #hostname# kernel: [201236.675845]  [<ffffffff811d8dbb>] ? do_fsync+0x4b/0x70
Sep 17 02:35:27 #hostname# kernel: [201236.675848]  [<ffffffff811d902c>] ? SyS_fsync+0xc/0x10
Sep 17 02:35:27 #hostname# kernel: [201236.675850]  [<ffffffff8151a48d>] ? system_call_fast_compare_end+0x10/0x15
Sep 17 02:35:27 #hostname# kernel: [201236.675851] Code: f0 ff ff 48 8b 36 8b 76 08 c1 e6 08 09 f1 89 4a 40 48 8b 4f 08 48 8b 09 8b 49 08 89 48 4c 8b 4f 58 b8 01 00 00 00 d3 e0 89 42 38 <48> 8b 47 08 48 8b 00 f6 80 d0 02 00 00 80 48 8b 10 8b 48 08 48
Sep 17 02:35:27 #hostname# kernel: [201235.772094] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 1.230 msecs
Sep 17 02:35:27 #hostname# spamd[3402]: spamd: clean message (-1.9/7.0) for info@xxx:30 in 37.8 seconds, 4878 bytes.
Sep 17 02:35:27 #hostname# spamd[3402]: spamd: result: . -1 - BAYES_00,HEADER_FROM_DIFFERENT_DOMAINS,NO_RELAYS,URIBL_BLOCKED scantime=37.8,size=4878,user=info@xxx,uid=30,required_score=7.0,rhost=#hostname#.de,raddr=::1,rport=56932,mid=<20170917003450.1F8084C5908@#hostname#.de>,bayes=0.000000,autolearn=ham autolearn_force=no
Sep 17 02:35:28 #hostname# spamd[1170]: prefork: child states: II
����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������Sep 17 09:54:13 #hostname# rsyslogd: [origin software="rsyslogd" swVersion="8.4.2" x-pid="836" x-info="http://www.rsyslog.com"] start
Sep 17 09:54:13 #hostname# kernel: [    0.000000] Initializing cgroup subsys cpuset
Sep 17 09:54:13 #hostname# kernel: [    0.000000] Initializing cgroup subsys cpu
Sep 17 09:54:13 #hostname# kernel: [    0.000000] Initializing cgroup subsys cpuacct
Sep 17 09:54:13 #hostname# kernel: [    0.000000] Linux version 3.16.0-4-amd64 (debian-kernel@lists.debian.org) (gcc version 4.8.4 (Debian 4.8.4-1) ) #1 SMP Debian 3.16.43-2+deb8u3 (2017-08-15)
Sep 17 09:54:13 #hostname# kernel: [    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-3.16.0-4-amd64 root=UUID=a2b74f3f-670d-406c-9831-53dd6fa25e94 ro quiet
Sep 17 09:54:13 #hostname# kernel: [    0.000000] e820: BIOS-provided physical RAM map:
Sep 17 09:54:13 #hostname# kernel: [    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
Sep 17 09:54:13 #hostname# kernel: [    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
Sep 17 09:54:13 #hostname# kernel: [    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
Sep 17 09:54:13 #hostname# kernel: [    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd9fff] usable
Sep 17 09:54:13 #hostname# kernel: [    0.000000] BIOS-e820: [mem 0x00000000bffda000-0x00000000bfffffff] reserved
Sep 17 09:54:13 #hostname# kernel: [    0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
Sep 17 09:54:13 #hostname# kernel: [    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
Sep 17 09:54:13 #hostname# kernel: [    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000023fffffff] usable
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!