Issue // Elastix VM Auto Shutdown

O

ors_86

New Member
Nov 15, 2011
16
0
1
Hi,

I am facing some issues with my proxmox production installation.

Let me explain.
Before 3 days, i found my elastix vm not working (after 3 months online), and i reset it.
Then, i turned it on, and get a kernel panic. I have restored a backup from Sunday, and everything seems to be working fine, expect i found it "shut down" every morning, 3 days now. I turned it on and everthing works fine.
Where i can find logs in order to replicate and find the issue?

Onother issue, older that this, is the "IO APIC" problem. I have noticed at my server 2003 that time change about 1.5 hours forward or backward.
Is there any way to fix that?

I would appreciate any help because this is a production installation.

Thanks in advance,
Orestis
 
I have done many attempts to replicate that issue, and i noticed that if any backup jod run, the virtueal elastix machine is okay.
When i run a backup jod, after it complets then the virtueal machine is at offine state.
Can anyone inform me for that issue?

Thanks
 
post the full backup log.
 
Here is the log, please inform what whats is wrong.

Nov 19 15:40:01 INFO: Starting Backup of VM 301 (qemu)
Nov 19 15:40:01 INFO: running
Nov 19 15:40:01 INFO: status = running
Nov 19 15:40:06 INFO: mode failure - unable to detect lvm volume group
Nov 19 15:40:06 INFO: trying 'suspend' mode instead
Nov 19 15:40:06 INFO: backup mode: suspend
Nov 19 15:40:06 INFO: ionice priority: 7
Nov 19 15:40:06 INFO: suspend vm
Nov 19 15:40:06 INFO: creating archive '/media/samba/DiskB2000/Backup/Proxmox//$
Nov 19 15:40:06 INFO: adding '/media/samba/DiskB2000/Backup/Proxmox//vzdump-qem$
Nov 19 15:40:06 INFO: adding '/media/samba/DiskA1000/proxmox//images/301/vm-dis$
Nov 19 15:44:20 INFO: Total bytes written: 2665984512 (10.01 MiB/s)
Nov 19 15:44:20 INFO: archive file size: 1013MB
Nov 19 15:44:20 INFO: delete old backup '/media/samba/DiskB2000/Backup/Proxmox/$
Nov 19 15:44:21 INFO: resume vm
Nov 19 15:44:21 INFO: vm is online again after 255 seconds
Nov 19 15:44:21 INFO: Finished Backup of VM 301 (00:04:20)
 
Last edited:
you try to do a online backup with snapshots but you got not the right LVM setup.

post the output of the command 'pvs'.
 
The strange this is that this issue happend from nowhere.
I mean that everthing was working smooth for about 6 months now.

Here is the output of the pvs command:

proxmox:~# pvs
/dev/dm-5: read failed after 0 of 4096 at 0: Input/output error
PV VG Fmt Attr PSize PFree
/dev/sda2 pve lvm2 a- 297.59G 2.99G
 
ors_86 said:
The strange this is that this issue happend from nowhere.
I mean that everthing was working smooth for about 6 months now.

Here is the output of the pvs command:

proxmox:~# pvs
/dev/dm-5: read failed after 0 of 4096 at 0: Input/output error
PV VG Fmt Attr PSize PFree
/dev/sda2 pve lvm2 a- 297.59G 2.99G
Click to expand...
Hi,
one storage is gone (dm-5).
Look with
Code: 
dmsetup info
The one with the minor-number 5 ist the failed one.

Udo
 
Hi,

Thanks for the reply.
Im sorry but i didn't understand what i should do.

The proxmox installation is at a 250GB hdd. What i should do?

Thanks in advance,
Orestis
 
Thanks for the quick reply.

I have run that command, and found that with minor number 5, is was the first.
proxmox:~# dmsetup info
Name: pve-vzsnap--proxmox--0
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 251, 5
Number of targets: 1
UUID: LVM-YBZZqvIbjd7DGF1FVlEmHDislBOtfG6BhqQNczoORRDhu89cFhUWR1TpxNzTfx02

Name: pve-data-real
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 2
Event number: 0
Major, minor: 251, 2
Number of targets: 1
UUID: LVM-YBZZqvIbjd7DGF1FVlEmHDislBOtfG6B4DvoL78SRcz4bU66f8K8WnqKlT38je2U-real

Name: pve-swap
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 251, 0
Number of targets: 1
UUID: LVM-YBZZqvIbjd7DGF1FVlEmHDislBOtfG6B2XLYoXoOE9h48ltHr0iUwpxoarheXuY1

Name: pve-root
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 251, 1
Number of targets: 1
UUID: LVM-YBZZqvIbjd7DGF1FVlEmHDislBOtfG6B7Vnd1Yhok3OGxXUwT0IJBPyaXO47xQV2

Name: pve-data
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 251, 3
Number of targets: 1
UUID: LVM-YBZZqvIbjd7DGF1FVlEmHDislBOtfG6B4DvoL78SRcz4bU66f8K8WnqKlT38je2U

Name: pve-vzsnap--proxmox--0-cow
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 251, 4
Number of targets: 1
UUID: LVM-YBZZqvIbjd7DGF1FVlEmHDislBOtfG6BhqQNczoORRDhu89cFhUWR1TpxNzTfx02-cow
Click to expand...

How can we fix that?
 
e100 said:
That first entry "pve-vzsnap--proxmox--0" is likely a snapshot that is left over from a previous failed backup.
Any idea what "pve-data-real" is? I've not seen that before.

Gathering more info would be helpful in solving this.
What is the output of:
Code: 
lvscan
and:
Code: 
mount
Click to expand...
Hi Orestis,
you have also an snapshot of an qcow-file!? pve-vzsnap--proxmox--0-cow

I guess you don't have any free space in the VG due the snapshots. Look with "vgdisplay" for free extends, and remove your snapshots.

Udo
 
Hi e100 and Udo,

Thanks for the quick replys both of you. I am not very familiar with proxmox, so i need your help.

Here are the outputs of the commants you mentioned to me:


proxmox:~# lvscan
/dev/dm-5: read failed after 0 of 4096 at 0: Input/output error
ACTIVE '/dev/pve/swap' [4.00 GB] inherit
ACTIVE '/dev/pve/root' [74.50 GB] inherit
inactive Original '/dev/pve/data' [215.09 GB] inherit
inactive Snapshot '/dev/pve/vzsnap-proxmox-0' [1.00 GB] inherit



proxmox:~# mount
/dev/mapper/pve-root on / type ext3 (rw,errors=remount-ro)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
/dev/mapper/pve-data on /var/lib/vz type ext3 (rw)
/dev/sda1 on /boot type ext3 (rw)
/dev/sdd1 on /media/samba/DiskA1000 type ext3 (rw)
/dev/sdc1 on /media/samba/DiskB2000 type ext3 (rw)
/dev/sdb1 on /media/samba/DiskA2000 type fuseblk (rw,allow_other,blksize=4096)


Also, I must search for a qcow-file and delete it, right?

Thanks again,
Orestis
 
First thing you should do is ensure you have a working backup of your VMs

proxmox:~# lvscan
/dev/dm-5: read failed after 0 of 4096 at 0: Input/output error
inactive Snapshot '/dev/pve/vzsnap-proxmox-0' [1.00 GB] inherit

To remove the snapshot:
Code: 
lvremove /dev/pve/vzsnap-proxmox-0

What is the output of:
Code: 
ls -l /dev/mapper
 
I have run the command:
Code: 
lvremove /dev/pve/vzsnap-proxmox-0

And removed the snapshot. The virtueal machines are still working fine.

Then i run the command you mentioned to me, and the output is the following:
Code: 
proxmox:~# ls -l /dev/mapper
total 0
crw-rw---- 1 root root  10, 59 Nov 16 12:53 control
brw-rw---- 1 root disk 251,  3 Nov 16 12:53 pve-data
brw-rw---- 1 root disk 251,  1 Nov 16 12:53 pve-root
brw-rw---- 1 root disk 251,  0 Nov 16 12:53 pve-swap

What we should do next?

Thanks again,
Orestis
 
I run again the command "lvscan" and the output is the following:
proxmox:/var/lib# lvscan
ACTIVE '/dev/pve/swap' [4.00 GB] inherit
ACTIVE '/dev/pve/root' [74.50 GB] inherit
ACTIVE '/dev/pve/data' [215.09 GB] inherit
 
This is the new output:
Code: 
proxmox:~# dmsetup info
Name:              pve-swap
State:             ACTIVE
Read Ahead:        256
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      251, 0
Number of targets: 1
UUID: LVM-YBZZqvIbjd7DGF1FVlEmHDislBOtfG6B2XLYoXoOE9h48ltHr0iUwpxoarheXuY1

Name:              pve-root
State:             ACTIVE
Read Ahead:        256
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      251, 1
Number of targets: 1
UUID: LVM-YBZZqvIbjd7DGF1FVlEmHDislBOtfG6B7Vnd1Yhok3OGxXUwT0IJBPyaXO47xQV2

Name:              pve-data
State:             ACTIVE
Read Ahead:        256
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      251, 3
Number of targets: 1
UUID: LVM-YBZZqvIbjd7DGF1FVlEmHDislBOtfG6B4DvoL78SRcz4bU66f8K8WnqKlT38je2U

I run again a snapshot backup, but the result was the same, after it compets, the virtual machine turned off.
 
You must log in or register to reply here.
Top Bottom