pvestatd Crash - Where to get more information?

T

TheReelaatiiv

Member
Mar 29, 2012
89
0
6
Good evening (it's 10:25 pm in austria),

pvestatd suddenly sayed goodbye to my server:

root@node1 /etc/init.d # ./pvestatd restart
Restarting PVE Status Daemon: pvestatdstart-stop-daemon: warning: failed to kill 9805: No such process
.

Where can i get additional information why pvestatd has suddenly crashed?
 
AW: pvestatd Crash - Where to get more information?

upgrade to latest. AFAIK this is fixed already.
 
Re: AW: pvestatd Crash - Where to get more information?

tom said:
upgrade to latest. AFAIK this is fixed already.
Click to expand...

Noticed a there were 3 or 4 updates since last week and was just looking around hoping to find similar issue. Hope this helps as pvestatd seemed to become unstable since last weeks update.

Thanks!
 
Re: AW: pvestatd Crash - Where to get more information?

no, afaik its fixed. what version do you run? post your 'pveversion -v'
 
Re: AW: pvestatd Crash - Where to get more information?

tom said:
no, afaik its fixed. what version do you run? post your 'pveversion -v'
Click to expand...

Mine is currently running after update this am

Code: 
pve-manager: 2.2-26 (pve-manager/2.2/c1614c8c)
running kernel: 2.6.32-16-pve
proxmox-ve-2.6.32: 2.2-80
pve-kernel-2.6.32-11-pve: 2.6.32-66
pve-kernel-2.6.32-16-pve: 2.6.32-80
pve-kernel-2.6.32-14-pve: 2.6.32-74
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.4-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.93-1
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.9-1
pve-cluster: 1.0-28
qemu-server: 2.0-64
pve-firmware: 1.0-21
libpve-common-perl: 1.0-37
libpve-access-control: 1.0-25
libpve-storage-perl: 2.0-34
vncterm: 1.0-3
vzctl: 4.0-1pve2
vzprocps: 2.0.11-2
vzquota: 3.1-1
pve-qemu-kvm: 1.2-7
ksm-control-daemon: 1.1-1
 
Re: AW: pvestatd Crash - Where to get more information?

and you have still issues?
 
Re: AW: pvestatd Crash - Where to get more information?

Me, too soon. VMs would go missing at random on each node. Guessing I will have some idea over the next few days.
 
Re: AW: pvestatd Crash - Where to get more information?

pvestatd stills keeps crashing on one node:

Code: 
pve-manager: 2.2-32 (pve-manager/2.2/3089a616)
running kernel: 2.6.32-17-pve
proxmox-ve-2.6.32: 2.2-83
pve-kernel-2.6.32-17-pve: 2.6.32-83
pve-kernel-2.6.32-16-pve: 2.6.32-82
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.4-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.93-2
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.9-1
pve-cluster: 1.0-34
qemu-server: 2.0-71
pve-firmware: 1.0-21
libpve-common-perl: 1.0-41
libpve-access-control: 1.0-25
libpve-storage-perl: 2.0-36
vncterm: 1.0-3
vzctl: 4.0-1pve2
vzprocps: 2.0.11-2
vzquota: 3.1-1
pve-qemu-kvm: 1.3-10
ksm-control-daemon: 1.1-1

On another node it still runs without any problems (exactly same versions)
 
Since my last post, pvestatd crashed about 15 times while it still runs on another node like a charm... Now i started strace in screen to find out why it keeps crashing and surely i'll post the output here...
 
At the logging-eof there is a segmentation fault...

I am going to reset the server in a few days. Hopefully that fixes the problem ...

PS: The only difference is that in the server which's pvestatd crashed periodically is a hardware raid controller (Adaptec 6405E)
 
Last edited:
Also experienced pvestatd not running today. Looked at syslog and it crashed approx 2 hours into the backup last night. We are also running an adaptec raid controller (different model):

TheReelaatiiv said:
PS: The only difference is that in the server which's pvestatd crashed periodically is a hardware raid controller (Adaptec 6405E)
Click to expand...

Not saying that has anything to do w/ this. Version 2.2 totally up to date (just did an apt-get update and nothing changed.

pvestatd in syslog appears to attempt to start at 20:51 yesterday, but I don't think it actually did (since that is when the data stops in the gui graphs), then again at 22:07 and then pvestatd does not appear again until I started it today.

Code: 
Jan 23 20:51:35 penny pvestatd[1641]: restarting server after 457 cycles to reduce memory usage (free 36708352 bytes)
Jan 23 20:51:35 penny pvestatd[1641]: server shutdown (restart)
Jan 23 20:51:36 penny pvestatd[1641]: restarting server
Jan 23 21:17:01 penny /USR/SBIN/CRON[379943]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Jan 23 21:45:46 penny rrdcached[1396]: flushing old values
Jan 23 21:45:46 penny rrdcached[1396]: rotating journals
Jan 23 21:45:46 penny rrdcached[1396]: started new journal /var/lib/rrdcached/journal//rrd.journal.1358995546.675880
Jan 23 21:45:46 penny rrdcached[1396]: removing old journal /var/lib/rrdcached/journal//rrd.journal.1358988346.675985
Jan 23 22:07:36 penny pvestatd[1641]: restarting server after 457 cycles to reduce memory usage (free 36708352 bytes)
Jan 23 22:07:36 penny pvestatd[1641]: server shutdown (restart)
Jan 23 22:12:14 penny vzdump[377559]: INFO: Finished Backup of VM 101 (01:36:43)
Jan 23 22:12:14 penny vzdump[377559]: INFO: Starting Backup of VM 102 (qemu)
Jan 23 22:12:14 penny qm[381581]: <root@pam> update VM 102: -lock backup

pveversion info:
Code: 
# pveversion -v
pve-manager: 2.2-24 (pve-manager/2.2/7f9cfa4c)
running kernel: 2.6.32-16-pve
proxmox-ve-2.6.32: 2.2-80
pve-kernel-2.6.32-16-pve: 2.6.32-80
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.4-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.93-1
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.9-1
pve-cluster: 1.0-28
qemu-server: 2.0-62
pve-firmware: 1.0-21
libpve-common-perl: 1.0-36
libpve-access-control: 1.0-25
libpve-storage-perl: 2.0-34
vncterm: 1.0-3
vzctl: 4.0-1pve2
vzprocps: 2.0.11-2
vzquota: 3.1-1
pve-qemu-kvm: 1.2-7
ksm-control-daemon: 1.1-1
 
TheReelaatiiv said:
I can't:
View attachment 1260
Click to expand...

run 'apt-get update && apt-get dist-upgrade'
see also http://pve.proxmox.com/wiki/Downloads#Update_a_running_Proxmox_Virtual_Environment_2.x_to_latest_2.2

additionally, you mentioned earlier that you run adaptec 6 controller. some user reported issues which seems to be fixed with newer firmware and linux drivers (you can update them manually by downloading from adaptec website).
as a reference, see https://bugzilla.proxmox.com/show_bug.cgi?id=320
 
Still only fail2ban and munin to upgrade:

Code: 
root@bedrock ~ # apt-get dist-upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
  fail2ban munin-common munin-node munin-plugins-core munin-plugins-extra
5 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 783 kB of archives.
After this operation, 20.5 kB of additional disk space will be used.
Do you want to continue [Y/n]? ^C
 
post your /etc/apt/sources.list and the result of 'apt-get update'

and finally again your 'pveversion -v'
 
Code: 
deb http://ftp.de.debian.org/debian/ squeeze main non-free contrib
deb-src http://ftp.de.debian.org/debian/ squeeze main non-free contrib

deb http://security.debian.org/ squeeze/updates main non-free contrib
deb-src http://security.debian.org/ squeeze/updates main non-free contrib

deb http://ftp.de.debian.org/debian/ squeeze-updates main non-free contrib
deb-src http://ftp.de.debian.org/debian/ squeeze-updates main non-free contrib

deb http://packages.dotdeb.org squeeze all
deb-src http://packages.dotdeb.org squeeze all

deb http://packages.dotdeb.org squeeze-php54 all
deb-src http://packages.dotdeb.org squeeze-php54 all

deb http://download.proxmox.com/debian squeeze pve

deb http://backports.debian.org/debian-backports squeeze-backports main

Code: 
root@bedrock /usr/share/perl5/PVE # apt-get update
Hit http://security.debian.org squeeze/updates Release.gpg
Ign http://security.debian.org/ squeeze/updates/contrib Translation-en                                                                                                               
Ign http://security.debian.org/ squeeze/updates/contrib Translation-en_US                                                                                                            
Ign http://security.debian.org/ squeeze/updates/main Translation-en                                                                                                                  
Ign http://security.debian.org/ squeeze/updates/main Translation-en_US                                                                                                               
Ign http://security.debian.org/ squeeze/updates/non-free Translation-en                                                                                                              
Ign http://security.debian.org/ squeeze/updates/non-free Translation-en_US                                                                                                           
Hit http://security.debian.org squeeze/updates Release                                                                                                                                
Hit http://security.debian.org squeeze/updates/main Sources                                                                                                                            
Hit http://security.debian.org squeeze/updates/non-free Sources                                                                                                                   
Hit http://security.debian.org squeeze/updates/contrib Sources                                                                                           
Hit http://security.debian.org squeeze/updates/main amd64 Packages                                                                                       
Hit http://security.debian.org squeeze/updates/non-free amd64 Packages                                                                                                         
Hit http://security.debian.org squeeze/updates/contrib amd64 Packages                                                                                                          
Hit http://download.proxmox.com squeeze Release.gpg                                                                                                      
Ign http://download.proxmox.com/debian/ squeeze/pve Translation-en                                                                                       
Ign http://download.proxmox.com/debian/ squeeze/pve Translation-en_US                                                   
Hit http://ftp.de.debian.org squeeze Release.gpg                                                                        
Ign http://ftp.de.debian.org/debian/ squeeze/contrib Translation-en                               
Hit http://download.proxmox.com squeeze Release                                                   
Ign http://download.proxmox.com squeeze/pve amd64 Packages                                                               
Ign http://ftp.de.debian.org/debian/ squeeze/contrib Translation-en_US                                                              
Ign http://ftp.de.debian.org/debian/ squeeze/main Translation-en                                                                    
Ign http://ftp.de.debian.org/debian/ squeeze/main Translation-en_US                                                                 
Ign http://ftp.de.debian.org/debian/ squeeze/non-free Translation-en                                                                
Ign http://ftp.de.debian.org/debian/ squeeze/non-free Translation-en_US                                                             
Hit http://download.proxmox.com squeeze/pve amd64 Packages                                                                          
Hit http://ftp.de.debian.org squeeze-updates Release.gpg                                                                            
Ign http://ftp.de.debian.org/debian/ squeeze-updates/contrib Translation-en                                   
Ign http://ftp.de.debian.org/debian/ squeeze-updates/contrib Translation-en_US     
Ign http://ftp.de.debian.org/debian/ squeeze-updates/main Translation-en           
Hit http://backports.debian.org squeeze-backports Release.gpg                      
Ign http://backports.debian.org/debian-backports/ squeeze-backports/main Translation-en
Ign http://backports.debian.org/debian-backports/ squeeze-backports/main Translation-en_US
Ign http://ftp.de.debian.org/debian/ squeeze-updates/main Translation-en_US        
Ign http://ftp.de.debian.org/debian/ squeeze-updates/non-free Translation-en
Ign http://ftp.de.debian.org/debian/ squeeze-updates/non-free Translation-en_US
Hit http://ftp.de.debian.org squeeze Release                 
Hit http://backports.debian.org squeeze-backports Release                                                 
Hit http://ftp.de.debian.org squeeze-updates Release                                
Hit http://backports.debian.org squeeze-backports/main amd64 Packages/DiffIndex    
Hit http://ftp.de.debian.org squeeze/main Sources                                  
Hit http://ftp.de.debian.org squeeze/non-free Sources        
Hit http://ftp.de.debian.org squeeze/contrib Sources         
Hit http://backports.debian.org squeeze-backports/main amd64 Packages
Hit http://ftp.de.debian.org squeeze/main amd64 Packages     
Hit http://ftp.de.debian.org squeeze/non-free amd64 Packages 
Hit http://ftp.de.debian.org squeeze/contrib amd64 Packages
Hit http://ftp.de.debian.org squeeze-updates/main Sources/DiffIndex
Hit http://ftp.de.debian.org squeeze-updates/non-free Sources
Hit http://ftp.de.debian.org squeeze-updates/contrib Sources 
Hit http://ftp.de.debian.org squeeze-updates/main amd64 Packages/DiffIndex
Hit http://ftp.de.debian.org squeeze-updates/non-free amd64 Packages
Hit http://ftp.de.debian.org squeeze-updates/contrib amd64 Packages
Hit http://ftp.de.debian.org squeeze-updates/main amd64 Packages
Hit http://packages.dotdeb.org squeeze Release.gpg     
Ign http://packages.dotdeb.org/ squeeze/all Translation-en
Ign http://packages.dotdeb.org/ squeeze/all Translation-en_US
Hit http://packages.dotdeb.org squeeze-php54 Release.gpg
Ign http://packages.dotdeb.org/ squeeze-php54/all Translation-en
Ign http://packages.dotdeb.org/ squeeze-php54/all Translation-en_US
Hit http://packages.dotdeb.org squeeze Release
Hit http://packages.dotdeb.org squeeze-php54 Release
Hit http://packages.dotdeb.org squeeze/all Sources
Hit http://packages.dotdeb.org squeeze/all amd64 Packages
Hit http://packages.dotdeb.org squeeze-php54/all Sources
Hit http://packages.dotdeb.org squeeze-php54/all amd64 Packages
Reading package lists... Done

Code: 
root@bedrock /usr/share/perl5/PVE # pveversion -v
pve-manager: 2.2-32 (pve-manager/2.2/3089a616)
running kernel: 2.6.32-17-pve
proxmox-ve-2.6.32: 2.2-83
pve-kernel-2.6.32-17-pve: 2.6.32-83
pve-kernel-2.6.32-16-pve: 2.6.32-82
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.4-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.93-2
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.9-1
pve-cluster: 1.0-34
qemu-server: 2.0-72
pve-firmware: 1.0-21
libpve-common-perl: 1.0-41
libpve-access-control: 1.0-25
libpve-storage-perl: 2.0-36
vncterm: 1.0-3
vzctl: 4.0-1pve2
vzprocps: 2.0.11-2
vzquota: 3.1-1
pve-qemu-kvm: 1.3-10
ksm-control-daemon: 1.1-1
 
You must log in or register to reply here.
Top Bottom