[SOLVED] Proxmox disconnecting from itself

ConnorMcF

Member
Feb 12, 2018
5
1
8
Hi there, my Proxmox instance keeps disconnecting from itself (although it isn't the red icon, unknown?). It is setup as a single instance, not a cluster.

1585410172693.png

I've tried restarting services but the command just hangs when I attempt it in this state. A reboot resolves the problem temporarily but it occurs again a few hours later.

If there are any log files that could help figure this out, please let me know.

Best,
Connor
 
* Check the system's journal (`journalctl -r` for the journal in reverse order)
* Is pvestatd running and without errors?

I hope this helps!
 
Hi, thanks for the reply! I had to wait a bit for this to happen again as I'd rebooted and I don't seem to see anything in particular in the logs.

Code:
● pvestatd.service - PVE Status Daemon
   Loaded: loaded (/lib/systemd/system/pvestatd.service; enabled; vendor preset: enabled)
   Active: active (running) since Sat 2020-03-28 16:54:47 GMT; 2 days ago
  Process: 1492 ExecStart=/usr/bin/pvestatd start (code=exited, status=0/SUCCESS)
 Main PID: 1506 (pvestatd)
    Tasks: 2 (limit: 4915)
   Memory: 99.0M
   CGroup: /system.slice/pvestatd.service
           ├─1506 pvestatd
           └─9225 /sbin/vgs --separator : --noheadings --units b --unbuffered --nosuffix --options vg_name,vg_size,vg_free,lv_count

The only thing I see in the logs around the time of when this happened was this, it seemed to happen when I tried to add an external IP to a VM (possibly relevant?).

Code:
Mar 31 01:19:35 ix1vm01 pveproxy[1535]: worker 24928 started
Mar 31 01:19:35 ix1vm01 pveproxy[1535]: starting 1 worker(s)
Mar 31 01:19:35 ix1vm01 pveproxy[1535]: worker 19394 finished
Mar 31 01:19:35 ix1vm01 pveproxy[19394]: worker exit

Best,
Connor
 
hmm - on a hunch - if it happens again - how long does the following command take:
Code:
/sbin/vgs --separator : --noheadings --units b --unbuffered --nosuffix --options vg_name,vg_size,vg_free,lv_count

might be that the vgs command hangs and this keeps pvestatd from updating everything else.
 
  • Like
Reactions: ConnorMcF
Yeah, looks like the command just hangs indefinitely on /dev/sdb.

Code:
stat("/dev/loop6", {st_mode=S_IFBLK|0660, st_rdev=makedev(0x7, 0x6), ...}) = 0
openat(AT_FDCWD, "/dev/loop6", O_RDONLY|O_DIRECT|O_NOATIME) = 3
fstat(3, {st_mode=S_IFBLK|0660, st_rdev=makedev(0x7, 0x6), ...}) = 0
ioctl(3, BLKGETSIZE64, [0])             = 0
close(3)                                = 0
stat("/dev/loop7", {st_mode=S_IFBLK|0660, st_rdev=makedev(0x7, 0x7), ...}) = 0
openat(AT_FDCWD, "/dev/loop7", O_RDONLY|O_DIRECT|O_NOATIME) = 3
fstat(3, {st_mode=S_IFBLK|0660, st_rdev=makedev(0x7, 0x7), ...}) = 0
ioctl(3, BLKGETSIZE64, [0])             = 0
close(3)                                = 0
stat("/dev/sdb", {st_mode=S_IFBLK|0660, st_rdev=makedev(0x8, 0x10), ...}) = 0
openat(AT_FDCWD, "/dev/sdb", O_RDONLY|O_DIRECT|O_NOATIME
 
hmm - that's odd - any messages in `dmesg`?
does running `udevadm trigger` change anything?
 
I don't see anything in dmesg and udevadm trigger didn't seem to help.

I've realised what /dev/sdb is though, it's a little USB stick which is probably dead or something. Perhaps it's trying to query it but it's just too far gone which is causing it to hang?

Code:
root@ix1vm01:~# udevadm info --query=all -n /dev/sdb | grep ID_BUS
E: ID_BUS=usb
 
hmm - do you have anything important on that stick?
otherwise if you have physical access to the box you could simply unplug it
if you don't have access you can try to exclude it from being checked by the lvm-utils - edit '/etc/lvm/lvm.conf' (look for global_filter (and the explanation in the lines above it)

I hope this helps!
 
  • Like
Reactions: ConnorMcF

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!