Proxmox web gui not working(and more); VM works

jaceqp

Well-Known Member
May 28, 2018
95
7
48
44
Hi there...
I've installed Proxmox with ZFS on an old server (1x 4core cpu, 12GB RAM, 2x HDD SATA 1TB).
I've installed OpenMediaVault on it as a single VM.

Now after... lets say... 2 months I noticed some weired behavior in gui. Everything worked fine except missing icon in left panel VM/storage list (exclamation mark icons instead of default ones).
Now, after 2-3 days I can no longer access to PM web qui (page not found).
Funny (or not) is that VM itself still works fine. After a reboot the problem remains.
At least I can still remote access via SHH command line...

But there's more, I tried to do some troubleshooting but for example executing 'systemctl' shows:
Code:
# systemctl
Failed to list units: Connection timed out

By the way, version:
Code:
# pveversion
pve-manager/5.2-1/0fcd7879 (running kernel: 4.15.17-1-pve)

I wasn't doing any modifications for weeks. No updates what so ever...

Any ideas?
 
does dmesg/syslog say anything? maybe a broken disk ?

if even systemctl fails, i am not sure what can be done...
 
If you still can get to it via ssh.
As fast as you can get the all config out. Do a vm backup and get it off the machine.
I lost whole setup like this.
Depending on your hardware it can be your psu
 
I've checked ZFS status (seems fine):
Code:
# zpool list
NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
rpool   928G   184G   744G         -    22%    19%  1.00x  ONLINE  -

# zpool status
  pool: rpool
 state: ONLINE
  scan: scrub repaired 0B in 1h11m with 0 errors on Sun Dec  9 01:35:56 2018
config:

        NAME        STATE     READ WRITE CKSUM
        rpool       ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            sda2    ONLINE       0     0     0
            sdb2    ONLINE       0     0     0

errors: No known data errors

I'm also using ZFS filesystem snapshot solution to make full/differential disk backups:

Code:
# zfs get written
NAME                                                               PROPERTY  VAL                UE    SOURCE
rpool                                                              written   0                        -
rpool@zfs-auto-snap_monthly-2018-11-01-0552                        written   128                K     -
rpool@zfs-auto-snap_monthly-2018-12-01-0552                        written   0                        -
rpool@zfs-auto-snap_daily-2018-12-03-0525                          written   0                        -
rpool@zfs-auto-snap_daily-2018-12-04-0525                          written   0                        -
rpool@zfs-auto-snap_daily-2018-12-05-0525                          written   0                        -
rpool@zfs-auto-snap_daily-2018-12-06-0525                          written   0                        -
rpool@zfs-auto-snap_daily-2018-12-07-0525                          written   0                        -
rpool@zfs-auto-snap_daily-2018-12-08-0525                          written   0                        -
rpool@zfs-auto-snap_daily-2018-12-09-0525                          written   0                        -
rpool@zfs-auto-snap_daily-2018-12-10-0525                          written   0                        -
rpool@zfs-auto-snap_daily-2018-12-11-0525                          written   0                        -
rpool@zfs-auto-snap_daily-2018-12-12-0525                          written   0                        -
rpool@zfs-auto-snap_daily-2018-12-13-0525                          written   0                        -
rpool@zfs-auto-snap_daily-2018-12-14-0525                          written   0                        -
rpool@zfs-auto-snap_daily-2018-12-15-0525                          written   0                        -
rpool@zfs-auto-snap_daily-2018-12-16-0525                          written   0                        -
rpool/ROOT                                                         written   96K                      -
rpool/ROOT/pve-1                                                   written   5.1                4M    -
rpool/ROOT/pve-1@zfs-auto-snap_monthly-2018-11-01-0552             written   6.3                7G    -
rpool/ROOT/pve-1@zfs-auto-snap_monthly-2018-12-01-0552             written   6.1                3M    -
rpool/ROOT/pve-1@zfs-auto-snap_daily-2018-12-03-0525               written   4.9                8M    -
rpool/ROOT/pve-1@zfs-auto-snap_daily-2018-12-04-0525               written   2.8                1M    -
rpool/ROOT/pve-1@zfs-auto-snap_daily-2018-12-05-0525               written   3.0                4M    -
rpool/ROOT/pve-1@zfs-auto-snap_daily-2018-12-06-0525               written   2.6                0M    -
rpool/ROOT/pve-1@zfs-auto-snap_daily-2018-12-07-0525               written   3.0                9M    -
rpool/ROOT/pve-1@zfs-auto-snap_daily-2018-12-08-0525               written   3.1                1M    -
rpool/ROOT/pve-1@zfs-auto-snap_daily-2018-12-09-0525               written   3.1                7M    -
rpool/ROOT/pve-1@zfs-auto-snap_daily-2018-12-10-0525               written   4.6                4M    -
rpool/ROOT/pve-1@zfs-auto-snap_daily-2018-12-11-0525               written   2.9                8M    -
rpool/ROOT/pve-1@zfs-auto-snap_daily-2018-12-12-0525               written   5.4                9M    -
rpool/ROOT/pve-1@zfs-auto-snap_daily-2018-12-13-0525               written   2.3                0M    -
rpool/ROOT/pve-1@zfs-auto-snap_daily-2018-12-14-0525               written   3.1                3M    -
rpool/ROOT/pve-1@zfs-auto-snap_daily-2018-12-15-0525               written   3.0                9M    -
rpool/ROOT/pve-1@zfs-auto-snap_daily-2018-12-16-0525               written   3.2                9M    -
rpool/data                                                         written   0                        -
rpool/data@zfs-auto-snap_monthly-2018-11-01-0552                   written   96K                      -
rpool/data@zfs-auto-snap_monthly-2018-12-01-0552                   written   0                        -
rpool/data@zfs-auto-snap_daily-2018-12-03-0525                     written   0                        -
rpool/data@zfs-auto-snap_daily-2018-12-04-0525                     written   0                        -
rpool/data@zfs-auto-snap_daily-2018-12-05-0525                     written   0                        -
rpool/data@zfs-auto-snap_daily-2018-12-06-0525                     written   0                        -
rpool/data@zfs-auto-snap_daily-2018-12-07-0525                     written   0                        -
rpool/data@zfs-auto-snap_daily-2018-12-08-0525                     written   0                        -
rpool/data@zfs-auto-snap_daily-2018-12-09-0525                     written   0                        -
rpool/data@zfs-auto-snap_daily-2018-12-10-0525                     written   0                        -
rpool/data@zfs-auto-snap_daily-2018-12-11-0525                     written   0                        -
rpool/data@zfs-auto-snap_daily-2018-12-12-0525                     written   0                        -
rpool/data@zfs-auto-snap_daily-2018-12-13-0525                     written   0                        -
rpool/data@zfs-auto-snap_daily-2018-12-14-0525                     written   0                        -
rpool/data@zfs-auto-snap_daily-2018-12-15-0525                     written   0                        -
rpool/data@zfs-auto-snap_daily-2018-12-16-0525                     written   0                        -
rpool/disks                                                        written   0                        -
rpool/disks@zfs-auto-snap_monthly-2018-11-01-0552                  written   96K                      -
rpool/disks@zfs-auto-snap_monthly-2018-12-01-0552                  written   0                        -
rpool/disks@zfs-auto-snap_daily-2018-12-03-0525                    written   0                        -
rpool/disks@zfs-auto-snap_daily-2018-12-04-0525                    written   0                        -
rpool/disks@zfs-auto-snap_daily-2018-12-05-0525                    written   0                        -
rpool/disks@zfs-auto-snap_daily-2018-12-06-0525                    written   0                        -
rpool/disks@zfs-auto-snap_daily-2018-12-07-0525                    written   0                        -
rpool/disks@zfs-auto-snap_daily-2018-12-08-0525                    written   0                        -
rpool/disks@zfs-auto-snap_daily-2018-12-09-0525                    written   0                        -
rpool/disks@zfs-auto-snap_daily-2018-12-10-0525                    written   0                        -
rpool/disks@zfs-auto-snap_daily-2018-12-11-0525                    written   0                        -
rpool/disks@zfs-auto-snap_daily-2018-12-12-0525                    written   0                        -
rpool/disks@zfs-auto-snap_daily-2018-12-13-0525                    written   0                        -
rpool/disks@zfs-auto-snap_daily-2018-12-14-0525                    written   0                        -
rpool/disks@zfs-auto-snap_daily-2018-12-15-0525                    written   0                        -
rpool/disks@zfs-auto-snap_daily-2018-12-16-0525                    written   0                        -
rpool/disks/vm-100-disk-1                                          written   589                M     -
rpool/disks/vm-100-disk-1@zfs-auto-snap_monthly-2018-11-01-0552    written   3.3                3G    -
rpool/disks/vm-100-disk-1@zfs-auto-snap_monthly-2018-12-01-0552    written   0                        -
rpool/disks/vm-100-disk-1@zfs-auto-snap_daily-2018-12-03-0525      written   0                        -
rpool/disks/vm-100-disk-1@zfs-auto-snap_daily-2018-12-04-0525      written   0                        -
rpool/disks/vm-100-disk-1@zfs-auto-snap_daily-2018-12-05-0525      written   0                        -
rpool/disks/vm-100-disk-1@zfs-auto-snap_daily-2018-12-06-0525      written   0                        -
rpool/disks/vm-100-disk-1@zfs-auto-snap_daily-2018-12-07-0525      written   0                        -
rpool/disks/vm-100-disk-1@zfs-auto-snap_daily-2018-12-08-0525      written   0                        -
rpool/disks/vm-100-disk-1@zfs-auto-snap_daily-2018-12-09-0525      written   0                        -
rpool/disks/vm-100-disk-1@zfs-auto-snap_daily-2018-12-10-0525      written   0                        -
rpool/disks/vm-100-disk-1@zfs-auto-snap_daily-2018-12-11-0525      written   0                        -
rpool/disks/vm-100-disk-1@zfs-auto-snap_daily-2018-12-12-0525      written   756                M     -
rpool/disks/vm-100-disk-1@zfs-auto-snap_daily-2018-12-13-0525      written   461                M     -
rpool/disks/vm-100-disk-1@zfs-auto-snap_daily-2018-12-14-0525      written   428                M     -
rpool/disks/vm-100-disk-1@zfs-auto-snap_daily-2018-12-15-0525      written   412                M     -
rpool/disks/vm-100-disk-1@zfs-auto-snap_daily-2018-12-16-0525      written   419                M     -
rpool/disks/vm-101-disk-1                                          written   970                M     -
rpool/disks/vm-101-disk-1@zfs-auto-snap_monthly-2018-11-01-0552    written   13.                3G    -
rpool/disks/vm-101-disk-1@zfs-auto-snap_monthly-2018-12-01-0552    written   1.9                5G    -
rpool/disks/vm-101-disk-1@zfs-auto-snap_daily-2018-12-03-0525      written   960                M     -
rpool/disks/vm-101-disk-1@zfs-auto-snap_daily-2018-12-04-0525      written   682                M     -
rpool/disks/vm-101-disk-1@zfs-auto-snap_daily-2018-12-05-0525      written   518                M     -
rpool/disks/vm-101-disk-1@zfs-auto-snap_daily-2018-12-06-0525      written   507                M     -
rpool/disks/vm-101-disk-1@zfs-auto-snap_daily-2018-12-07-0525      written   661                M     -
rpool/disks/vm-101-disk-1@zfs-auto-snap_daily-2018-12-08-0525      written   582                M     -
rpool/disks/vm-101-disk-1@zfs-auto-snap_daily-2018-12-09-0525      written   758                M     -
rpool/disks/vm-101-disk-1@zfs-auto-snap_daily-2018-12-10-0525      written   591                M     -
rpool/disks/vm-101-disk-1@zfs-auto-snap_daily-2018-12-11-0525      written   475                M     -
rpool/disks/vm-101-disk-1@zfs-auto-snap_daily-2018-12-12-0525      written   590                M     -
rpool/disks/vm-101-disk-1@zfs-auto-snap_daily-2018-12-13-0525      written   450                M     -
rpool/disks/vm-101-disk-1@zfs-auto-snap_daily-2018-12-14-0525      written   518                M     -
rpool/disks/vm-101-disk-1@zfs-auto-snap_daily-2018-12-15-0525      written   455                M     -
rpool/disks/vm-101-disk-1@zfs-auto-snap_daily-2018-12-16-0525      written   800                M     -
rpool/iso                                                          written   1.8                9G    -
rpool/storage                                                      written   0                        -
rpool/storage@zfs-auto-snap_monthly-2018-11-01-0552                written   96K                      -
rpool/storage@zfs-auto-snap_monthly-2018-12-01-0552                written   0                        -
rpool/storage@zfs-auto-snap_daily-2018-12-03-0525                  written   0                        -
rpool/storage@zfs-auto-snap_daily-2018-12-04-0525                  written   0                        -
rpool/storage@zfs-auto-snap_daily-2018-12-05-0525                  written   0                        -
rpool/storage@zfs-auto-snap_daily-2018-12-06-0525                  written   0                        -
rpool/storage@zfs-auto-snap_daily-2018-12-07-0525                  written   0                        -
rpool/storage@zfs-auto-snap_daily-2018-12-08-0525                  written   0                        -
rpool/storage@zfs-auto-snap_daily-2018-12-09-0525                  written   0                        -
rpool/storage@zfs-auto-snap_daily-2018-12-10-0525                  written   0                        -
rpool/storage@zfs-auto-snap_daily-2018-12-11-0525                  written   0                        -
rpool/storage@zfs-auto-snap_daily-2018-12-12-0525                  written   0                        -
rpool/storage@zfs-auto-snap_daily-2018-12-13-0525                  written   0                        -
rpool/storage@zfs-auto-snap_daily-2018-12-14-0525                  written   0                        -
rpool/storage@zfs-auto-snap_daily-2018-12-15-0525                  written   0                        -
rpool/storage@zfs-auto-snap_daily-2018-12-16-0525                  written   0                        -
rpool/storage/vm-101-disk-1                                        written   133                M     -
rpool/storage/vm-101-disk-1@zfs-auto-snap_monthly-2018-11-01-0552  written   135                G     -
rpool/storage/vm-101-disk-1@zfs-auto-snap_monthly-2018-12-01-0552  written   655                M     -
rpool/storage/vm-101-disk-1@zfs-auto-snap_daily-2018-12-03-0525    written   112                M     -
rpool/storage/vm-101-disk-1@zfs-auto-snap_daily-2018-12-04-0525    written   53.                2M    -
rpool/storage/vm-101-disk-1@zfs-auto-snap_daily-2018-12-05-0525    written   48.                8M    -
rpool/storage/vm-101-disk-1@zfs-auto-snap_daily-2018-12-06-0525    written   29.                7M    -
rpool/storage/vm-101-disk-1@zfs-auto-snap_daily-2018-12-07-0525    written   53.                3M    -
rpool/storage/vm-101-disk-1@zfs-auto-snap_daily-2018-12-08-0525    written   45.                6M    -
rpool/storage/vm-101-disk-1@zfs-auto-snap_daily-2018-12-09-0525    written   79.                0M    -
rpool/storage/vm-101-disk-1@zfs-auto-snap_daily-2018-12-10-0525    written   60.                0M    -
rpool/storage/vm-101-disk-1@zfs-auto-snap_daily-2018-12-11-0525    written   54.                6M    -
rpool/storage/vm-101-disk-1@zfs-auto-snap_daily-2018-12-12-0525    written   46.                0M    -
rpool/storage/vm-101-disk-1@zfs-auto-snap_daily-2018-12-13-0525    written   38.                4M    -
rpool/storage/vm-101-disk-1@zfs-auto-snap_daily-2018-12-14-0525    written   38.                0M    -
rpool/storage/vm-101-disk-1@zfs-auto-snap_daily-2018-12-15-0525    written   46.                9M    -
rpool/storage/vm-101-disk-1@zfs-auto-snap_daily-2018-12-16-0525    written   58.                8M    -
rpool/swap                                                         written   7.5                7G    -
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!