[SOLVED] User Interface Unstable

manjotsc

Member
Jul 2, 2020
48
3
8
Montreal, Quebec
Hi,

I just upgraded to Proxmox 6.3, after rebooting the server. I found that User Interface is not showing correct information about VMs, LXCs and Storages. It only showing 1 LXC and 1 QEMU are running, when in fact there are more LXCs that are running and all the hard drives and SSDs have question mark aside them also under Restore and Backup cannot see Storages.

Thanks,

Screenshot 2021-01-11 030057.png

Storages

Screenshot 2021-01-11 030115.png

Backup and Restore

Screenshot 2021-01-11 032400.png
 
Last edited:

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
2,792
314
88
hi,

could you please post:
* pveversion -v
* cat /etc/pve/storage.cfg
 

manjotsc

Member
Jul 2, 2020
48
3
8
Montreal, Quebec
Code:
root@vms:~# pveversion -v                                                                                                                                                                                     
proxmox-ve: 6.3-1 (running kernel: 5.4.78-2-pve)                                                                                                                                                               
pve-manager: 6.3-3 (running version: 6.3-3/eee5f901)                                                                                                                                                           
pve-kernel-5.4: 6.3-3                                                                                                                                                                                         
pve-kernel-helper: 6.3-3                                                                                                                                                                                       
pve-kernel-5.4.78-2-pve: 5.4.78-2                                                                                                                                                                             
pve-kernel-5.4.34-1-pve: 5.4.34-2                                                                                                                                                                             
ceph-fuse: 14.2.16-pve1                                                                                                                                                                                       
corosync: 3.0.4-pve1                                                                                                                                                                                           
criu: 3.11-3                                                                                                                                                                                                   
glusterfs-client: 5.5-3                                                                                                                                                                                       
ifupdown: 0.8.35+pve1                                                                                                                                                                                         
ksm-control-daemon: 1.3-1                                                                                                                                                                                     
libjs-extjs: 6.0.1-10                                                                                                                                                                                         
libknet1: 1.16-pve1                                                                                                                                                                                           
libproxmox-acme-perl: 1.0.7                                                                                                                                                                                   
libproxmox-backup-qemu0: 1.0.2-1                                                                                                                                                                               
libpve-access-control: 6.1-3                                                                                                                                                                                   
libpve-apiclient-perl: 3.1-3                                                                                                                                                                                   
libpve-common-perl: 6.3-2                                                                                                                                                                                     
libpve-guest-common-perl: 3.1-3                                                                                                                                                                               
libpve-http-server-perl: 3.1-1                                                                                                                                                                                 
libpve-storage-perl: 6.3-3                                                                                                                                                                                     
libqb0: 1.0.5-1                                                                                                                                                                                               
libspice-server1: 0.14.2-4~pve6+1                                                                                                                                                                             
lvm2: 2.03.02-pve4                                                                                                                                                                                             
lxc-pve: 4.0.3-1                                                                                                                                                                                               
lxcfs: 4.0.3-pve3                                                                                                                                                                                             
novnc-pve: 1.1.0-1                                                                                                                                                                                             
proxmox-backup-client: 1.0.6-1                                                                                                                                                                                 
proxmox-mini-journalreader: 1.1-1                                                                                                                                                                             
proxmox-widget-toolkit: 2.4-3                                                                                                                                                                                 
pve-cluster: 6.2-1                                                                                                                                                                                             
pve-container: 3.3-2                                                                                                                                                                                           
pve-docs: 6.3-1                                                                                                                                                                                               
pve-edk2-firmware: 2.20200531-1                                                                                                                                                                               
pve-firewall: 4.1-3                                                                                                                                                                                           
pve-firmware: 3.1-3                                                                                                                                                                                           
pve-ha-manager: 3.1-1                                                                                                                                                                                         
pve-i18n: 2.2-2                                                                                                                                                                                               
pve-qemu-kvm: 5.1.0-7                                                                                                                                                                                         
pve-xtermjs: 4.7.0-3                                                                                                                                                                                           
qemu-server: 6.3-2                                                                                                                                                                                             
smartmontools: 7.1-pve2                                                                                                                                                                                       
spiceterm: 3.1-1                                                                                                                                                                                               
vncterm: 1.6-2                                                                                                                                                                                                 
zfsutils-linux: 0.8.5-pve1

Storages

Code:
root@vms:~# cat /etc/pve/storage.cfg
dir: local
        path /var/lib/vz
        content rootdir,snippets,vztmpl,iso
        shared 0


dir: VMDrive
        path /mnt/VMDrive
        content iso,backup,vztmpl,snippets,images,rootdir
        maxfiles 0
        shared 1


dir: Seagate4TB
        path /mnt/4TB
        content vztmpl,backup,snippets,images,rootdir,iso
        maxfiles 0
        shared 1


dir: S1TB
        path /mnt/S1TB
        content iso,images,rootdir,snippets,vztmpl,backup
        maxfiles 0
        shared 0


lvmthin: LVM-THIN
        thinpool data
        vgname pve
        content images,rootdir


dir: OneTB
        path /mnt/pve/OneTB
        content images,rootdir,vztmpl,backup,snippets,iso
        is_mountpoint 1
        maxfiles 20
        nodes vms
        shared 0


cifs: ProgramRepo
        path /mnt/pve/ProgramRepo
        server repo.manjot.net
        share ProgramRepo
        content images
        domain manjot.net
        maxfiles 0
        username repo


dir: SSDStorage
        path /mnt/pve/SSDStorage
        content images,rootdir,backup,vztmpl,snippets,iso
        is_mountpoint 1
        nodes vms


cifs: VMShare
        path /mnt/pve/VMShare
        server server.manjot.net
        share VMShare
        content snippets,vztmpl,backup,images,rootdir,iso
        domain manjot.net
        maxfiles 1
        username vm


dir: TenTB
        path /mnt/pve/TenTB
        content vztmpl,backup,snippets,iso,rootdir,images
        is_mountpoint 1
        nodes vms
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
2,792
314
88
most likely one of your cifs share being unavailable is causing this. could you take a look at syslog, dmesg, journalctl for any such signs?
 

manjotsc

Member
Jul 2, 2020
48
3
8
Montreal, Quebec
@oguz I removed all of the CIFS storages but still the issue still persist and this is where the Proxmox server gets stuck at boot screen, but gui remains accessible and vm and lxc run normally, gui still unstable.

20210111_224138_2.jpg
 
Last edited:

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
2,792
314
88
could you check

* /var/log/syslog
* journalctl
* dmesg
 

manjotsc

Member
Jul 2, 2020
48
3
8
Montreal, Quebec
Syslog and DMESG are flooded with repated messages. Need guidance

Thanks,

JournalCTL

Code:
root@vms:~# journalctl                                                                                                                                                                                        
-- Logs begin at Wed 2021-01-13 04:22:56 EST, end at Wed 2021-01-13 05:52:05 EST. --                                                                                                                          
Jan 13 04:22:56 vms smartd[960]: Device: /dev/sdd [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 29 to 30                                                                                
Jan 13 04:22:56 vms systemd-udevd[3250144]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.                                                          
Jan 13 04:22:56 vms kernel: nvidia-nvlink: Nvlink Core is being initialized, major device number 238                                                                                                          
Jan 13 04:22:56 vms kernel: NVRM: The NVIDIA probe routine was not called for 1 device(s).                                                                                                                    
Jan 13 04:22:56 vms kernel: NVRM: This can occur when a driver such as:                                                                                                                                      
                            NVRM: nouveau, rivafb, nvidiafb or rivatv                                                                                                                                        
                            NVRM: was loaded and obtained ownership of the NVIDIA device(s).                                                                                                                  
Jan 13 04:22:56 vms kernel: NVRM: Try unloading the conflicting kernel module (and/or                                                                                                                        
                            NVRM: reconfigure your kernel without the conflicting                                                                                                                            
                            NVRM: driver(s)), then try loading the NVIDIA kernel module                                                                                                                      
                            NVRM: again.                                                                                                                                                                      
Jan 13 04:22:56 vms kernel: NVRM: No NVIDIA devices probed.                                                                                                                                                  
Jan 13 04:22:56 vms kernel: nvidia-nvlink: Unregistered the Nvlink Core, major device number 238                                                                                                              
Jan 13 04:22:56 vms systemd-udevd[3250482]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.                                                          
Jan 13 04:22:56 vms systemd-udevd[3250482]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.                                                          
Jan 13 04:22:56 vms systemd-udevd[3250144]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.                                                          
Jan 13 04:22:56 vms kernel: nvidia-nvlink: Nvlink Core is being initialized, major device number 238                                                                                                          
Jan 13 04:22:56 vms kernel: NVRM: The NVIDIA probe routine was not called for 1 device(s).                                                                                                                    
Jan 13 04:22:56 vms kernel: NVRM: This can occur when a driver such as:                                                                                                                                      
                            NVRM: nouveau, rivafb, nvidiafb or rivatv                                                                                                                                        
                            NVRM: was loaded and obtained ownership of the NVIDIA device(s).                                                                                                                  
Jan 13 04:22:56 vms kernel: NVRM: Try unloading the conflicting kernel module (and/or                                                                                                                        
                            NVRM: reconfigure your kernel without the conflicting                                                                                                                            
                            NVRM: driver(s)), then try loading the NVIDIA kernel module                                                                                                                      
                            NVRM: again.                                                                                                                                                                      
Jan 13 04:22:56 vms kernel: NVRM: No NVIDIA devices probed.                                                                                                                                                  
Jan 13 04:22:56 vms kernel: nvidia-nvlink: Unregistered the Nvlink Core, major device number 238                                                                                                              
Jan 13 04:22:56 vms systemd-udevd[3250482]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.                                                          
Jan 13 04:22:56 vms systemd-udevd[3250482]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.                                                          
Jan 13 04:22:56 vms systemd-udevd[3250144]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.                                                          
Jan 13 04:22:56 vms kernel: nvidia-nvlink: Nvlink Core is being initialized, major device number 238                                                                                                          
Jan 13 04:22:56 vms kernel: NVRM: The NVIDIA probe routine was not called for 1 device(s).                                                                                                                    
Jan 13 04:22:56 vms kernel: NVRM: This can occur when a driver such as:                                                                                                                                      
                            NVRM: nouveau, rivafb, nvidiafb or rivatv                                                                                                                                        
                            NVRM: was loaded and obtained ownership of the NVIDIA device(s).                                                                                                                  
Jan 13 04:22:56 vms kernel: NVRM: Try unloading the conflicting kernel module (and/or                                                                                                                        
                            NVRM: reconfigure your kernel without the conflicting                                                                                                                            
                            NVRM: driver(s)), then try loading the NVIDIA kernel module                                                                                                                      
                            NVRM: again.                                                                                                                                                                      
Jan 13 04:22:56 vms kernel: NVRM: No NVIDIA devices probed.                                                                                                                                                  
Jan 13 04:22:56 vms kernel: nvidia-nvlink: Unregistered the Nvlink Core, major device number 238                                                                                                              
Jan 13 04:22:56 vms systemd-udevd[3250482]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.                                                          
Jan 13 04:22:56 vms systemd-udevd[3250482]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.                                                          
Jan 13 04:22:56 vms systemd-udevd[3250144]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.                                                          
Jan 13 04:22:56 vms kernel: nvidia-nvlink: Nvlink Core is being initialized, major device number 238

DMESG

Code:
root@vms:~# dmesg
[35946.070138] NVRM: No NVIDIA devices probed.
[35946.070676] nvidia-nvlink: Unregistered the Nvlink Core, major device number 238
[35946.224742] nvidia-nvlink: Nvlink Core is being initialized, major device number 238
[35946.225743] NVRM: The NVIDIA probe routine was not called for 1 device(s).
[35946.226174] NVRM: This can occur when a driver such as:
               NVRM: nouveau, rivafb, nvidiafb or rivatv
               NVRM: was loaded and obtained ownership of the NVIDIA device(s).
[35946.227830] NVRM: Try unloading the conflicting kernel module (and/or
               NVRM: reconfigure your kernel without the conflicting
               NVRM: driver(s)), then try loading the NVIDIA kernel module
               NVRM: again.
[35946.229833] NVRM: No NVIDIA devices probed.
[35946.230562] nvidia-nvlink: Unregistered the Nvlink Core, major device number 238
[35946.405775] nvidia-nvlink: Nvlink Core is being initialized, major device number 238
[35946.406597] NVRM: The NVIDIA probe routine was not called for 1 device(s).
[35946.407360] NVRM: This can occur when a driver such as:
               NVRM: nouveau, rivafb, nvidiafb or rivatv
               NVRM: was loaded and obtained ownership of the NVIDIA device(s).
[35946.408710] NVRM: Try unloading the conflicting kernel module (and/or
               NVRM: reconfigure your kernel without the conflicting
               NVRM: driver(s)), then try loading the NVIDIA kernel module
               NVRM: again.

Syslog


Code:
Jan 13 00:00:08 vms rsyslogd:  [origin software="rsyslogd" swVersion="8.1901.0" x-pid="947" x-info="https://www.rsyslog.com"] rsyslogd was HUPed
Jan 13 00:00:08 vms systemd-udevd[1529488]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.
Jan 13 00:00:08 vms systemd[1]: logrotate.service: Succeeded.
Jan 13 00:00:08 vms systemd[1]: Started Rotate log files.
Jan 13 00:00:08 vms systemd-udevd[1529488]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.
Jan 13 00:00:08 vms systemd-udevd[1529488]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.
Jan 13 00:00:08 vms systemd-udevd[1529497]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.
Jan 13 00:00:08 vms kernel: [14841.599074] nvidia-nvlink: Nvlink Core is being initialized, major device number 238
Jan 13 00:00:08 vms kernel: [14841.600075] NVRM: The NVIDIA probe routine was not called for 1 device(s).
Jan 13 00:00:08 vms kernel: [14841.600470] NVRM: This can occur when a driver such as:
Jan 13 00:00:08 vms kernel: [14841.600470] NVRM: nouveau, rivafb, nvidiafb or rivatv
Jan 13 00:00:08 vms kernel: [14841.600470] NVRM: was loaded and obtained ownership of the NVIDIA device(s).
Jan 13 00:00:08 vms kernel: [14841.601566] NVRM: Try unloading the conflicting kernel module (and/or
Jan 13 00:00:08 vms kernel: [14841.601566] NVRM: reconfigure your kernel without the conflicting
Jan 13 00:00:08 vms kernel: [14841.601566] NVRM: driver(s)), then try loading the NVIDIA kernel module
Jan 13 00:00:08 vms kernel: [14841.601566] NVRM: again.
Jan 13 00:00:08 vms kernel: [14841.603794] NVRM: No NVIDIA devices probed.
Jan 13 00:00:08 vms kernel: [14841.604733] nvidia-nvlink: Unregistered the Nvlink Core, major device number 238
Jan 13 00:00:08 vms systemd-udevd[1529488]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.
Jan 13 00:00:08 vms systemd-udevd[1529488]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.
Jan 13 00:00:08 vms systemd-udevd[1529488]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.
Jan 13 00:00:08 vms systemd-udevd[1529488]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.
Jan 13 00:00:08 vms systemd-udevd[1529488]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.
Jan 13 00:00:08 vms systemd-udevd[1529488]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.
Jan 13 00:00:08 vms systemd-udevd[1529488]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.
Jan 13 00:00:08 vms systemd-udevd[1529488]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.
Jan 13 00:00:08 vms systemd-udevd[1529488]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.
Jan 13 00:00:08 vms systemd-udevd[1529488]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.
Jan 13 00:00:08 vms systemd-udevd[1529497]: Process '/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 666 /dev/nvidia*'' failed with exit code 9.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!