[SOLVED] "local" storage: unable to display content

D

Donald F

Member
Sep 30, 2020
5
0
6
30
Hello,

Following the update of proxmox 5 to 6, the contents of the "local" storage (/var/lib/vz) of one of the servers of the cluster are no longer accessible from the web interface.

No problem to display the assembly summary,
However an error occurs when displaying the content (from any server in the cluster on this specific server local storage):
"Can't call method "mode" on an undefined value at /usr/share/perl5/PVE/Storage/Plugin.pm line 786. (500)"

I do not see a problem of access rights on the content or on the script.
2 VMs on this server have disks on this storage and I am able to start them without problem.

Do you have any idea of the problem?

The server is up to date:
Code: 
pveversion -v
proxmox-ve: 6.2-2 (running kernel: 5.4.65-1-pve)
pve-manager: 6.2-12 (running version: 6.2-12/b287dd27)
pve-kernel-5.4: 6.2-7
pve-kernel-helper: 6.2-7
pve-kernel-5.4.65-1-pve: 5.4.65-1
pve-kernel-4.15: 5.4-19
pve-kernel-4.15.18-30-pve: 4.15.18-58
pve-kernel-4.13.8-2-pve: 4.13.8-28
pve-kernel-4.4.95-1-pve: 4.4.95-99
pve-kernel-4.4.49-1-pve: 4.4.49-86
pve-kernel-4.4.44-1-pve: 4.4.44-84
pve-kernel-4.4.35-1-pve: 4.4.35-77
ceph: 14.2.11-pve1
ceph-fuse: 14.2.11-pve1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.5
libpve-access-control: 6.1-2
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.2-2
libpve-guest-common-perl: 3.1-3
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.2-6
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.3-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
proxmox-backup-client: 0.8.21-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.2-12
pve-cluster: 6.1-8
pve-container: 3.2-2
pve-docs: 6.2-6
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.1-3
pve-ha-manager: 3.1-1
pve-i18n: 2.2-1
pve-qemu-kvm: 5.1.0-2
pve-xtermjs: 4.7.0-2
qemu-server: 6.2-14
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 0.8.4-pve1
 
Could you please post the following?

Code: 
sed -n '770,800p' /usr/share/perl5/PVE/Storage/Plugin.pm
ls -lah sed -n '770,800p' /usr/share/perl5/PVE/Storage/
pvesh get nodes/<YOUR_NODE_NAME>/storage/local/content --output-format json
 
Here is :

Code: 
root@proxmox2:~# sed -n '770,800p' /usr/share/perl5/PVE/Storage/Plugin.pm
        unlink($path) || die "unlink '$path' failed - $!\n";
    }

    # try to cleanup directory to not clutter storage with empty $vmid dirs if
    # all images from a guest got deleted
    my $dir = dirname($path);
    rmdir($dir);

    return undef;
}

sub file_size_info {
    my ($filename, $timeout) = @_;

    my $st = File::stat::stat($filename);

    if (S_ISDIR($st->mode)) {
        return wantarray ? (0, 'subvol', 0, undef, $st->ctime) : 1;
    }

    my $json = '';
    eval {
        run_command(['/usr/bin/qemu-img', 'info', '--output=json', $filename],
            timeout => $timeout,
            outfunc => sub { $json .= shift },
            errfunc => sub { warn "$_[0]\n" }
        );
    };
    warn $@ if $@;

    my $info = eval { decode_json($json) };

Code: 
root@proxmox2:~# ls -lah  /usr/share/perl5/PVE/Storage/
total 224K
drwxr-xr-x  3 root root 4.0K Sep 29 15:08 .
drwxr-xr-x 17 root root 4.0K Sep 29 13:39 ..
-rw-r--r--  1 root root 6.1K Jul 24 19:10 CephFSPlugin.pm
-rw-r--r--  1 root root 6.7K Jul 24 19:10 CIFSPlugin.pm
-rw-r--r--  1 root root 3.4K Jul 24 19:10 DirPlugin.pm
-rw-r--r--  1 root root 9.2K Jul 24 19:10 DRBDPlugin.pm
-rw-r--r--  1 root root 8.7K Jul 24 19:10 GlusterfsPlugin.pm
-rw-r--r--  1 root root 5.6K Jul 24 19:10 ISCSIDirectPlugin.pm
-rw-r--r--  1 root root 9.9K Jul 24 19:10 ISCSIPlugin.pm
drwxr-xr-x  2 root root 4.0K Sep 29 13:37 LunCmd
-rw-r--r--  1 root root  18K Jul 24 19:10 LVMPlugin.pm
-rw-r--r--  1 root root 9.8K Jul 24 19:10 LvmThinPlugin.pm
-rw-r--r--  1 root root 4.2K Jul 24 19:10 NFSPlugin.pm
-rw-r--r--  1 root root  15K Jul 24 19:10 PBSPlugin.pm
-rw-r--r--  1 root root  39K Jul 24 19:10 Plugin.pm
-rw-r--r--  1 root root  20K Jul 24 19:10 RBDPlugin.pm
-rw-r--r--  1 root root  11K Jul 24 19:10 ZFSPlugin.pm
-rw-r--r--  1 root root  19K Jul 24 19:10 ZFSPoolPlugin.pm

Code: 
root@proxmox2:~# pvesh get nodes/proxmox2/storage/local/content --output-format json
Can't call method "mode" on an undefined value at /usr/share/perl5/PVE/Storage/Plugin.pm line 786.
 
Thank you! The first 2 code snippets look exactly the same on my machine, but the API call (pvesh ...) works for me.

Could you, as a first step, try to simply reinstall the relevant package apt update && apt install --reinstall libpve-storage-perl?
 
Thanks for your feedback,
Following reinstallation of the package, the result is not improved on the local storage of the server in question.
The script itself works on all cluster servers, there is no problem to list the content of other local storage on the other cluster servers
Code: 
root@proxmox2:~# pvesh get nodes/proxmox2/storage/local/content --output-format json
Can't call method "mode" on an undefined value at /usr/share/perl5/PVE/Storage/Plugin.pm line 786.

root@proxmox2:~# pvesh get nodes/proxmox3/storage/local/content --output-format json
[{"content":"iso","ctime":1507724837,"format":"iso","size":304087040,"volid":"local:iso/debian-9.2.0-amd64-netinst.iso"},{"content":"iso","ctime":1511171922,"format":"iso","size":1046243328,"volid":"local:iso/SW_DVD5_Office_Professional_Plus_2016_64Bit_French_MLF_X20-42435.ISO"},{"content":"iso","ctime":1581496593,"format":"iso","size":5316775936,"volid":"local:iso/SW_DVD9_Win_Pro_10_1909_64BIT_French_Pro_Ent_EDU_N_MLF_X22-17398.ISO"},{"content":"iso","ctime":1545204620,"format":"iso","size":6006167552,"volid":"local:iso/SW_DVD9_Win_Server_STD_CORE_2016_64Bit_French_-4_DC_STD_MLF_X21-70530.ISO"},{"content":"iso","ctime":1507728145,"format":"iso","size":1587609600,"volid":"local:iso/ubuntu-16.04.3-desktop-amd64.iso"},{"content":"iso","ctime":1581497296,"format":"iso","size":371732480,"volid":"local:iso/virtio-win-0.1.171.iso"},{"content":"iso","ctime":1507706455,"format":"iso","size":170649600,"volid":"local:iso/virtio-win.iso"},{"content":"vztmpl","ctime":1567597275,"format":"tgz","size":885569513,"volid":"local:vztmpl/debian-9-turnkey-gitlab_15.5-1_amd64.tar.gz"}]
root@proxmox2:~#

Maybe, I can test deleted and recreated local storage
 
Last edited:
could you post the content of /etc/pve/storage.cfg? what does ls -lhR /var/lib/vz display?
 
Content of /etc/pve/storage.conf :
Code: 
dir: local
        path /var/lib/vz
        content images,rootdir,backup,vztmpl,iso
        maxfiles 2
        shared 0

lvmthin: local-lvm
        thinpool data
        vgname pve
        content images,rootdir

rbd: cephproxmox
        content images,rootdir
        krbd 1
        monhost 10.10.10.1;10.10.10.2;10.10.10.3
        pool rbd
        username admin


Command : ls -lhR /var/lib/vz
Code: 
root@proxmox2:~# ls -lhR /var/lib/vz
/var/lib/vz:
total 24K
drwxr-xr-x 2 root root 4.0K Mar 16  2017 dump
drwxr-xr-x 4 root root 4.0K Dec  2  2017 images
drwxr-xr-x 2 root root 4.0K Dec  9  2016 private
drwxr-xr-x 2 root root 4.0K Dec  9  2016 root
drwxr-xr-x 2 root root 4.0K Sep 29 15:25 snippets
drwxr-xr-x 5 root root 4.0K Mar 16  2017 template

/var/lib/vz/dump:
total 0

/var/lib/vz/images:
total 8.0K
drwxr----- 2 root root 4.0K Dec  1  2017 101
drwxr----- 2 root root 4.0K Mar 21  2017 104

/var/lib/vz/images/101:
total 0

/var/lib/vz/images/104:
total 0
lrwxrwxrwx 1 root root 34 Mar 21  2017 vm-104-disk-1.qcow2 -> /media/distant/vm-100-disk-2.qcow2
-rw-r----- 1 root root  0 Mar 21  2017 vm-104-disk-1.qcow2.sav

/var/lib/vz/private:
total 0

/var/lib/vz/root:
total 0

/var/lib/vz/snippets:
total 0

/var/lib/vz/template:
total 12K
drwxr-xr-x 2 root root 4.0K Oct 11  2017 cache
drwxr-xr-x 2 root root 4.0K Jul 26  2018 iso
drwxr-xr-x 2 root root 4.0K Dec  9  2016 qemu

/var/lib/vz/template/cache:
total 0

/var/lib/vz/template/iso:
total 16G
-rw------- 1 root root 290M Oct 11  2017 debian-9.2.0-amd64-netinst.iso
-rw------- 1 root root 188M Oct 11  2017 SW_DVD5_Office_Professional_Plus_2016_W32_French_MLF_X20-41354.ISO
-rw------- 1 root root 2.9G Oct 11  2017 SW_DVD5_Win_Pro_7w_SP1_64BIT_French_-2_MLF_X17-59310.ISO
-rw-r----- 1 root root 5.1G Sep 22  2017 SW_DVD9_Windows_Svr_Std_and_DataCtr_2012_R2_64Bit_French_-4_MLF_X19-82899.ISO
-rwxrwxrwx 1 root root 5.6G Jun 29  2017 SW_DVD9_Win_Svr_STD_Core_and_DataCtr_Core_2016_64Bit_French_-3_MLF_X21-30354.ISO
-rw------- 1 root root 1.5G Oct 11  2017 ubuntu-16.04.3-desktop-amd64.iso
-rw------- 1 root root 302M Jul 26  2018 virtio-win-0.1.141.iso
-rw-r--r-- 1 root root 163M Oct 11  2017 virtio-win.iso

/var/lib/vz/template/qemu:
total 0


If I try recreate a logical volume in local storage, won't that be a problem? it seems to me that this storage is particular
Code: 
root@proxmox2:~# lvdisplay
...
--- Logical volume ---
  LV Path                /dev/pve/root
  LV Name                root
  VG Name                pve
  LV UUID                iOW3CW-mf4r-VFDq-Keuo-x6Ob-23KG-dMNK7l
  LV Write Access        read/write
  LV Creation host, time proxmox, 2017-03-16 13:47:45 +0100
  LV Status              available
  # open                 1
  LV Size                34.00 GiB
  Current LE             8704
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:1
...
Code: 
lvremove root
lvcreate -L34816 -n root pve
 
no, don't try to change anything about LVM! pve/root is the LV where the whole system is installed!

the problem is the following invalid symlink:

Code: 
lrwxrwxrwx 1 root root 34 Mar 21  2017 vm-104-disk-1.qcow2 -> /media/distant/vm-100-disk-2.qcow2
 
Exactly, removing /var/lib/vz/images/104 solved the problem, you are too good at it, thanks ;-)
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back