I broke my proxmox node ( Can't locate PVE/OTP.pm in @INC )

Nicholas Merrill

New Member
Feb 27, 2017
5
0
1
51
Hi

I now know that what I did was wrong, but I did apt-get update ; apt-get upgrade ... instead of dist-upgrade

Now things are broken and I don't seem to be able to fix it. The WebGUI no longer works, and in fact a bunch of PVE services don't, and I get this error when I try to run pveversion:


Code:
Can't locate PVE/OTP.pm in @INC (you may need to install the PVE::OTP module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.20.2 /usr/local/share/perl/5.20.2 /usr/lib/x86_64-linux-gnu/perl5/5.20 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.20 /usr/share/perl/5.20 /usr/local/lib/site_perl .) at /usr/share/perl5/PVE/AccessControl.pm line 13.
BEGIN failed--compilation aborted at /usr/share/perl5/PVE/AccessControl.pm line 13.
Compilation failed in require at /usr/share/perl5/PVE/RPCEnvironment.pm line 14.
BEGIN failed--compilation aborted at /usr/share/perl5/PVE/RPCEnvironment.pm line 14.
Compilation failed in require at /usr/share/perl5/PVE/API2/APT.pm line 20.
BEGIN failed--compilation aborted at /usr/share/perl5/PVE/API2/APT.pm line 20.
Compilation failed in require at /usr/bin/pveversion line 7.
BEGIN failed--compilation aborted at /usr/bin/pveversion line 7.

I don't have console access to this server, only ssh access .. is there a way from the command-line to fix this without losing my existing data and VM's ?

thanks in advance,

Nick
 
try to install all package again.

> apt-get update
> apt-get install proxmox-ve
 
try to install all package again.

> apt-get update
> apt-get install proxmox-ve

If I do that I get this:

apt-get update works just fine.. but when I try to 'apt-get install proxmox-ve' I run into the same problem with the missing OTP.pm

Code:
# apt-get install proxmox-ve

Reading package lists... Done

Building dependency tree      

Reading state information... Done

proxmox-ve is already the newest version.

0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.

4 not fully installed or removed.

After this operation, 0 B of additional disk space will be used.

Do you want to continue? [Y/n]

Setting up pve-firewall (3.0-5) ...

Job for pve-firewall.service failed. See 'systemctl status pve-firewall.service' and 'journalctl -xn' for details.

dpkg: error processing package pve-firewall (--configure):

 subprocess installed post-installation script returned error exit status 1

dpkg: dependency problems prevent configuration of qemu-server:

 qemu-server depends on pve-firewall; however:

  Package pve-firewall is not configured yet.


dpkg: error processing package qemu-server (--configure):

 dependency problems - leaving unconfigured

dpkg: dependency problems prevent configuration of pve-manager:

 pve-manager depends on qemu-server (>= 1.1-1); however:

  Package qemu-server is not configured yet.

 pve-manager depends on pve-firewall; however:

  Package pve-firewall is not configured yet.


dpkg: error processing package pve-manager (--configure):

 dependency problems - leaving unconfigured

dpkg: dependency problems prevent configuration of proxmox-ve:

 proxmox-ve depends on pve-manager; however:

  Package pve-manager is not configured yet.

 proxmox-ve depends on qemu-server; however:

  Package qemu-server is not configured yet.


dpkg: error processing package proxmox-ve (--configure):

 dependency problems - leaving unconfigured

Processing triggers for pve-ha-manager (2.0-4) ...

Errors were encountered while processing:

 pve-firewall

 qemu-server

 pve-manager

 proxmox-ve

E: Sub-process /usr/bin/dpkg returned an error code (1)

if I run "systemctl status pve-firewall.service" as suggested in the. output, I get:

Code:
# systemctl status pve-firewall.service

● pve-firewall.service - Proxmox VE firewall

   Loaded: loaded (/lib/systemd/system/pve-firewall.service; enabled)

   Active: active (running) (Result: exit-code) since Sun 2018-01-14 11:18:04 EST; 1 weeks 6 days ago

  Process: 20011 ExecReload=/usr/sbin/pve-firewall restart (code=exited, status=2)

 Main PID: 3212 (pve-firewall)

   CGroup: /system.slice/pve-firewall.service

          └─3212 pve-firewall


Jan 27 22:20:19 hostname pve-firewall[20011]: Can't locate PVE/OTP.pm in @INC (you may need to install the PVE::OTP module) (@INC contains: ...

Jan 27 22:20:19 hostname pve-firewall[20011]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/AccessControl.pm line 13.

Jan 27 22:20:19 hostname pve-firewall[20011]: Compilation failed in require at /usr/share/perl5/PVE/RPCEnvironment.pm line 14.

Jan 27 22:20:19 hostname pve-firewall[20011]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/RPCEnvironment.pm line 14.

Jan 27 22:20:19 hostname pve-firewall[20011]: Compilation failed in require at /usr/share/perl5/PVE/Service/pve_firewall.pm line 13.

Jan 27 22:20:19 hostname pve-firewall[20011]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Service/pve_firewall.pm line 13.

Jan 27 22:20:19 hostname pve-firewall[20011]: Compilation failed in require at /usr/sbin/pve-firewall line 6.

Jan 27 22:20:19 hostname pve-firewall[20011]: BEGIN failed--compilation aborted at /usr/sbin/pve-firewall line 6.

Jan 27 22:20:19 hostname systemd[1]: pve-firewall.service: control process exited, code=exited status=2

Jan 27 22:20:19 hostname systemd[1]: Reload failed for Proxmox VE firewall.

Hint: Some lines were ellipsized, use -l to show in full.
 
try
Code:
apt-get install --reinstall libpve-common-perl
and then install proxmox-ve
 
Hi, thanks for the pointer.. I tried that - but libpve-common-perl doesn't seem to include OTP.pm .. at least not in the one I am getting from the proxmox enterprise repo .. which is libpve-common-perl_4.0-96_all.deb

I even built a second experimental proxmox server and tried to find OTP.pm on it but it isn't there either.

Quite a mystery!

Code:
~/tmp/usr/share/perl5/PVE# ls -al

total 255

drwxr-xr-x 2 root root    17 Jul 14  2017 .

drwxr-xr-x 3 root root     3 Jul 14  2017 ..

-rw-r--r-- 1 root root   297 Jul 14  2017 AtomicFile.pm

-rw-r--r-- 1 root root 12371 Jul 14  2017 CLIHandler.pm

-rw-r--r-- 1 root root  3645 Jul 14  2017 CpuSet.pm

-rw-r--r-- 1 root root 18436 Jul 14  2017 Daemon.pm

-rw-r--r-- 1 root root  2668 Jul 14  2017 Exception.pm

-rw-r--r-- 1 root root 34505 Jul 14  2017 INotify.pm

-rw-r--r-- 1 root root 47103 Jul 14  2017 JSONSchema.pm

-rw-r--r-- 1 root root 16980 Jul 14  2017 Network.pm

-rw-r--r-- 1 root root 10127 Jul 14  2017 ProcFSTools.pm

-rw-r--r-- 1 root root 15106 Jul 14  2017 RESTEnvironment.pm

-rw-r--r-- 1 root root 17920 Jul 14  2017 RESTHandler.pm

-rw-r--r-- 1 root root   799 Jul 14  2017 SafeSyslog.pm

-rw-r--r-- 1 root root 11212 Jul 14  2017 SectionConfig.pm

-rw-r--r-- 1 root root  4420 Jul 14  2017 Ticket.pm

-rw-r--r-- 1 root root 38408 Jul 14  2017 Tools.pm
 
can you post 'pveversion -v'
and your repositories under /etc/apt/sources.list and /etc/apt/sources.list.d/* ?

btw the OTP.pm only exists in pve 5
 
Code:
# pveversion -v

Can't locate PVE/OTP.pm in @INC (you may need to install the PVE::OTP module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.20.2 /usr/local/share/perl/5.20.2 /usr/lib/x86_64-linux-gnu/perl5/5.20 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.20 /usr/share/perl/5.20 /usr/local/lib/site_perl .) at /usr/share/perl5/PVE/AccessControl.pm line 13.

BEGIN failed--compilation aborted at /usr/share/perl5/PVE/AccessControl.pm line 13.

Compilation failed in require at /usr/share/perl5/PVE/RPCEnvironment.pm line 14.

BEGIN failed--compilation aborted at /usr/share/perl5/PVE/RPCEnvironment.pm line 14.

Compilation failed in require at /usr/share/perl5/PVE/API2/APT.pm line 20.

BEGIN failed--compilation aborted at /usr/share/perl5/PVE/API2/APT.pm line 20.

Compilation failed in require at /usr/bin/pveversion line 7.

BEGIN failed--compilation aborted at /usr/bin/pveversion line 7.

and here are apt sources:

Code:
# cat /etc/apt/sources.list

deb http://ftp.us.debian.org/debian jessie main contrib

# security updates

deb http://security.debian.org jessie/updates main contrib


# cat /etc/apt/sources.list.d/pve-enterprise.list

deb https://enterprise.proxmox.com/debian jessie pve-enterprise

I also have a disabled pve-nosubscription.list with stretch(!) maybe that's why it's messed up?

Code:
# cat pve-nosubscription.list.disabled

# PVE pve-no-subscription repository provided by proxmox.com,

# NOT recommended for production use

deb http://download.proxmox.com/debian/pve stretch pve-no-subscription
 
You use Debian "Jessie" and install Proxmox VE "Stretch", this cannot work.
 
yeah it seems that you installed a single/some packages from pve5 (stretch), maybe try to reinstall all pve packages one after another and see which one it was
here a pveversion output for a list of the packages:

proxmox-ve: 4.4-105 (running kernel: 4.4.98-5-pve)
pve-manager: 4.4-22 (running version: 4.4-22/2728f613)
pve-kernel-4.4.98-5-pve: 4.4.98-105
lvm2: 2.02.116-pve3
corosync-pve: 2.4.2-2~pve4+1
libqb0: 1.0.1-1
pve-cluster: 4.0-54
qemu-server: 4.0-115
pve-firmware: 1.1-11
libpve-common-perl: 4.0-96
libpve-access-control: 4.0-23
libpve-storage-perl: 4.0-76
pve-libspice-server1: 0.12.8-2
vncterm: 1.3-2
pve-docs: 4.4-4
pve-qemu-kvm: 2.9.1-6~pve4
pve-container: 1.0-104
pve-firewall: 2.0-33
pve-ha-manager: 1.0-41
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u3
lxc-pve: 2.0.7-4
lxcfs: 2.0.6-pve1
criu: 1.6.0-1
novnc-pve: 0.5-9
smartmontools: 6.5+svn4324-1~pve80
zfsutils: 0.6.5.9-pve15~bpo80
 
I was able to recover a system that had this happen to it. I think the key bit was to run:

<code>dpkg --purge libappconfig-perl libjs-extjs libtemplate-perl</code>

Not totally sure about all three packages, and I sure don't want to go through that again.

Just recording here for future generations.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!