PVEProxy Compilation Failed

Discussion in 'Proxmox VE: Installation and configuration' started by RedBlood9, Apr 16, 2018.

  1. RedBlood9

    RedBlood9 New Member

    Joined:
    Apr 16, 2018
    Messages:
    16
    Likes Received:
    0
    Hi to All
    We've a really big problem in prodution with two PVE 4.4-1 woth lots of KVM and LXC running on them.
    The service pveproxy.service won't start because of this error:

    We've searched in some logs, but we didn't find anything to resolv the problem.
    Here's all the information that might help us:
    Code:
    pveversion -v
    proxmox-ve: 4.4-76 (running kernel: 4.4.35-1-pve)
    pve-manager: 4.4-1 (running version: 4.4-1/eb2d6f1e)
    pve-kernel-4.4.35-1-pve: 4.4.35-76
    lvm2: 2.02.116-pve3
    corosync-pve: 2.4.0-1
    libqb0: 1.0-1
    pve-cluster: 4.0-48
    qemu-server: 4.0-101
    pve-firmware: 1.1-10
    libpve-common-perl: 4.0-83
    libpve-access-control: 4.0-19
    libpve-storage-perl: 4.0-70
    pve-libspice-server1: 0.12.8-1
    vncterm: 1.2-1
    pve-docs: 4.4-1
    pve-qemu-kvm: 2.7.0-9
    pve-container: 1.0-88
    pve-firewall: 2.0-33
    pve-ha-manager: 1.0-38
    ksm-control-daemon: 1.2-1
    glusterfs-client: 3.5.2-2+deb8u2
    lxc-pve: 2.0.6-2
    lxcfs: 2.0.5-pve1
    criu: 1.6.0-1
    novnc-pve: 0.5-8
    smartmontools: 6.5+svn4324-1~pve80
    zfsutils: 0.6.5.8-pve13~bpo80
    
    dpkg -l | grep perl
    ii  libanyevent-http-perl          2.22-1                         all          simple non-blocking HTTP/HTTPS client
    ii  libanyevent-perl               7.070-3                        amd64        event loop framework with multiple implementations
    ii  libapparmor-perl               2.9.0-3                        amd64        AppArmor library Perl bindings
    ii  libapt-pkg-perl                0.1.29+b2                      amd64        Perl interface to libapt-pkg
    ii  libauthen-pam-perl             0.16-3+b1                      amd64        Perl interface to PAM library
    ii  libclass-isa-perl              0.36-5                         all          report the search path for a class's ISA tree
    ii  libclone-perl                  0.37-1+b1                      amd64        module for recursively copying Perl datatypes
    ii  libcommon-sense-perl           3.73-2+b3                      amd64        module that implements some sane defaults for Perl programs
    ii  libconvert-asn1-perl           0.27-1                         all          Perl module for encoding and decoding ASN.1 data structures
    ii  libcrypt-openssl-bignum-perl   0.04-4+b2                      amd64        Perl module to access OpenSSL multiprecision integer arithmetic libraries
    ii  libcrypt-openssl-random-perl   0.04-2+b1                      amd64        module to access the OpenSSL pseudo-random number generator
    ii  libcrypt-openssl-rsa-perl      0.28-2+b1                      amd64        module for RSA encryption using OpenSSL
    ii  libcrypt-ssleay-perl           0.58-1+b2                      amd64        OpenSSL support for LWP
    ii  libdevel-cycle-perl            1.11-2                         all          Perl module to detect memory cycles in Perl objects
    ii  libdigest-hmac-perl            1.03+dfsg-1                    all          module for creating standard message integrity checks
    ii  libencode-locale-perl          1.03-1                         all          utility to determine the locale encoding
    ii  libev-perl                     4.18-1                         amd64        Perl interface to libev, the high performance event loop
    ii  libfile-chdir-perl             0.1008-1                       all          more sensible way to change directories
    ii  libfile-listing-perl           6.04-1                         all          module to parse directory listings
    ii  libfile-readbackwards-perl     1.05-1                         all          Perl module for reading a file backwards by lines
    ii  libfile-sync-perl              0.11-2+b1                      amd64        Perl interface to sync() and fsync()
    ii  libfilesys-df-perl             0.92-5+b1                      amd64        Module to obtain filesystem disk space information
    ii  libhtml-parser-perl            3.71-1+b3                      amd64        collection of modules that parse HTML text documents
    ii  libhtml-tagset-perl            3.20-2                         all          Data tables pertaining to HTML
    ii  libhtml-tree-perl              5.03-1                         all          Perl module to represent and create HTML syntax trees
    ii  libhttp-cookies-perl           6.01-1                         all          HTTP cookie jars
    ii  libhttp-daemon-perl            6.01-1                         all          simple http server class
    ii  libhttp-date-perl              6.02-1                         all          module of date conversion routines
    ii  libhttp-message-perl           6.06-1                         all          perl interface to HTTP style messages
    ii  libhttp-negotiate-perl         6.00-2                         all          implementation of content negotiation
    ii  libintl-perl                   1.23-1+deb8u1                  all          Uniforum message translations system compatible i18n library
    ii  libio-html-perl                1.001-1                        all          open an HTML file with automatic charset detection
    ii  libio-multiplex-perl           1.13-1                         all          object-oriented interface to select() for Perl
    ii  libio-socket-inet6-perl        2.72-1                         all          object interface for AF_INET6 domain sockets
    ii  libio-socket-ssl-perl          2.002-2+deb8u1                 all          Perl module implementing object oriented interface to SSL sockets
    ii  libio-stringy-perl             2.110-5                        all          Perl modules for IO from scalars and arrays
    ii  libjson-perl                   2.61-1                         all          module for manipulating JSON-formatted data
    ii  libjson-xs-perl                2.340-1+b2                     amd64        module for manipulating JSON-formatted data (C/XS-accelerated)
    ii  liblinux-inotify2-perl         1:1.22-1+b1                    amd64        scalable directory and file change notification
    ii  liblocale-gettext-perl         1.05-8+b1                      amd64        module using libc functions for internationalization in Perl
    ii  liblockfile-simple-perl        0.208-1                        all          Perl module for simple advisory file locking
    ii  liblog-agent-perl              1.000-1                        all          Perl module providing helper logging routines
    ii  liblwp-mediatypes-perl         6.02-1                         all          module to guess media type for a file or a URL
    ii  liblwp-protocol-https-perl     6.06-2                         all          HTTPS driver for LWP::UserAgent
    ii  libmath-bigint-perl            1.9993-1                       all          arbitrary size integer/float math package
    ii  libmime-base32-perl            1.02a-1                        all          Base32 encoder/decoder
    ii  libnet-dbus-perl               1.0.0-2+b2                     amd64        Perl extension for the DBus bindings
    ii  libnet-dns-perl                0.81-2+deb8u1                  amd64        Perform DNS queries from a Perl script
    ii  libnet-http-perl               6.07-1                         all          module providing low-level HTTP connection client
    ii  libnet-ip-perl                 1.26-1                         all          Perl extension for manipulating IPv4/IPv6 addresses
    ii  libnet-ldap-perl               1:0.6400+dfsg-2                all          client interface to LDAP servers
    ii  libnet-ssleay-perl             1.65-1+deb8u1                  amd64        Perl module for Secure Sockets Layer (SSL)
    ii  libperl4-corelibs-perl         0.003-1                        all          libraries historically supplied with Perl 4
    ii  libperl5.20                    5.20.2-3+deb8u9                amd64        shared Perl library
    ii  libpve-common-perl             4.0-83                         all          Proxmox VE base library
    ii  libpve-storage-perl            4.0-70                         all          Proxmox VE storage management library
    ii  librados2-perl                 1.0-3                          amd64        Perl bindings for librados
    ii  librrds-perl                   1.4.8-1.2                      amd64        time-series data storage and display system (Perl interface, shared)
    ii  libsocket6-perl                0.25-1+b1                      amd64        Perl extensions for IPv6
    ii  libstring-shellquote-perl      1.03-1                         all          quote strings for passing through the shell
    ii  libswitch-perl                 2.17-2                         all          switch statement for Perl
    ii  libterm-readline-gnu-perl      1.24-2+b1                      amd64        Perl extension for the GNU ReadLine/History Library
    ii  libtext-charwidth-perl         0.04-7+b3                      amd64        get display widths of characters on the terminal
    ii  libtext-iconv-perl             1.7-5+b2                       amd64        converts between character sets in Perl
    ii  libtext-unidecode-perl         1.22-1                         all          Text::Unidecode -- US-ASCII transliterations of Unicode text
    ii  libtext-wrapi18n-perl          0.06-7                         all          internationalized substitute of Text::Wrap
    ii  libtimedate-perl               2.3000-2                       all          collection of modules to manipulate date/time information
    ii  liburi-perl                    1.64-1                         all          module to manipulate and access URI strings
    ii  libuuid-perl                   0.05-1+b1                      amd64        Perl extension for using UUID interfaces as defined in e2fsprogs
    ii  libwww-perl                    6.08-1                         all          simple and consistent interface to the world-wide web
    ii  libwww-robotrules-perl         6.01-1                         all          database of robots.txt-derived permissions
    ii  libxml-libxml-perl             2.0116+dfsg-1+deb8u1           amd64        Perl interface to the libxml2 library
    ii  libxml-namespacesupport-perl   1.11-1                         all          Perl module for supporting simple generic namespaces
    ii  libxml-parser-perl             2.41-3                         amd64        Perl module for parsing XML files
    ii  libxml-sax-base-perl           1.07-1                         all          base class for SAX drivers and filters
    ii  libxml-sax-perl                0.99+dfsg-2                    all          Perl module for using and building Perl SAX2 XML processors
    ii  libxml-twig-perl               1:3.48-1                       all          Perl module for processing huge XML documents in tree mode
    ii  perl                           5.20.2-3+deb8u9                amd64        Larry Wall's Practical Extraction and Report Language
    ii  perl-base                      5.20.2-3+deb8u9                amd64        minimal Perl system
    ii  perl-modules                   5.20.2-3+deb8u9                all          Core Perl modules
    
    All the VM and LXC are now running, but we are quite unable to do everything else on our hosts.
    What can we do?
     
  2. dcsapak

    dcsapak Proxmox Staff Member
    Staff Member

    Joined:
    Feb 1, 2016
    Messages:
    2,524
    Likes Received:
    226
    please upgrade to the latest 4.4 with
    Code:
    apt-get update
    apt-get dist-upgrade
    
     
  3. RedBlood9

    RedBlood9 New Member

    Joined:
    Apr 16, 2018
    Messages:
    16
    Likes Received:
    0
    Hi dcsapak
    We can't do that right now.
    We are in production envoriment and can't stop the VM.
    There's anything else we can do?

    Red
     
  4. dcsapak

    dcsapak Proxmox Staff Member
    Staff Member

    Joined:
    Feb 1, 2016
    Messages:
    2,524
    Likes Received:
    226
    can you post the full error message when you do
    Code:
    journalctl -b -u pveproxy
    
     
  5. RedBlood9

    RedBlood9 New Member

    Joined:
    Apr 16, 2018
    Messages:
    16
    Likes Received:
    0
    Here the full message error:
     
  6. dcsapak

    dcsapak Proxmox Staff Member
    Staff Member

    Joined:
    Feb 1, 2016
    Messages:
    2,524
    Likes Received:
    226
    can you post the content of /usr/share/perl5/PVE/Report.pm ?
     
  7. RedBlood9

    RedBlood9 New Member

    Joined:
    Apr 16, 2018
    Messages:
    16
    Likes Received:
    0
    Here the requested file:
    Code:
    cat /usr/share/perl5/PVE/Report.pm
    package PVE::Report;
    
    use strict;
    use warnings;
    use PVE::pvecfg;
    use PVE::Tools;
    
    $ENV{'PATH'} = '/sbin:/bin:/usr/sbin:/usr/bin';
    
    my $report;
    
    my @general = ('hostname', 'pveversion --verbose', 'cat /etc/hosts', 'top -b -n 1  | head -n 15',
      'pvesubscription get', 'lscpu');
    
    my @storage = ('cat /etc/pve/storage.cfg', 'pvesm status', 'cat /etc/fstab', 'mount', 'df --human');
    
    my @volumes = ('lvs', 'vgs');
    # command -v is the posix equivalent of 'which'
    if (system('command -v zfs > /dev/null 2>&1') == 0) {
        push @volumes, 'zpool status', 'zfs list'
    }
    
    my @disks = ('lsblk --ascii');
    if (system('command -v multipath > /dev/null 2>&1') == 0) {
        push @disks, 'multipath -ll', 'multipath -v3'
    }
    
    my @machines = ('qm list', sub { dir2text('/etc/pve/qemu-server/', '\d.*conf') });
    
    my @net = ('ifconfig', 'cat /etc/network/interfaces', sub { dir2text('/etc/pve/firewall/', '.*fw') },
      'iptables-save');
    
    my @cluster = ('pvecm nodes', 'pvecm status');
    
    my @bios = ('dmidecode -t bios');
    
    if (PVE::pvecfg::version() >= 4.0) {
        push @cluster, 'cat /etc/pve/corosync.conf 2> /dev/null' ;
        push @machines, sub { dir2text('/etc/pve/lxc/', '\d.*conf') };
    } else {
        push @general, 'grep --max-count=1 "model name" /proc/cpuinfo';
        push @machines, sub { dir2text('/etc/pve/openvz/', '\d.*conf') };
        push @cluster,  'clustat', 'cat /etc/cluster.conf 2> /dev/null';
    }
    
    my $general_report = {
        title => 'general system info',
        commands => \@general,
    };
    
    my $storage_report = {
        title => 'info about storage (lvm and zfs)',
        commands => \@storage,
    };
    
    my $volume_report = {
        title => 'info about virtual machines',
        commands => \@machines,
    };
    
    my $net_report = {
        title => 'info about network and firewall',
        commands => \@net,
    };
    
    my $cluster_report = {
        title => 'info about clustering',
        commands => \@cluster,
    };
    
    my $bios_report = {
        title => 'info about bios',
        commands => \@bios,
    };
    
    my $disks_report = {
        title => 'info about disks',
        commands => \@disks,
    };
    
    my $volumes_report = {
        title => 'info about volumes',
        commands => \@volumes,
    };
    
    my @global_report = ($general_report, $storage_report, $volume_report, $net_report,
                $cluster_report, $bios_report, $disks_report, $volumes_report);
    
    # output the content of all the files of a directory
    sub dir2text {
        my ($target_dir, $regexp) = @_;
    
        PVE::Tools::dir_glob_foreach($target_dir, $regexp, sub {
       my ($file) = @_;
       $report .=  "# cat $target_dir$file\n";
       $report .= PVE::Tools::file_get_contents($target_dir.$file)."\n";
        });
    }
    
    # execute commands and display their output as if they've been done on a interactive shell
    # so the local sysadmin can reproduce what we're doing
    sub do_execute {
        my ($command) = @_;
        $report .= "# $command \n";
        open (COMMAND, "$command 2>&1 |");
        while (<COMMAND>) {
       $report .= $_;
        }
    }
    
    sub generate {
        foreach my $subreport (@global_report) {
       my $title = $subreport->{'title'};
       my @commands = @{$subreport->{'commands'}};
    
       $report .= "\n==== $title ====\n";
       foreach my $command (@commands) {
           if (ref $command eq 'CODE') {
           &$command;
           } else {
           do_execute($command);
           next;
           }
       }
        }
    return $report;
    }
    
     
  8. dcsapak

    dcsapak Proxmox Staff Member
    Staff Member

    Joined:
    Feb 1, 2016
    Messages:
    2,524
    Likes Received:
    226
    did anything happen out of the ordinary when it stopped working ?
    i just tested here and a fresh installation with the same versions as you, i have no problem
     
  9. RedBlood9

    RedBlood9 New Member

    Joined:
    Apr 16, 2018
    Messages:
    16
    Likes Received:
    0
    No, nothing in particular.
    The only thing changed are the following, but i've already done a rollback:
    - disabled ipv6
    - enabled auditd
    - make persistante journald
    - configured new ntp server
    - reconfigured resolv.conf
    installed this packages: htop, wget, netcat, net-tools, dnsutils, screen, pwgen, expect, mailutils, mlocalte, rsync, bash-completition, snmpd, telnet, traceroute, ftp, zip, unzip, vim, xauth, zabbix-agent

    All these action are done with ansible, and the playbook is already tested on other machine
     
  10. RedBlood9

    RedBlood9 New Member

    Joined:
    Apr 16, 2018
    Messages:
    16
    Likes Received:
    0
    For info,
    We've noticed that even the packes auditd and audispd-plugins we're installed, and doing the same on a test server, broken pve.
    We've removed the two packages, but the situation did not resolv.

    How can we resolv the problem?
     
  11. dcsapak

    dcsapak Proxmox Staff Member
    Staff Member

    Joined:
    Feb 1, 2016
    Messages:
    2,524
    Likes Received:
    226
    do you have any special config for those packages? i installed them here on a test machine, and it still works without any problems...
     
  12. RedBlood9

    RedBlood9 New Member

    Joined:
    Apr 16, 2018
    Messages:
    16
    Likes Received:
    0
    Yes,
    here's the config:
    Code:
    # Managed by Ansible
    #
    # This file contains the auditctl rules that are loaded
    # whenever the audit daemon is started via the initscripts.
    # The rules are simply the parameters that would be passed
    # to auditctl.
    
    # First rule - delete all
    -D
    
    # Increase the buffers to survive stress events.
    # Make this bigger for busy systems
    -b 8192
    
    # Feel free to add below this line. See auditctl man page
    
    -a exit,always -F arch=b64  -F euid=0 -S execve -k root-activity
    -a exit,always -F arch=b32  -F euid=0 -S execve -k root-activity
    
     
  13. dcsapak

    dcsapak Proxmox Staff Member
    Staff Member

    Joined:
    Feb 1, 2016
    Messages:
    2,524
    Likes Received:
    226
    still works here with that config.. so it has to be something else..
     
  14. RedBlood9

    RedBlood9 New Member

    Joined:
    Apr 16, 2018
    Messages:
    16
    Likes Received:
    0
    there's something more I can pass you?
    We need to resolve the problem as soon as possibile
     
  15. dcsapak

    dcsapak Proxmox Staff Member
    Staff Member

    Joined:
    Feb 1, 2016
    Messages:
    2,524
    Likes Received:
    226
    mhmm.. sorry i am out of ideas, maybe you could try to reproduce it while trying to do one thing at a time (what your ansible config does)
     
  16. RedBlood9

    RedBlood9 New Member

    Joined:
    Apr 16, 2018
    Messages:
    16
    Likes Received:
    0
    just a question
    After installing audit, have you tried restart the pveproxy service or the server?
    I've tried the same playbook in a test server and that killed pve
     
  17. dcsapak

    dcsapak Proxmox Staff Member
    Staff Member

    Joined:
    Feb 1, 2016
    Messages:
    2,524
    Likes Received:
    226
    yes, i tried both
     
  18. RedBlood9

    RedBlood9 New Member

    Joined:
    Apr 16, 2018
    Messages:
    16
    Likes Received:
    0
    Has maybe pve particolar file for ntp.conf, .bashrc, resolv.conf, .profile or something else?
     
  19. RedBlood9

    RedBlood9 New Member

    Joined:
    Apr 16, 2018
    Messages:
    16
    Likes Received:
    0
    In the log we've found that there was on update of these files:
    ii perl 5.20.2-3+deb8u9 amd64 Larry Wall's Practical Extraction and Report Language
    ii perl-base 5.20.2-3+deb8u9 amd64 minimal Perl system
    ii perl-modules 5.20.2-3+deb8u9 all Core Perl modules

    from version
    ii perl 5.20.2-3+deb8u6 amd64 Larry Wall's Practical Extraction and Report Language
    ii perl-base 5.20.2-3+deb8u6 amd64 minimal Perl system
    ii perl-modules 5.20.2-3+deb8u6 all Core Perl modules

    Could be this?
     
  20. RedBlood9

    RedBlood9 New Member

    Joined:
    Apr 16, 2018
    Messages:
    16
    Likes Received:
    0
    In the test hosts, after the reboot we're receiving these errors:
    ‚óŹ pvedaemon.service - PVE API Daemon
    Loaded: loaded (/lib/systemd/system/pvedaemon.service; enabled; vendor preset: enabled)
    Active: failed (Result: exit-code) since Mon 2018-04-16 16:58:58 CEST; 6s ago
    Process: 38011 ExecStart=/usr/bin/pvedaemon start (code=exited, status=255)
    CPU: 1.059s

    Apr 16 16:58:58 pve pvedaemon[38011]: Compilation failed in require at /usr/share/perl5/PVE/API2.pm line
    Apr 16 16:58:58 pve pvedaemon[38011]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/API2.pm l
    Apr 16 16:58:58 pve pvedaemon[38011]: Compilation failed in require at /usr/share/perl5/PVE/Service/pveda
    Apr 16 16:58:58 pve pvedaemon[38011]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Service/p
    Apr 16 16:58:58 pve pvedaemon[38011]: Compilation failed in require at /usr/bin/pvedaemon line 11.
    Apr 16 16:58:58 pve pvedaemon[38011]: BEGIN failed--compilation aborted at /usr/bin/pvedaemon line 11.
    Apr 16 16:58:58 pve systemd[1]: pvedaemon.service: Control process exited, code=exited status=255
    Apr 16 16:58:58 pve systemd[1]: Failed to start PVE API Daemon.
    Apr 16 16:58:58 pve systemd[1]: pvedaemon.service: Unit entered failed state.
    Apr 16 16:58:58 pve systemd[1]: pvedaemon.service: Failed with result 'exit-code'.


    What are the error about? What exactly fails to compile?
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice