Can't connect to PVE9 consoles [solved]

R

Red Squirrel

Renowned Member
May 31, 2014
71
12
73
I recently moved my VMs over to a new PVE 9.2.3 instance on an OVH server, was able to access the consoles initially, but now I can't access any of them, it just sits at connecting forever and never connects. For a while I also couldn't connect to the web interface at all as it would just hang loading forever. Restarting pveproxy service solved that particular issue, but still can't connect to the VM consoles.

Anything else I can try to get this to work? I also ran apt update and apt upgrade but no luck.

Oh and my home IP is allowed to connect to any port on the server as I have a rule setup in iptables, so don't think it's firewall related.

I don't have physical access to this server so rebooting is not an option, as I'm worried it might not come back up properly and given it's a weekend I would be SOL until at least Monday. My VMs are still running fine, just can't access the console.


A bit more info from troubleshooting. When I do systemctl status pveproxy I get a bunch of this error:

Code: 
EV: error in callback (ignoring): connect to 'localhost:5900' failed: Connection timed out at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 579

This code seems to be failing:

/usr/share/perl5/PVE/APIServer/AnyEvent.pm Line 579
Code: 
        tcp_connect $remhost, $remport, sub {
            my ($fh) = @_
                or die "connect to '$remhost:$remport' failed: $!";

I'm not familiar with perl but I believe this means the arguments are not being sent to the function for some reason. What would cause this?
 
Last edited:
Just tried that but it did not change anything. Still can't connect to any consoles. Host or VMs. At one point I could connect to one specific VM but it was a fluke, it stopped working now.

This is what I get in journald any time I try:

Code: 
May 30 12:40:15 sr05 pvedaemon[1724311]: connection timed out
May 30 12:40:15 sr05 pvedaemon[1552917]: <root@pam> end task UPID:[id]:vncproxy:100:root@pam: connection timed out

Edit: removed the ID part, just in case it's something that shouldn't be public.
 
Last edited:
EDIT: Got it working!

Nothing makes you panic troubleshoot more than when all sites go down... Had to add this to my firewall script:

Code: 
ip6tables -A INPUT -s ::1/128 -j ACCEPT

I started wondering if the issue was IPv6 related and started exploring that. I guess the console uses IPv6 internally and not IPv4 so have to allow that traffic through. What I wonder is why I was not finding any of this info in my search... I can't be the only one that has run into this.


Previous post:
Well this is bad now...

I thought maybe it was an Ipv6 issue so I removed that entry from /etc/network/interfaces file. Now I can't connect to any of my VMs. I see they are running but it's not passing traffic. I am willing to pay someone to help, I need this up asap.

I put the IPv6 info back but it's still not working, I am completely dead in the water now. This is a brand new install I don't get what's going on. How do I get this working again?

I just rebooted because I had nothing to lose. Thankfully the VMs can pass traffic now, but still in the same spot I was. The consoles do not open.
 
Last edited:
It works now but in case someone runs into this, here's my firewall script :

ipv4:
Code: 
#!/usr/bin/env iptables-restore
*filter
:FORWARD DROP [0:0]
:INPUT DROP [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT


#allow localhost:
-A INPUT -s 127.0.0.0/8 -j ACCEPT

#my local IP
-A INPUT -s x.x.x.x -j ACCEPT


#ports to allow
#-------------------------------

-A INPUT -i lo -j ACCEPT

-A INPUT -p tcp --dport 4248 -j ACCEPT

COMMIT


ipv6:

Code: 
#!/bin/bash
echo "Applying ipv6 firewall rules..."

#allow only local traffic:
ip6tables -A INPUT -s ::1/128 -j ACCEPT

#block all ipv6 traffic
ip6tables -P INPUT DROP


The allow only local traffic rule is what fixed it.

These scripts run at startup.
 
You must log in or register to reply here.
Top Bottom
Back