ProxVE 1.1->1.2 upgrade; KVM win2003 VM won't start now ?

[fortechitsolutions]

Hi,

I just finished an update on two ProxVE servers, in a 'cluster' config

I have one Win2003 KVM virtual server on one of these hosts (1 virtual cpu, 1 gig ram) which has run smoothly in the past. It was shut down prior to doing the upgrade on ProxVE.

Basic issue: I can start the VM but I can't get the VNC console; it tells me, "Network error: Could not connect to server, LOCAL_IP_ADDRESS:5900"

I can't ping nor RDP into the windows VM, and I'm not convinced it is started cleanly. If I migrate it to the other ProxVE machine in the cluster, the behaviour is the same.

If I create another (new from scratch) KVM virtual machine, then power it on and try to connect to VNC console - I see the same behaviour.

In contrast, I have a number of OpenVZ based VM's on these ProxVE servers and they are running fine, and I can get to "VNC Console" for them smoothly.

I did a similar upgrade of ProxVE hardware late last week, also in cluster config, also updating from ver 1.1 to 1.1, and also with Win2003 VM's -- and had no problems there. I'm unclear what is so different now that makes things 'a hassle'.

but any thoughts on how/where to proceed with debug would certainly be greatly appreciated.

Checking the "monitor" tab for the Win2003 KVM based VM .. appears to indicate that stuff is happening; memory is allocated ; etc. But I'm not familiar enough with KVM to dig in any more / see how else to connect / see where logs are that may give me hints as to why it is choked.

But right now I'm stuck, and really could use a pointer on how to proceed with debug on this. I certainly would like to get this VM online "asap".


Thanks,


Tim

 

[fortechitsolutions]

Small footnote on this,

after a bit more digging in the forums, I found a post on VNC access testing, and did the following,

- on console of the ProxVE host where the Win2003 VM was running:

issue as root, the command:

nc -l -p 5900 -c "qm vncproxy 108 PASSWORD"


Then from a nearby system, called up an instance of vncviewer, pointed it at the IP of the ProxVE server in question / supplied 5900 as the port, and then bingo, got a password prompt. Gave the password, bingo, I have my Win2003 console awaiting login.

After login it was clear that Win2003 believed there was new hardware detected. In particular it found a new VirtIO network interface, usurping the prior instance of such.

I manually configured the static IP on the new interface, then ignored the warning that this same IP was already assigned to another interface (that just no longer happens to be physically present in the system at the moment) - and then bingo,I can ping out or in to the win2003 server.

Alas, I still can not get my normal web-management-interface based VNC console via the usual ProxmoxVE web interface. I'm not sure what is up with this.

But at least I'm happy that I was able to verify, that my VM was running fine; it was just inaccessible due to NIC hardware changes (? part of the ProxVE platform update I guess - but not something I had expected...) - and so the critical things are dealt with for now.

But certainly if there is any more feedback on how to proceed with restoring normal VNC admin console access to my Win2003 KVM based VM .. that info certainly would be massively appreciated.


Many thanks,


Tim Chipman

 

[fortechitsolutions]

small footnote on this, in case it helps.

Via web interface, in the logs section, I can see the following errors reported:


Jun 3 12:51:31 pvedaemon 23950 starting vnc proxy UPID:23950-5872804:1244044291:vncproxy:2:108:root:5900:BkdO61gLy5XP8C0zXsxk7De7lxI
Jun 3 12:51:31 pvedaemon 23950 CMD: /bin/nc -l -p 5900 -w 30 -c /usr/bin/ssh -T -o BatchMode=yes 10.10.2.12 /usr/sbin/qm vncproxy 108 BkdO61gLy5XP8C0zXsxk7De7lxI 2>/dev/null
Jun 3 12:51:31 pvedaemon 23950 VM 108 vnc proxy failed - -1


Based on this logged error, I attempt on root console of the ProxVE host, the same command which appears to have failed:

pvm2:~# /bin/nc -l -p 5900 -w 30 -c /usr/bin/ssh -T -o BatchMode=yes 10.10.2.12 /usr/sbin/qm vncproxy 108 BkdO61gLy5XP8C0zXsxk7De7lxI

BatchMode=yes: forward host lookup failed: Unknown host

pvm2:~# ping 10.10.2.12
PING 10.10.2.12 (10.10.2.12) 56(84) bytes of data.
64 bytes from 10.10.2.12: icmp_seq=1 ttl=64 time=0.014 ms


At least we appear to have an error now - a problem with Name Resolution ?

I have checked, on the proxmoxVE host, it can do pings to outside world / name resolution of remote or local hosts appears to be fine.

I'm not sure if this extra info is of any use. I'll try a bit more debug with this shortly, but wanted to put the extra detail here in the forum 'promptly'.


--Tim

 

[fortechitsolutions]

Final footnote,

if I issue the command as root,

pvm2:~# nc -l -p 5900 -c "ssh -T 10.10.2.12 /usr/sbin/qm vncproxy 108 pass"


then from another host I'm able to point a command-line vncviewer client against 10.10.2.12 port 5900 and I get a win2003 console once I provider my "pass" password

so - this suggests to me; something is odd with the command syntax being used for the vnc proxy startup .. as per the proxVE web interface .. on this particular system (?)

Rather odd.

But any comments are appreciated.


--Tim

 

[dietmar]

You are using the wrong version of netcat (nc)? We need the netcat-traditional package (please remove netcat-bsd or any other version). Does that help?

 

[fortechitsolutions]

Hi,

I don't think I have the wrong version installed. Certainly not something I've done intentionally. But below is my 'diagnostic work' so far:

---paste---


pvm2:~# which nc
/bin/nc

pvm2:~# ls -la /bin/nc
lrwxrwxrwx 1 root root 20 2009-06-02 20:05 /bin/nc -> /etc/alternatives/nc

pvm2:~# ls -la /etc/alternatives/nc
lrwxrwxrwx 1 root root 19 2009-06-02 20:05 /etc/alternatives/nc -> /bin/nc.traditional

pvm2:~# ls -la /bin/nc.traditional
-rwxr-xr-x 1 root root 23992 2008-06-19 18:34 /bin/nc.traditional

pvm2:~# /bin/nc.traditional -h
[v1.10-38]
connect to somewhere: nc [-options] hostname port [ports] ...
listen for inbound: nc -l -p port [-options] [hostname] [port]
options:

...etc... truncated here for brevity...

---endpaste----


So it looks to me like I'm calling nc.traditional


Additionally, I'm not sure that I have other versions installed. For example:

pvm2:~# apt-get remove netcat-openbsd
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package netcat-openbsd is not installed, so not removed
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
pvm2:~#



Is there anything else that springs to mind which I should be seeking here ?


Thanks for the pointers ..


Tim

 

[dietmar]

The question is why does it work anywhere else. What is the output of

# pveversion -v

Do you have any local modifications?

 

[dietmar]

so - this suggests to me; something is odd with the command syntax being used for the vnc proxy startup .. as per the proxVE web interface .. on this particular system (?)

no, the log shows the correct command line (only the quotes are missing in the log).

does the folowing work:

# /bin/nc -l -p 5900 -w 30 -c "/usr/bin/ssh -T -o BatchMode=yes 10.10.2.12 /usr/sbin/qm vncproxy 108 BkdO61gLy5XP8C0zXsxk7De7lxI"

 

[fortechitsolutions]

Hi,

The output is pasted below.

The only local modification is adjustment to networking script to enable jumbo frames (MTU 9000). Other than that, there is no customization on the ProxVE host.

Tim

-----paste-------

pvm2:~# pveversion -v
pve-manager: 1.2-1 (pve-manager/1.2/3982)
qemu-server: 1.0-12
pve-kernel: 2.6.24-6
pve-kvm: 85-1
pve-firmware: 1
vncterm: 0.9-2
vzctl: 3.0.23-1pve3
vzdump: 1.1-1
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pvm2:~#

 

[fortechitsolutions]

Hi,

Just to confirm,

the snippet as you provided works perfectly. Once is has been called, I'm able to manually launch VNCViewer from a nearby host, and then access the VM VNC session for VMID 108 once I give the proper/specified password.

Thanks,

Tim