I think this is a client side problem. I noticed this with Firefox 15 on ubuntu 12.04 64bit:
1. Login to PVE
2. Select node, click shell and it will open normally.
3. Close Shell.
4. Select KVM and open Console.
5. Go back to node and open shell.
6. You will get error: could not connect ot...