Serial port support for VMs

[jdw]

While doing some research on the possibility of setting up Xen-style serial consoles for KVM virtual machines on Proxmox, I found that past comments on the subject by Proxmox folks are basically, "There is ssh and there is VNC. There is no use-case for anything else."

The big obvious problem with this is that I am doing this research because the VNC used by Proxmox requires Java. Setting aside the inherent security flaws of Java in the browser that a wider and wider contingent of IT departments are solving by just not allowing Java in the browser, I was just been burned by another failed Java update that left the old version unusable. (Known issue; any attempt to run Java just forwards to the update page.) Thanks Oracle! Was unable to log into a deadlocked VM for over an hour while trying to unf*** my Java install. Thank goodness it was not a production machine.

Maybe there is some way to tease out what machine & port VNC is running on for a particular VM, so a non-DOA client could connect to it, but I don't know what it is. The command line to kvm makes it look like it's running against a Unix domain socket, which will be pretty tough to connect to from a remote server without help.

So I would just like to add a vote for the integrated support in the web gui for adding virtualized serial ports on VM's and making them easy to find from the command line.

Because Java.

And because, unlike VNC, serial terminals are quite useful for e.g. kernel debugging.

And because, unlike VNC, serial terminals can be automated against with Expect and similar tools for operations like OS updates performed in single user mode.

Please reconsider your position on this; there are many valid use-cases for serial support. These are just some of them.

 

[jdw]

Perhaps I should add that after I did eventually get Java working again, it takes multiple "Don't run this!" popups to get the console running, including one that reads:

Block potentially unsafe components from being run?

Application: com.tigervnc.vncviewer.VncViewer

"Java has discovered application components that could indicate a security concern. Contact the application vendor to ensure that it has not been tampered with." The application contains both signed and unsigned code.

It's basically saying, "This specific app, the VNC Viewer from Proxmox? Yeah, don't run that, it's probably malware."

Fortunately I know better, but stuff like that doesn't make the Java-in-the-browser requirement for console access any more appealing.

 

[dietmar]

We will support spice in future: http://pve.proxmox.com/wiki/SPICE

 

[jdw]

As a console solution, that might be good for many people. (Although I don't think SPICE options on Mac are very good, though I have never had occasion to try.)

Nonetheless, that won't help configure and manage serial ports on virtual machines, which as I hoped to illustrate has many useful purposes that won't be served by SPICE any better than VNC.

 

[dietmar]

Nonetheless, that won't help configure and manage serial ports on virtual machines, which as I hoped to illustrate has many useful purposes that won't be served by SPICE any better than VNC.

So what do you want to do exactly? You can already map a serial device into a VM (for example 'serial0: /dev/ttyS2')

 

[jdw]

It would be desirable to add serial ports via the UI, using virtual serial ports, not physical ones. There are not enough physical ones to go around.

So something like: -serial unix:/path/to/serials/123.tty,server,nowait

Then some command line equivalent to "xm console 123." Maybe "pxeconsole 123." Doesn't need to be minicom or anything carried away like that. Just something that transparently passes characters and lets the terminal at my end do the emulation and listens for an escape sequence to exit. Would have used "tip" or "cu" in the old days, but these things don't seem to come with Linux these days. I believe "screen" can do it in a pinch and has other useful functionality.

Of course Javascript terminals also exist that could connect to the serial port from the web UI, and that would be really slick and compatible with every modern HTML+JS browser on Earth. That might be valuable to many people, but honestly I could live without it, because while I personally own many devices that can't Java, I own none that can't ssh.

 

[jdw]

Oh, and we haven't upgraded to 3.0 yet, so I haven't played with templates at all, but the ability to have virtual serial ports created automatically with unique, predictable file names when cloning a template would be fantastic as well.

As long as the unique, predictable name is based on the VM id, "pxeconsole" becomes a one-line wrapper around screen at that point.

 

[jdw]

As I think about it, the "-serial telnet:localhost:1234,server,nowait" option might work better than Unix domain sockets, because telnet is already present in the default install and it makes sending breaks (important for kernel debugging) more straightforward/standardized.

Port assignments would be harder to clone and track, though, which is why I shied away from it initially.

 

[dietmar]

So something like: -serial unix:/path/to/serials/123.tty,server,nowait

Then some command line equivalent to "xm console 123."

This only works if the VM spawn a getty on the console, so this does not work by default (needs special config inside VM)?

 

[jdw]

This only works if the VM spawn a getty on the console, so this does not work by default (needs special config inside VM)?

In some cases, it does. Although when required the config is typically minimal.

On physical hardware, FreeBSD and some Linux distros have it enabled by default and can be installed from CD-ROM using only serial console, though this sometimes requires BIOS serial console support which you may not have.

We use PXE boot and our network install images are preconfigured to support both serial and VGA console. I believe some OS's also auto-fallback to the serial port if there is no VGA device present. (Deleting VGA entirely would be a nice option to save some small resource, but might be farther than you are willing to go at this time.) Solaris, I think, falls into this category.

Kernel debugging over the serial port of course requires extensive configuration. (In that case typically the serial port cannot also be used as console or have a getty.)

But not in any way requesting you make whatever guest OS config is needed to make use of the serial port. Merely that you make usable serial ports easy to obtain and track via the web UI.

 

[dietmar]

But not in any way requesting you make whatever guest OS config is needed to make use of the serial port. Merely that you make usable serial ports easy to obtain and track via the web UI.

Played around, and it is quite easy to extend the serial configuration like (I have a patch for that):

serial0: socket

This will add a serial device to the guest, and listens to a unix socket on the host (/var/run/qemu-server/<VMID>.serial0).

You can then connect to that using minicom:

# minicom -D unix\#/var/run/qemu-server/<VMID>.serial0

Not sure if that is what you want?

 

[dietmar]

You can then connect to that using minicom:

# minicom -D unix\#/var/run/qemu-server/<VMID>.serial0

Not sure if that is what you want?

using socat will also work (we use control-O to terminate socat)

# socat UNIX-CONNECT:/var/run/qemu-server/<VMID>.serial0 STDIO,raw,echo=0,escape=0x0f

 

[jdw]

Played around, and it is quite easy to extend the serial configuration like (I have a patch for that):

serial0: socket

This will add a serial device to the guest, and listens to a unix socket on the host (/var/run/qemu-server/<VMID>.serial0).

You can then connect to that using minicom:

# minicom -D unix\#/var/run/qemu-server/<VMID>.serial0

Not sure if that is what you want?

When you say "serial0: socket" where in the UI would this be applied?

It doesn't look like Unix domain sockets are the best approach. It's not clear if it's possible to send a serial BRK over a Unix domain socket (or generate one from e.g. socat, minicom can probably do that). The KVM -serial telnet backend can definitely do this. It's quite important functionality for debugging, as it is used to stop the running system and break into the debugger.

Likewise, depending on minicom is quite a heavyweight solution. Telnet is already there.

Thanks!

 

[dietmar]

The KVM -serial telnet backend can definitely do this. It's quite important functionality for debugging, as it is used to stop the running system and break into the debugger.

Likewise, depending on minicom is quite a heavyweight solution. Telnet is already there.

This is the wrong place to discuss development related things. Please can you join the pve-devel mailing list instead?

http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

 

[dietmar]

Deleting VGA entirely would be a nice option to save some small resource, but might be farther than you are willing to go at this time.) Solaris, I think, falls into this category.

Just implemented 'qm terminal <vmid>" command using socat, and added the ability to run a VM without real vga card (vga: serial[n]) - Seems to work great.

 

[ebiss]

What is the benefit run a VM without real vga card (vga: serial[n]) ?

 

[dietmar]

What is the benefit run a VM without real vga card (vga: serial[n]) ?

just saves resources (and it was easy to implement).

 

[bert64]

Very useful, thanks for implementing this.
I was previously adding a serial port manually using args: and a bunch of minicom rc files (one for each vm), but this had its drawbacks - eg cloning the image didn't change the socket name etc...

Any ETA on when these changes will make their way to pvetest?

 

[dietmar]

Any ETA on when these changes will make their way to pvetest?

usually within a week.

 

[bert64]

It appears that setting the console to a serial port fails unless i define a serial port in the config, which i have done manually by adding "serial0: socket".. Is there any way to add a serial port through the web interface?