[TUTORIAL] HOWTO: Proxmox host serial console (UEFI-boot), useful when passthrough your only GPU

[jena]

The original ideas are from:
https://forum.proxmox.com/threads/serial-port-on-host-to-proxmox-console.81045/
https://help.ubuntu.com/community/SerialConsoleHowto
https://tldp.org/HOWTO/Remote-Serial-Console-HOWTO/configure-kernel.html
https://www.rogerirwin.co.nz/open-source/enabling-a-serial-port-console/

The OP and Ubuntu tutorial used GRUB boot.
However, I cannot find definitive answers for the UEFI-boot equivalent.

Hardware requirement:
1. Proxmox host: DTE serial port (see Wikipedia-DataTerminalEquipment )
2. USB to Null Modem Serial Adapter Cable (which connects to a DTE device, i.e. your Proxmox host) The one worked for me is Tripp Lite U209-18N-NULL
3. Another computer or a laptop

Proxmox host side
1.
# For a USB serial adaptor
systemctl enable serial-getty@ttyUSB#.service

# For a built in serial port /dev/ttyS0
systemctl enable serial-getty@ttyS#.service

tty0 is your default virtual console, i.e. video card output
ttyS# is your COM port.
If your motherboard has onboard COM port, it is likely to be COM1 (ttyS0);
Like mine, adding a PCIE-serial card, use lspci -v to find the PCIE card hardware id.
For USB-serial port: ttyUSB#
For example:
44:00.0 Serial controller: MosChip Semiconductor Technology Ltd. MCS9900 Multi-I/O Controller

then dmesg | grep tty to find the ttyS#
It matches PCIE 44:00 hardware id

[    0.000000] printk: console [tty0] enabled
[    0.######] serial8250: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[    0.######] 0000:44:00.0: ttyS4 at I/O 0x4000 (irq = 66, base_baud = 115200) is a ST16650V2
[    1.######] tty ttyS6: hash matches

2.
To prevent bricking anything, I first tried to edit a temporary kernel/cmdline by press e when grub boot version select appears.
To have permanent setting, edit
(uefi boot) /etc/kernel/cmdline instead of (legacy boot) /etc/default/grub

add at the end
console=tty0 console=ttyS#,115200n8

3. pve-efiboot-tool refresh and reboot

The other computer
1. Install putty
2. connect the usb serial null cable to both Proxmox host and this computer
3. In Device Manager, find COM port number set baud rate to 115200, leave other default (default: parity=none, data bits=8, stop bits=1, flow control=none)
4. In putty, type in COM port and baud rate and connect.
5. If you connect before boot, putty will print out kernel messages. If you connect after boot or reconnect putty, it might show blank, just press enter, then login should show up.

 

[ronron555]

Hey, what pci-e card did you use?

 

[jena]

Hey, what pci-e card did you use?

Proxmox side: PCIE: StarTech PEX1S553LP
Laptop side: Tripp Lite U209-18N-NULL

 

[leesteken]

I always add a virtual serial console to VMs with GPU passthrough (in a very similar way) but thank you for showing how to do it for the Proxmox host.

 

[sbellon]

Thanks for your write-up. Actually the step 1 is not required as systemd automatically creates the serial-getty for login if the console=ttyS# kernel boot parameter is present. I just did your step 2 and 3 and everything works fine.

However one question regarding the order of the two console parameters. The kernel documentation states:

You can specify multiple console= options on the kernel command line.
Output will appear on all of them. The last device will be used when
you open ``/dev/console``.

Concerning this information, shouldn't we put console=tty0 at the end to be compatible with default behavior?

 

[HankFlaggerty]

Is there an updated version of this guide for Proxmox VE 8? Getting SOL to work is important. Thanks so much!

 

[ks94]

Is there an updated version of this guide for Proxmox VE 8? Getting SOL to work is important. Thanks so much!

I have tested that this still works in Proxmox VE 8.1.4. To summarize what I have done differently:

1. No need to explicitly enable the services using systemctl in step 1 (As mentioned in #5)
2. Put console=tty0 at the end of the kernel command line instead (as suggested in #5)
3. Use proxmox-boot-tool refresh instead as it is mentioned in official docs, however I believe it should essentially be the same thing as pve-efiboot-tool

 

[SInisterPisces]

EDIT 2, Part 1: I got it working. See below.

Thanks so much for this.

It's not quite working for me yet, and was a bit more complex to get half-working.

I used this for my cmdline, as I have a USB device sitting at ttyUSB0.

# dmesg -T | grep -i tty
[Tue Apr  9 14:17:48 2024] Command line: initrd=\EFI\proxmox\6.5.13-1-pve\initrd.img-6.5.13-1-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt console=ttyUSB0,115200n8 console=tty0
[Tue Apr  9 14:17:48 2024] Kernel command line: initrd=\EFI\proxmox\6.5.13-1-pve\initrd.img-6.5.13-1-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt console=ttyUSB0,115200n8 console=tty0
[Tue Apr  9 14:17:48 2024] printk: console [tty0] enabled
[Tue Apr  9 14:17:48 2024] 0000:00:16.3: ttyS4 at I/O 0x3088 (irq = 19, base_baud = 115200) is a 16550A
[Tue Apr  9 14:17:48 2024] tty tty26: hash matches
[Tue Apr  9 14:17:51 2024] systemd[1]: Created slice system-getty.slice - Slice /system/getty.
[Tue Apr  9 14:17:51 2024] systemd[1]: Created slice system-serial\x2dgetty.slice - Slice /system/serial-getty.
[Tue Apr  9 14:17:51 2024] systemd[1]: Expecting device dev-ttyUSB0.device - /dev/ttyUSB0...
[Tue Apr  9 14:17:51 2024] usb 1-7: FTDI USB Serial Device converter now attached to ttyUSB0

# cat /etc/kernel/cmdline
root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt console=ttyUSB0,115200n8 console=tty0

I did find that for my FTDI USB-C adapter, Proxmox didn't automatically create and start serial-getty@ttyUSB0.service. Manually enabling it and starting it as provided above seemed to be necessary to get it to appear. I suspect this is because at least on this machine the USB ports don't activate early enough in the boot process for whatever auto-generates the services to catch the USB adapter (notice how it says "expecting device," even when I force it?), but I don't have any way to test that.

At any rate, here's what my service looks like:

# systemctl status serial-getty@ttyUSB0.service
● serial-getty@ttyUSB0.service - Serial Getty on ttyUSB0
     Loaded: loaded (/lib/systemd/system/serial-getty@.service; enabled; preset: enabled)
     Active: active (running) since Tue 2024-04-09 14:12:43 CDT; 3min 35s ago
       Docs: man:agetty(8)
             man:systemd-getty-generator(8)
             https://0pointer.de/blog/projects/serial-console.html
   Main PID: 2739 (agetty)
      Tasks: 1 (limit: 37546)
     Memory: 196.0K
        CPU: 1ms
     CGroup: /system.slice/system-serial\x2dgetty.slice/serial-getty@ttyUSB0.service
             └─2739 /sbin/agetty -o "-p -- \\u" --keep-baud 115200,57600,38400,9600 - vt220

I've got the Proxmox server's serial adapter plugged into a serial terminal server, and I do see activity on that serial line, but I've yet to be able to get a login prompt.

So, either Proxmox isn't fully configured somehow, or I need to tweak a setting on my terminal server.

Does anything in the above output suggest any issues to y'all?

EDIT: Well, I rebooted the Proxmox node and noticed I now have tty1. Maybe that's the Proxmox web console?

root@andromeda2:~# systemctl status getty
getty-pre.target      getty-static.service  getty.target          getty@tty1.service 
root@andromeda2:~# systemctl status getty@tty1.service
● getty@tty1.service - Getty on tty1
     Loaded: loaded (/lib/systemd/system/getty@.service; enabled; preset: enabled)
     Active: active (running) since Tue 2024-04-09 14:47:27 CDT; 23min ago
       Docs: man:agetty(8)
             man:systemd-getty-generator(8)
             https://0pointer.de/blog/projects/serial-console.html
   Main PID: 1093 (agetty)
      Tasks: 1 (limit: 37546)
     Memory: 284.0K
        CPU: 1ms
     CGroup: /system.slice/system-getty.slice/getty@tty1.service
             └─1093 /sbin/agetty -o "-p -- \\u" --noclear - linux

EDIT 2, Part 2: I got it working after some googling about getty and auto-negotiation.

I ran into a post from someone who wanted to lock down the buad rate offered by their server, and demonstrated how to override the the default getty service settings. I figured this might be the issue, as my terminal server isn't configured to try to auto-negotiate at all.

Steps/Example, Using the Name of My Serial Device Service, which is Enabled and Started/Running:

mkdir /lib/systemd/system/serial-getty@ttyUSB0.service.d/
nano override.conf #should contain only lines 1-3 below

[CODE]
───────┬─────────────────────────────────────────────────────────────────
       │ File: override.conf
───────┼─────────────────────────────────────────────────────────────────
   1   │ [Service]
   2   │ ExecStart=
   3   │ ExecStart=-/sbin/agetty -o '-p -- \\u' 115200 %I $TERM
───────┴─────────────────────────────────────────────────────────────────

You could further override the agetty command however you want.

 

[motoridersd]

I'm having issues getting the console output to work. I can't get the PVE 8 installer to show any console output, but a Debian 12 installer works fine.

I installed PVE over Debian 12, but as soon as I run the PVE Kernel, I lose console access. GRUB has the console=ttyS0,115200n8 line for the default entry, but I get no output. Adding console=tty0 does nothing.

I had SSH access but then locked myself out when making the wrong network change, so now I can't get back in. I'm guessing my problem is similar to the last post where the serial service isn't getting started. My serial port is built-in. I have no video card in this device.

 

[SInisterPisces]

I'm having issues getting the console output to work. I can't get the PVE 8 installer to show any console output, but a Debian 12 installer works fine.

I installed PVE over Debian 12, but as soon as I run the PVE Kernel, I lose console access. GRUB has the console=ttyS0,115200n8 line for the default entry, but I get no output. Adding console=tty0 does nothing.

I had SSH access but then locked myself out when making the wrong network change, so now I can't get back in. I'm guessing my problem is similar to the last post where the serial service isn't getting started. My serial port is built-in. I have no video card in this device.

Hello,

The above tutorial is aimed at getting serial access to Proxmox after it's installed. Just to clarify, you're trying to use the serial terminal based installer that's available as an option on the boot ISO? If so, you might get better visibility on this question with a new thread.

Just in case, have you looked at this entry?
https://pve.proxmox.com/wiki/Installation

I think what is happening is that it's bringing you to the white screen where you are supposed to select a type of installation to continue, so on a keyboard at the physical server you'd need to press down 1 or 2 times to select the terminal installer option, which would then set up the serial port and start the thing.

I'm not sure if there's a way to get the ISO to automatically choose the serial terminal mode. I really think you'd get better visibility on this with a top level topic.

 

[motoridersd]

Hello,

The above tutorial is aimed at getting serial access to Proxmox after it's installed. Just to clarify, you're trying to use the serial terminal based installer that's available as an option on the boot ISO? If so, you might get better visibility on this question with a new thread.

Just in case, have you looked at this entry?
https://pve.proxmox.com/wiki/Installation

I think what is happening is that it's bringing you to the white screen where you are supposed to select a type of installation to continue, so on a keyboard at the physical server you'd need to press down 1 or 2 times to select the terminal installer option, which would then set up the serial port and start the thing.

I'm not sure if there's a way to get the ISO to automatically choose the serial terminal mode. I really think you'd get better visibility on this with a top level topic.

It's failing for me in both instances. I see the menu to install, but once the installer kicks off, I lose the console. Same after booting into the Proxmox kernel, so it seems to me like the Proxmox kernel is not able to output its console to the embedded serial port of this box (ttyS0) for some reason.

The agetty service doesn't start for me when trying to follow the procedure in this thread.

 

[SInisterPisces]

Something strange is happening here. This sounds more and more like a bug somewhere, likely related to your specific hardware.

Could you report a bug here, with as much information about your system as you can get?
(Maybe boot into another Linux ISO and run inxi -Fazy. If inxi is not installed, you can use apt within the Live ISO to pull it. It'll install onto a RAM disk.)

I suspect you can still install Proxmox on your system, but we need to figure out how to tell the installer to use your specific serial card correctly, which I don't understand how to do at all.

In the meantime, you might also want to ask this question on the Proxmox Reddit and at Level1Techs.
I've had good luck in both of those communities with super-niche Proxmox issues.

 

[Jordy8267]

Thanks it worked for me with Intel AMT on a HP EliteDesk 800 G4 SFF.


I have set the /etc/default/grub to the following


This is my dmesg | grep tty

root@lab-pve-01:~# dmesg | grep tty
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-6.8.12-4-pve root=ZFS=/ROOT/pve-1 ro root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction initcall_blacklist=sysfb_init video=simplefb:off video=vesafb:off video=efifb:off video=vesa:off disable_vga=1 vfio_iommu_type1.allow_unsafe_interrupts=1 kvm.ignore_msrs=1 modprobe.blacklist=radeon,nouveau,nvidia,nvidiafb,nvidia-gpu,snd_hda_intel,snd_hda_codec_hdmi,i915 console=tty0 console=ttyS4,115200n8
[    0.050801] Kernel command line: BOOT_IMAGE=/vmlinuz-6.8.12-4-pve root=ZFS=/ROOT/pve-1 ro root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction initcall_blacklist=sysfb_init video=simplefb:off video=vesafb:off video=efifb:off video=vesa:off disable_vga=1 vfio_iommu_type1.allow_unsafe_interrupts=1 kvm.ignore_msrs=1 modprobe.blacklist=radeon,nouveau,nvidia,nvidiafb,nvidia-gpu,snd_hda_intel,snd_hda_codec_hdmi,i915 console=tty0 console=ttyS4,115200n8
[    0.147741] printk: legacy console [tty0] enabled
[    0.428187] 00:06: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[    0.431733] printk: legacy console [ttyS4] disabled
[    0.452869] 0000:00:16.3: ttyS4 at I/O 0x3088 (irq = 19, base_baud = 115200) is a 16550A
[    0.452896] printk: legacy console [ttyS4] enabled
[    5.725785] systemd[1]: Created slice system-getty.slice - Slice /system/getty.
[    5.726735] systemd[1]: Created slice system-serial\x2dgetty.slice - Slice /system/serial-getty.
[    5.727560] systemd[1]: Expecting device dev-ttyS4.device - /dev/ttyS4...
[    6.130127] cdc_acm 1-1:1.0: ttyACM0: USB ACM device