[TUTORIAL] Adding Full Disk Encryption to Proxmox

[esi_y]

I described it here more detailed:
https://forum.proxmox.com/threads/recommend-way-for-encryption.148614/post-704287

tl;dr;
It boots and prompts for decryption-passphrase
when i try to connect with ssh while this, there is "no route to host".
But the prompt on the machine says the ip is 192.168.1.4, gw is 192.168.1.254 netmask is 255.255.255.0, as i configured.

I can hit 3 times enter or write the passphrase for luks on the prompt, the it boots until login (this is only a testdrive for encryption, not the root-device).

I have on the same machine in dualboot a pure debian bookworm beside. initramfs is configured with dropbear-initramfs exactly the same.

There i have NO prompt for password on the local machine. I have to login via ssh to dropbear from remote, fill in the luks passphrase, then the boot-process continues.

It looks like dropbear does not start and/or the network configuration for initramfs does not work at all...

Alright, I will have a look into the linked thread (a bit later) and see if something comes to my mind. NB If you do not insist on the dropbear, you could be also using something like tang/clevis for your purpose. But will take it to there then.

 

[e071cddf]

Here is a logfile from

update-initramfs -u

 

[esi_y]

Here is a logfile from

update-initramfs -u

Are you positive your hooks do not need chmod +x? (I probably should read the tutorial here too, I guess ...)

 

[e071cddf]

there are 2 scripts i tried (found them in an 5 year old thread, describing my problem.... but they did not work, so i removed the executable bit.

all other stuff is original from proxmox installation.

I found a video about tang/clevis. Never heard about before.
https://www.ogselfhosting.com/index.php/2023/12/25/tang-clevis-for-a-luks-encrypted-debian-server/

It sound very comfortable... yes.

BUT

I want to decrypt my device manually on boot. Why?
It is my personal nas at home, where i want to run some services (nextcloud, vaultwarden....) and when it got stolen (by a thief or the police) i dont want them to decrypt the data, when they plug in into the net and my tang-servers are reachable... It sounds... hmmm comfortable...
[edit]
ok... i've just read it tooo fast... if i configure two or more tang servers (somewhere remote and some of them in the local network), ALL of them have to be reachable. And if a thief or the police takes my homeserver (or my laptop) the are not connected to the local tang-server anymore. So automatic decryption won't work anymore... sounds very better
[/edit]

My intention is to startup the NAS-Box (Zimablade with proxmox) manually and unlock it via ssh from my laptop, which i can only connect with a pkcs11 smartcard/key (yubikey FIPS), or put in the very long password on the prompt, which is stored in a passwordstore...

On the other hand... how should it work with tang/clevis when the network is not ready? As it is the same problem i've now with dropbear...
For asking the tang-servers i have to have network on initramfs-level to connect them.... you know, what i mean?

 

[pmra]

when i try to connect with ssh while this, there is "no route to host".

FWIW: "no route to host" is a network issue on the client/network you connect from or an intermediate network, not an initramfs issue on the target.

 

[e071cddf]

It must be a problem of initramfs. Why?

When i ping the machine i get no answer until i unlocked the disk (or just hit 3x enter to continue without unlocking it), and when initram has finished and the system starts, i get answers from the machine and can reach it from remote.

 

[e071cddf]

Alright, I will have a look into the linked thread (a bit later) and see if something comes to my mind. NB If you do not insist on the dropbear, you could be also using something like tang/clevis for your purpose. But will take it to there then.

So, i setup a tang/clevis for this machine... and as i said... no network in initramfs-stage... it does not work.

 

[gfngfn256]

It must be a problem of initramfs. Why?

Not sure if this will help, but are you aware of the latest dropbear/initramfs directory structure:

From the source (that OP used) :

A note about config file locations:
The latest version of Debian (such as 12+), Ubuntu (22.04 LTS+), Linux Mint, and Pop!_OS uses the following new version config files and directories:


New Directory: /etc/dropbear/initramfs/
New config file: /etc/dropbear/initramfs/dropbear.conf
New files containing public keys for public key authentication: /etc/dropbear/initramfs/authorized_keys
The Older version (such as Debian 11 or Ubuntu 20.04 LTS) used the following config files:


Old Directory: /etc/dropbear-initramfs/
Old config file: /etc/dropbear-initramfs/config
Old files containing public keys for public key authentication: /etc/dropbear-initramfs/authorized_keys
Please update paths according to your Linux distro version.

Please note I don't use dropbear at all.

 

[e071cddf]

So, i setup a tang/clevis for this machine... and as i said... no network in initramfs-stage... it does not work.

Yes, I'm aware of this.
ans proxmox installer gives me the same debian as the pure-debian installer.
So i did the same on both installs.

But now... i tried somehow to get it work. And... tang/clevis works on proxmox but dropbear not.

Now i have to figure out, how to do a good setup for several tang-server...

I tested this now with only one if it works. But this is a public reachable one, secured with ssl and only one only for testing purpose.

Thanks for the hint. It's a good one

There is also a fallback, when no network is reachable (to unlock a laptop...). The proxmox-Server is not the best idea to host a tang-server... because it should decrypt itself from itself... maybe run a small device (raspi or something else) in the LAN elsewhere at home.

But it would be great, if someone else can verify, that with the current proxmox-installation the dropbear solution wont work in case of network issues.

 

[e071cddf]

Again... it does not work.

Bootprocess comes to "Please unlock disk sda4_crypt: Begin: clevis: Waiting for interface enp2s0 to become available... done.
Begin: Waiting up to 180 secs for enp2s0 to become available... done

Then is the network config printed and the prompt waits for the password, forever... not only 180sec.

But now arp -a gives me ip domain/hostname and mac-address... so why is network not working?

 

[esi_y]

Again... it does not work.

Bootprocess comes to "Please unlock disk sda4_crypt: Begin: clevis: Waiting for interface enp2s0 to become available... done.
Begin: Waiting up to 180 secs for enp2s0 to become available... done

Then is the network config printed and the prompt waits for the password, forever... not only 180sec.

But now arp -a gives me ip domain/hostname and mac-address... so why is network not working?

Is there ANY chance you have an IP conflict on the network segment? You know, the silly thing with PVE is that it pushes hard its static IP idea, it is quite common here to troubleshoot for hours something that is simply forgotten IP that was statically assigned that also resides in DHCP pool to be sometimes to be given out.

 

[e071cddf]

Oh my goosh...
I found the problem...

I've setup tang/clive along this blogpost https://www.ogselfhosting.com/index.php/2023/12/25/tang-clevis-for-a-luks-encrypted-debian-server/

There is a hook-script for curl, which sets a "nameserver 1.1.1.1" entry into initramfs /etc/resolv.conf

It's named as "fix for Debian 11+12"

So... local resolving did not work in initramfs... public resolving works... now it's clear.

The problem behind is, the clive-scripts from debian bookworm for initramfs ignore dns completely.
It ignores dns from the IP=... in initramfs.conf and it ignores dns delivered by dhcp if set IP=::::::dhcp

installing clevis with apt on debian 12 installes this script

/usr/share/initramfs-tools/scripts/local-top/clevis and in this script is the function do_configure_networking()

It calles configure_networking from scripts/functions.
i added the 3 lines at the end to set IPVeDNS[01] and DNSDOMAIN at the end of configure_networking. (in the main-script from debian itself!!!)
According to manpage is the domain entry in resolv.conf deprecated for a search entry with only one domain. So i set it to "search" instead of "domain" in resolv.conf

        # source ipconfig output
        if [ -n "${DEVICE}" ]; then
                # source specific bootdevice
                . "/run/net-${DEVICE}.conf"
        else
                # source any interface...
                # ipconfig should have quit after first response
                . /run/net-*.conf
        fi

        [ -n "${IPV4DNS0:-}" ] && echo "nameserver ${IPV4DNS0}" >> /etc/resolv.conf
        [ -n "${IPV4DNS1:-}" ] && echo "nameserver ${IPV4DNS1}" >> /etc/resolv.conf
        [ -n "${DNSDOMAIN:-}" ] && echo "search ${DNSDOMAIN}" >> /etc/resolv.conf
}

What does the code?
The function "do_configure_networking" is called, after "configure_networking", which is a standard-function in initramfs-tools for scripts.
configure_networking sources /run/net-<nic>.conf for each network-device. And after IP_CONFIG is the nameserver set, which is brought along with dhcp.

I deleted the workaround in the curl-hook from the tutorial which sets nameserver to 1.1.1.1 because dhcp from local network gives me the nameserver(s) for the LAN... there is all the resolving done, which is needed.

I've tested this hack, and it works.
What I've not done is testing it, when dns-entries are set in "IP=::::" in initramfs.config

What a bad hack needed for Debian

 

[esi_y]

Oh my goosh...
I found the problem...

I've setup tang/clive along this blogpost https://www.ogselfhosting.com/index.php/2023/12/25/tang-clevis-for-a-luks-encrypted-debian-server/

There is a hook-script for curl, which sets a "nameserver 1.1.1.1" entry into initramfs /etc/resolv.conf

It's named as "fix for Debian 11+12"

So... local resolving did not work in initramfs... public resolving works... now it's clear.

The problem behind is, the clive-scripts from debian bookworm for initramfs ignore dns completely.
It ignores dns from the IP=... in initramfs.conf and it ignores dns delivered by dhcp if set IP=::::::dhcp

installing clevis with apt on debian 12 installes this script

/usr/share/initramfs-tools/scripts/local-top/clevis and in this script is the function do_configure_networking()

It calles configure_networking from scripts/functions.
i added the 3 lines at the end to set IPVeDNS[01] and DNSDOMAIN at the end of configure_networking. (in the main-script from debian itself!!!)
According to manpage is the domain entry in resolv.conf deprecated for a search entry with only one domain. So i set it to "search" instead of "domain" in resolv.conf

        # source ipconfig output
        if [ -n "${DEVICE}" ]; then
                # source specific bootdevice
                . "/run/net-${DEVICE}.conf"
        else
                # source any interface...
                # ipconfig should have quit after first response
                . /run/net-*.conf
        fi

        [ -n "${IPV4DNS0:-}" ] && echo "nameserver ${IPV4DNS0}" >> /etc/resolv.conf
        [ -n "${IPV4DNS1:-}" ] && echo "nameserver ${IPV4DNS1}" >> /etc/resolv.conf
        [ -n "${DNSDOMAIN:-}" ] && echo "search ${DNSDOMAIN}" >> /etc/resolv.conf
}

What does the code?
The function "do_configure_networking" is called, after "configure_networking", which is a standard-function in initramfs-tools for scripts.
configure_networking sources /run/net-<nic>.conf for each network-device. And after IP_CONFIG is the nameserver set, which is brought along with dhcp.

I deleted the workaround in the curl-hook from the tutorial which sets nameserver to 1.1.1.1 because dhcp from local network gives me the nameserver(s) for the LAN... there is all the resolving done, which is needed.

I've tested this hack, and it works.
What I've not done is testing it, when dns-entries are set in "IP=::::" in initramfs.config

What a bad hack needed for Debian

Excellent! The only thing you lost me with is how you got from No route to host issue to no DNS.

 

[e071cddf]

Oh my goosh...
I found the problem...

I've setup tang/clive along this blogpost https://www.ogselfhosting.com/index.php/2023/12/25/tang-clevis-for-a-luks-encrypted-debian-server/

There is a hook-script for curl, which sets a "nameserver 1.1.1.1" entry into initramfs /etc/resolv.conf

It's named as "fix for Debian 11+12"

So... local resolving did not work in initramfs... public resolving works... now it's clear.

The problem behind is, the clive-scripts from debian bookworm for initramfs ignore dns completely.
It ignores dns from the IP=... in initramfs.conf and it ignores dns delivered by dhcp if set IP=::::::dhcp

installing clevis with apt on debian 12 installes this script

/usr/share/initramfs-tools/scripts/local-top/clevis and in this script is the function do_configure_networking()

It calles configure_networking from scripts/functions.
i added the 3 lines at the end to set IPVeDNS[01] and DNSDOMAIN at the end of configure_networking. (in the main-script from debian itself!!!)
According to manpage is the domain entry in resolv.conf deprecated for a search entry with only one domain. So i set it to "search" instead of "domain" in resolv.conf

        # source ipconfig output
        if [ -n "${DEVICE}" ]; then
                # source specific bootdevice
                . "/run/net-${DEVICE}.conf"
        else
                # source any interface...
                # ipconfig should have quit after first response
                . /run/net-*.conf
        fi

        [ -n "${IPV4DNS0:-}" ] && echo "nameserver ${IPV4DNS0}" >> /etc/resolv.conf
        [ -n "${IPV4DNS1:-}" ] && echo "nameserver ${IPV4DNS1}" >> /etc/resolv.conf
        [ -n "${DNSDOMAIN:-}" ] && echo "search ${DNSDOMAIN}" >> /etc/resolv.conf
}

What does the code?
The function "do_configure_networking" is called, after "configure_networking", which is a standard-function in initramfs-tools for scripts.
configure_networking sources /run/net-<nic>.conf for each network-device. And after IP_CONFIG is the nameserver set, which is brought along with dhcp.

I deleted the workaround in the curl-hook from the tutorial which sets nameserver to 1.1.1.1 because dhcp from local network gives me the nameserver(s) for the LAN... there is all the resolving done, which is needed.

I've tested this hack, and it works.
What I've not done is testing it, when dns-entries are set in "IP=::::" in initramfs.config

What a bad hack needed for Debian

I changed my comment. The hack for creating /etc/resolv.conf must go to scripts/functions
And with this hack, no configuration for "DEVICE=" or adding an "IP=..." line in /etc/initramfs-tools/initramfs.config is needed.

I use it now as it comes with the installation.
If nothing is configured there, it takes the first device which gives a dhcp-answer and the resolving and clive also dropbear works.

 

[e071cddf]

Excellent! The only thing you lost me with is how you got from No route to host issue to no DNS.

Oh my dear...

I lost debian on the same point

I think it was some kind of bad configuration of the NIC in case of IP=... entry in initramfs.config. So the NIC did not get configured and so no NIC was listening...

And then the host was reachable from outside.

I read the blogpost again and again to get the thing with 1.1.1.1 dns... then it was fully clear, debian does shit in initramfs.
How it looks, ubuntu does it better. But i have to have a look in ubuntu, how the script is there.

I'm really glad to got it solved!

 

[e071cddf]

Checked the code in ubuntu...

YEAH. There is a function netinfo_to_resolv_conf() in scripts/functions which writes the infos from the /run/net-<nic>.conf to resolv.conf.
Debians initramfs-tools are definitely missing them.

And Ubuntus initramfs-tools handles ipv6 also, which debians are also lacking.

 

[esi_y]

I read the blogpost again and again to get the thing with 1.1.1.1 dns... then it was fully clear, debian does shit in initramfs.
How it looks, ubuntu does it better. But i have to have a look in ubuntu, how the script is there.

I'm really glad to got it solved!

That's too bad, I was about to suggest clevis-dracut to compare with.

 

[e071cddf]

I've just seen, that in the next version i debian is the function als included

https://salsa.debian.org/kernel-team/initramfs-tools/-/blob/v0.145/scripts/functions?ref_type=tags

0.145 for initramfs-tools.

Sounds good.

 

[dmpm]

I've got a self-encrypting Crucial MX500 SSD and I've encrypted the root partition using cryptsetup luksFormat --hw-opal-only /dev/sda2 (sda1 being reserved for a 1G EFI boot partition) but I had to use Ubuntu for this, as Debian 12 has an older version of cryptsetup that doesn't support --hw-opal-only and I spent half a day trying to work out how to update it to v 2.7+ before giving up and using Ubuntu instead.

I just checked on my current Proxmox machine and cryptsetup on that is 2.7.5 even though it's based on Debian.

So I think I should be able to follow this guide by booting into Ubuntu rather than Debian, as then I'll be sure that I can unlock my drive with cryptsetup to copy the old installation across, and when I boot from the new drive hopefully Proxmox will be able to unlock the root partition.

 

[esi_y]

I've got a self-encrypting Crucial MX500 SSD and I've encrypted the root partition using cryptsetup luksFormat --hw-opal-only /dev/sda2 (sda1 being reserved for a 1G EFI boot partition) but I had to use Ubuntu for this, as Debian 12 has an older version of cryptsetup that doesn't support --hw-opal-only and I spent half a day trying to work out how to update it to v 2.7+ before giving up and using Ubuntu instead.

I just checked on my current Proxmox machine and cryptsetup on that is 2.7.5 even though it's based on Debian.

So I think I should be able to follow this guide by booting into Ubuntu rather than Debian, as then I'll be sure that I can unlock my drive with cryptsetup to copy the old installation across, and when I boot from the new drive hopefully Proxmox will be able to unlock the root partition.

That's a really nice find! I completely missed it, SED support now from cryptsetup!
https://lore.kernel.org/all/cd409f6c-5d51-482c-8a26-340822754ff1@gmail.com/T/

I am literally curious how that will work out for you now.