[TUTORIAL] HiveStation - Developer Workstation for Proxmox Virtual Environment

A

aureladmin

Renowned Member
Apr 15, 2016
72
35
83

HiveStation - Developer Workstation for Proxmox Virtual Environment​


HiveStation is a community project that automates the full installation of a developer workstation built on Proxmox VE 9, on top of Debian 13 (Trixie): the hypervisor, a choice of disk layouts, bootable snapshots, an optional desktop, themes and a GNOME monitoring widget - installed in a reproducible sequence of scripts (S1 to S8) you can read and audit.

It is inspired by the Proxmox VE wiki article [Developer Workstations with Proxmox VE and X11](https://pve.proxmox.com/wiki/Developer_Workstations_with_Proxmox_VE_and_X11): the idea of running a full desktop on the very machine where Proxmox VE runs your VMs and containers.

*HiveStation is an independent community project. It is not affiliated with, nor endorsed by, Proxmox Server Solutions GmbH. Proxmox is a registered trademark of Proxmox Server Solutions GmbH.*

Current version: V21.3.5 - thin orchestrators S1-21V9.sh ... S8-21V1.sh plus shared libraries (lib/) and runtime files shipped as real files (payload/), all messages bilingual FR/EN. The full step-by-step guide (HIVESTATION-V21-GUIDE-EN.md) is included in the archive.

What it provides​


- A Proxmox VE 9 hypervisor on Debian 13 (installed via debootstrap).
- Five disk modes, chosen at S1 launch (see table below) - Btrfs, LVM+EXT4, and since V21 ZFS as a first-class filesystem (single and mirror).
- Bootable snapshots: Snapper + grub-btrfs in Btrfs modes (roll back from the GRUB menu), Timeshift in LVM mode, and in ZFS modes Boot Environments with sanoid timelines - every snapshot bootable from the ZFSBootMenu menu, plus an APT hook that snapshots the Boot Environment before each operation.
- ZFS boot chain: themed GRUB on the ESP chains into ZFSBootMenu, which kexecs the Boot Environment - Boot Environments, snapshots and kernels selectable at boot; GRUB never probes the pool.
- Recovery from the boot menu: the "HiveStation Recovery" family - snapshots submenu and "Proven Kernel" entry in Btrfs modes, the full ZFSBootMenu menu (Boot Environments, snapshots, kernels, pool status) in ZFS modes.
- VM/LXC networking: vmbr0 bridge with dnsmasq DHCP and dynamic NAT toward the WAN interface (ethernet or Wi-Fi).
- Optional desktop: GNOME, KDE, XFCE, LMDE7 Cinnamon or COSMIC, with Flatpak + Flathub.
- Optional firmware & GPU: AMDGPU, Intel Wi-Fi, Realtek, NVIDIA (open modules recommended for Turing+, or proprietary legacy), optional XanMod kernel.
- Optional themes & tools: GRUB theme, adaptive Plymouth theme, asusctl/supergfxctl, GNOME Cinnamon-like layout, an optional Cockpit + ZFS manager GUI on ZFS installs, and a GNOME monitoring widget (CPU/mem/temps, SMART, storage health - Btrfs or ZFS pool - disk I/O, WAN uplink, Proxmox status, threshold notifications) whose privileged reads go through a single least-privilege root broker (one sudoers rule, per-verb allowlist; ZFS reads need no privilege at all). Drop a logo.png next to S8 to brand every theme with your own logo.
- Upgrade path: hivestation-upgrade.sh brings an existing V15-V20 install up to date without reinstalling (inventory-driven, pre-upgrade restore point, --dry-run, --rollback); an existing V21 ZFS install is refreshed in place. Adopting ZFS remains a fresh install (no conversion).
- Archive self-test: hivestation-check.sh validates the archive before you deploy it (manifest, anti-residue, runtime-contract coherence).

Disk modes (chosen at S1 launch)​


| Mode | Disks | Filesystem | System snapshots | High availability |
|---|---|---|---|---|
| single | 1 | Btrfs (single/dup) | Snapper + grub-btrfs | - |
| RAID1 | 2 identical | Btrfs RAID1 | Snapper + grub-btrfs | dual-ESP sync, SWAP failover, automatic disk replacement |
| LVM+EXT4 | 1 (64 GB or more) | ext4 on LVM (thin pool for VM/CT) | Timeshift (RSYNC) | - |
| ZFS single | 1 | ZFS pool (Boot Environments) | sanoid + bootable BE snapshots | - |
| ZFS mirror | 2 identical | ZFS mirrored pool | sanoid + bootable BE snapshots | dual-ESP sync, native degraded boot, guided disk replacement |

Install workflow​


| Script | Phase | Runs from | Reboot after? |
|---|---|---|---|
| S1 | Debian base (Btrfs, LVM or ZFS) + boot chain + network | Debian Live | No - run S2 first |
| S2 | HA scripts (RAID1 / ZFS mirror) + SSH | Debian Live (same session) | Yes |
| S3.1 | Proxmox kernel + PVE subvolumes (Btrfs) / dataset checks (ZFS) | Installed Debian | Yes |
| S3.2 | Proxmox VE (+ storage/snapshots choice, optional PCIe passthrough) | Installed Debian (PVE kernel) | Yes |
| S4 | Graphical desktop + Flatpak (optional) | Installed Proxmox | If a desktop was installed |
| S5 | Snapshots: Snapper+grub-btrfs (Btrfs), Timeshift (LVM) or sanoid + ZFS health (ZFS) | Installed Proxmox | Recommended |
| S6 | Firmware + NVIDIA (optional) | Installed Proxmox | If firmware was installed |
| S7 | vmbr0 + DHCP + NAT + boot tuning | Installed Proxmox | Yes |
| S8 | Themes, tools, monitoring widget, ZFS GUI (menu, optional) | Installed Proxmox | Recommended |

Requirements​


- 64-bit CPU (x86_64); VT-x/AMD-V recommended for nested virtualization.
- RAM: 4 GB minimum, 16 GB or more recommended. ZFS modes: budget the ARC cache on top of the VM needs (S1 asks for the cap, default 2 GiB).
- Disk: 1 disk (single, ZFS single, or 64 GB+ for LVM+EXT4), or 2 identical SSD/NVMe for RAID1 / ZFS mirror.
- UEFI firmware. Secure Boot is supported in the Btrfs/LVM modes; ZFS modes need Secure Boot off (the install-time boot chain carries an unsigned DKMS-built module).
- One ethernet or Wi-Fi interface. Internet access is required before running S1 (debootstrap downloads Debian; ZFS modes compile the module in the Live session, 5-10 minutes on the first run).

Optional disk encryption, TPM2 and Secure Boot​


- LUKS2/argon2id encryption is opt-in in the three Btrfs/LVM modes: cleartext Btrfs /boot + LUKS2 under the filesystem, passphrase typed once at the themed Plymouth prompt (keyboard layout honoured), bootable snapshots preserved. An encrypted RAID1 boots even with a missing disk (degraded initramfs unlock). Validated on real hardware in the Btrfs modes (RAID1 end to end, Secure Boot and hardware TPM 2.0 included); the LVM+LUKS variant is not yet validated on real hardware.
- Native ZFS encryption is opt-in in both ZFS modes: aes-256-gcm on root/home/var, kernels inside the encrypted pool, one passphrase at the themed ZFSBootMenu prompt (keyboard layout honoured - FR and DE proven on real hardware), LUKS2 swap outside the pool. Use ASCII-only passphrases in ZFS modes. Hibernation stays EXPERIMENTAL (firmware-dependent).
- TPM2 auto-unlock (opt-in): hivestation-tpm2.sh bind enrols the LUKS2 containers via clevis so the disks unlock without the passphrase; the passphrase always remains as fallback. Validated on a hardware TPM 2.0 as well as on a vTPM.
- Active Secure Boot is supported in the Btrfs/LVM modes (S3.1 installs proxmox-secure-boot-support so the PVE kernel boots) - validated on real hardware, including a full encrypted RAID1 install under Secure Boot with TPM2 auto-unlock. On NVIDIA machines the locked-down kernel rejects the DKMS modules - disabling Secure Boot in the BIOS is the fallback.

All of it is documented in detail in the guide.

Warning - destructive procedure​


S1 completely wipes the selected disk(s) - one disk in single/LVM/ZFS-single mode, two disks in RAID1 / ZFS mirror. All existing data is lost. Double-check the disk selection when prompted.

Why Btrfs or ZFS​


Btrfs is in-kernel and gives subvolume snapshots that grub-btrfs exposes directly in the GRUB menu, so a bad update is a one-reboot rollback; RAID1 mirrors two disks with checksummed self-heal, without an out-of-tree module. ZFS brings Boot Environments (the whole root is a bootable, rollbackable unit), native encryption, and the zfspool storage - Proxmox's production storage - with VM and CT snapshots out of the box. LVM+EXT4 is there for those who prefer the classic Proxmox layout (LVM-thin for VMs, Timeshift for the system).

Download and usage​


The archive is attached to this post: HIVESTATION-V21-COMPLETE.tar.gz (V21.3.5).

1. Extract it, read HIVESTATION-V21-GUIDE-EN.md (full step-by-step guide, EN).
2. Boot the target machine on a Debian 13 (Trixie) Live ISO, copy the scripts over.
3. Run S1 then S2 from the Live, reboot, then S3.1 -> S8 from the installed system.
4. Already on V15-V20? Run hivestation-upgrade.sh instead of reinstalling.

Feedback, bug reports and questions are welcome in this thread.
 

Attachments

Last edited:
  • Like
Reactions: c.dent, testermox and christophe
Why btrfs instead of zfs? Wouldn't it make sense to be able to use ZFS-only features?
 
Johannes S said:
Why btrfs instead of zfs? Wouldn't it make sense to be able to use ZFS-only features?
Click to expand...
I've been trying to do this with ZFS for a while now, but I'm running into bugs that aren't currently solvable. For example, zfsbootmenu doesn't work when used with the pre-compiled Void EFI image; no snapshot is visible at boot due to modifications made by the Proxmox VE installation. Creating the EFI image from scratch makes it even worse. Zfsbootmenu was developed primarily for Void; it works with other distributions, but I haven't found a solution for this particular problem.

Another point: I'm using this configuration on a laptop, and ZFS is extremely slow at startup, shutdown, and during daily use. The CPU load is between 5 and 10% higher. The same issue with RAM persisted. Even after trying to modify the ARC cache to minimize RAM usage, I experienced crashes. I had to allocate a minimum of 4GB to ARC for it to function, so ultimately, whithout any VM, RAM usage quickly climbed to around 8GB.

I couldn't get the laptop to boot even if a disk failed. If you use ZFS with Proxmox tools, it won't boot if the boot disk is faulty; you have to use the command line to resolve the problem. Without Proxmox tools, it's also impossible because once the ZFS partition is degraded, there's no way to boot from a mirror (it works on a RAIDZ1 array with three disks, but not in a mirror configuration, I don't know why).

There are other issues as well, but basically, that's why I chose BTRFS instead of ZFS.

If someone manages to solve the three biggest problems I mentioned, I would obviously use ZFS, but currently it's not possible in my case.

The reason I started this project, in case anyone asks, is that I had a problem with an SSD in late 2025 (both the boot and system SSD), and I wasted time reinstalling everything (no data loss).
 
Last edited:
I tried using the scripts. I got stuck after selecting the disks: ./S1-17V0.sh: line 378 sgdisk: command not found
gdisk is already installed

Edit: The failure was to run the script as root without sudo. Succeeded with sudo bash S1-17V0.sh
 
Last edited:
Update to V18 — three disk modes (single Btrfs, RAID1 Btrfs, LVM + EXT4)

What's new (highlights):
  • Three disk modes selectable at S1 launch: single Btrfs, RAID1 Btrfs (HA), or LVM + EXT4.
  • LVM mode: VG pve (ext4 root + swap + LVM-thin pool declared as local-lvm); system snapshots via Timeshift.
  • Monitoring widget is now disk-mode aware (Btrfs single/RAID1 or LVM/EXT4; Snapper or Timeshift).
  • GNOME Cinnamon-like: Nemo is now optional (keep Nautilus, install Nemo, or revert).
  • S5 creates a baseline snapshot so you can validate the snapshot chain before the first reboot.

V18 is a significant update. The biggest change is that the installer is no longer limited to Btrfs RAID1: at S1 launch you now choose between three disk modes. A single marker file (/etc/hivestation/install.conf) written by S1 is the single source of truth, and every following script (S2 → S8) adapts to the selected mode automatically.

Disk modes (chosen in S1)

ModeDisksLayoutSnapshotsHA
Single1Btrfs -d single -m dup (label prox_single)Snapper + grub-btrfsNo
RAID12Btrfs -d raid1 -m raid1 (label prox_raid1)Snapper + grub-btrfsFull (EFI sync, SWAP failover, auto-replace)
LVM + EXT41VG pve = root (ext4) + swap + LVM-thin "data"LVM-thin (VMs) + Timeshift (system)No

New: LVM + EXT4 mode

For those who prefer the classic Proxmox LVM layout over Btrfs, V18 adds a single-disk LVM + EXT4 mode:

  • EFI partition + one LVM PV; volume group pve containing root (ext4, label prox_lvm), swap, and a data LVM-thin pool for VM/CT disks.
  • The thin pool is automatically declared as the Proxmox local-lvm storage (S3.2).
  • VM/CT snapshots are handled natively by the LVM-thin pool. System (root) snapshots use Timeshift in RSYNC mode, installed automatically by S5 (since Snapper and grub-btrfs are Btrfs-only).
  • GRUB is configured with the lvm module preloaded; no degraded flag, no subvol rootflags.

Every Btrfs-specific step (PVE subvolumes, Display Manager subvolume isolation, Snapper, grub-btrfs) now detects a non-Btrfs root, prints a clear warning, and skips cleanly without aborting the script. So the graphical environments (GNOME, KDE, XFCE, LMDE7 Cinnamon, COSMIC) install normally in LVM mode too — only the optional Btrfs subvolume isolation is skipped.

Other improvements

  • S5 baseline snapshot — in Btrfs mode, S5 now creates an initial baseline snapshot and regenerates grub-btrfs.cfg at the end of setup, so you can validate the full Snapper + grub-btrfs + boot-menu chain before the first reboot.
  • GNOME Cinnamon-like — Nemo is now optional. A single menu offers: (1) keep Nautilus as default (theme + extensions still applied), (2) install Nemo and set it as default, or (3) revert to default Nautilus (file manager only, with an option to uninstall Nemo).
  • Monitoring widget v2 — disk-mode aware. The GNOME overlay now adapts its storage panel to the disk mode: "Btrfs single" / "Btrfs RAID1" / "LVM / EXT4" (with an LVM-thin usage bar), and shows Snapper or Timeshift snapshots accordingly.
  • Robustness fixes — reworked DHCP detection in S1, marker-based architecture for cross-script consistency, more robust grub-btrfs.cfg regeneration, and a snapshot-count fix in S5.

The rest of the workflow is unchanged: vmbr0 + dynamic NAT, NVIDIA driver options (open kernel modules recommended), optional XanMod kernel, PCI(e) passthrough, custom GRUB/Plymouth themes, and asusctl/supergfxctl.

Version

Current version: V18
All scripts remain bilingual (French / English). A full English guide is included in the archive.

As always, feedback and test reports are very welcome.

Download: see attachment below (HIVESTATION-V18-COMPLETE.tar.gz).
 

Attachments

Do you support disk encryption?
Unfortunately the PVE installer does not, so instead of generating a PVE-autoinstall iso, with first-boot-script, I usually install the Debian by hand and PVE afterwards.
For a server that is absolutely fine, but for my portable developmemt workstations (aka Laptop) that would be a super nice feature.
 
bl1mp said:
Do you support disk encryption?
Unfortunately the PVE installer does not, so instead of generating a PVE-autoinstall iso, with first-boot-script, I usually install the Debian by hand and PVE afterwards.
For a server that is absolutely fine, but for my portable developmemt workstations (aka Laptop) that would be a super nice feature.
Click to expand...
It's planned for later, but I still need to validate the two methods I have in testing with all the possibilities of file systems so as not to have to maintain too many different methods and I also need to think about future developments other than those I have planned, zfs for example on which I am stuck at the moment.
Unfortunately, I only have one laptop on which I can perform physical tests and it is my main computer; I have to take it apart every time, which takes me a lot of time.
 
Last edited:

HiveStation V19 - what's new since V18​


V19 is out (current build V19.16). Archive attached: HIVESTATION-V19-COMPLETE.tar.gz. Existing V15-V18 installs can move up with hivestation-upgrade.sh (no reinstall); a fresh install follows S1 -> S8 as usual. All scripts stay bilingual FR/EN, and the full guide is in the archive.

Reliability - Btrfs health arbitrated by the scrub. New hivestation-btrfs-health service + timer (Btrfs modes): one health verdict decided by the scrub, not by the raw btrfs device stats counters. This fixes a V18 case where old, already-repaired counters showed a false "MISSING DISK"/degraded alarm and a blocking scrub could fire on every boot. Plymouth (theme v2V7) and the widget now read the published state; auto-replace reacts only to a genuinely missing disk.

Plymouth theme v2V7 - splash panel matches the disk mode. The boot splash no longer always lists the RAID1 module set. The disk mode is baked into the theme before the initramfs rebuild, so the panel is correct even at the pre-unlock passphrase prompt: LVM shows an LVM/EXT4 row and hides the Btrfs-only rows, single hides the RAID1 HA rows, RAID1 keeps the full panel.

Monitoring widget v9. HiveStation branding, overlap-free split mode; WAN uplink name + type (Wi-Fi/RJ45/USB); vendor-neutral sensors (CPU freq/temp with fallbacks, battery, platform profile, SMART, disk I/O, Btrfs health line, GPU panels hidden when absent); collapsible sections; GNOME threshold notifications (rate-limited); precompiled translations (gettext no longer needed at install time); clean removal of the old extension name.

Network - interfaces reconciler. New hivestation-iface-reconcile (installed by S7) enforces "NetworkManager owns the physical WAN/mobility cards". Creating a Proxmox bridge - even one with no slave port - can no longer strand the physical uplink. Physical WAN cards stay under NetworkManager (DHCP client by default); vmbr0 keeps its static IP and its dnsmasq server. To pin a fixed WAN address, set it NetworkManager-side (nmcli).

Tooling. hivestation-upgrade.sh upgrades V15-V18 without reinstalling: inventory-driven, marker reconstruction, pre-upgrade restore point, per-session manifest, --dry-run, --rollback. hivestation-check.sh self-tests the archive before deployment (bash + shellcheck on every script including those inside the tarballs, widget JS, schemas, translations, systemd units).

Storage & snapshots choice (Btrfs modes). S3.2 now asks how VM/CT snapshots should be provided: keep local with qcow2 disks (snapshots out of the box), or a native Btrfs storage for VM+CT snapshots. The upgrade offers the same choice and migrates existing disks when needed.

Optional LUKS2 full-disk encryption (EXPERIMENTAL - not yet hardware-validated). Opt-in in all three disk modes, never on by default: cleartext Btrfs /boot + LUKS2/argon2id under the filesystem, passphrase typed once at the Plymouth prompt (keyboard layout honoured), bootable snapshots preserved (kernel retention + @boot archive). RAID1 uses one container per disk with a single passphrase entry. Test it on a VM with a blank disk only - see guide section 3.6.

Active Secure Boot supported (EXPERIMENTAL - OVMF VM only). S3.1 detects Secure Boot and installs proxmox-secure-boot-support so the PVE kernel boots (otherwise "bad shim signature"). Note: on NVIDIA machines, Secure Boot may break suspend/hibernation - the fallback is to disable it in the BIOS.

Housekeeping. chattr +C (nodatacow) on /var/lib/vz/images/ only; all-black Plymouth splash; per-script versioning (SX-19Vn); weekly SMART long self-test on every disk; documentation resynced with the shipped scripts.

Full release notes are in section 8.1 of the guide; the chronological development journal is in DEVLOG-V19.md at the root of the archive. Feedback welcome.
 

Attachments

fruchtzwerg said:
Thanks.
Feedback: I successfully set up systems with RAID1 Btrfs (v17) and single Btrfs (v18) without any problems.
Click to expand...
If you have the necessary equipment, I'd be interested in feedback on the LUKS encryption part; I don't have the necessary equipment at the moment. I'm trying to get hold of an old PC with two hard drives and a small Nvidia 3000 series (3050) graphics card so I can do some physical testing.
 
ubu said:
Can/Would you include ZFS Support instead of BTRFS?
Click to expand...
Planned for later as I lack the necessary equipment for testing. Furthermore, with ZFS I still have far too many bugs, especially with memory management which spikes and crashes the system when attempting to limit it.
 
Interesting, i run zfs on my servers without problems for years
 
I will try your scripts on a test machine and report back
 
ubu said:
Interesting, i run zfs on my servers without problems for years
Click to expand...

For a server, there's no problem with ZFS as long as you have enough RAM (which I use on my Proxmox and TrueNAS servers). This isn't necessarily the case on a laptop, which is my main goal here. The ZFS ARC limitation of 2GB or even 4GB is clearly problematic, especially when there's a graphics environment in addition to running VMs and various other applications on the host. If you limit ZFS ARC to 8GB, there's no problem, but the RAM usage is relatively high for a laptop. Often limited to 32GB or at most 64GB of RAM, if several VMs are running simultaneously, a crash is guaranteed (which happened to me during testing). The other major problem is hibernation for everyday use on a typical laptop, and finally, ZBM with GRUB is a real nightmare. The pre-configured ZBM image works but presents other problems with Debian. That's the current state of affairs regarding ZFS.

PS: I forgot to mention the significantly higher CPU usage with ZFS compared to other filesystems, which is clearly also a problem on a laptop.
 
aureladmin said:
The ZFS ARC limitation of 2GB or even 4GB is clearly problematic
Click to expand...
Do you have pointers to articles supporting this statement? A small size ARC should not rise problems, except performance wise...?

The practical lower limit is only reached when the ZFS internal data structures do not fit into it anymore. The concept: https://openzfs.github.io/openzfs-docs/Performance and Tuning/Workload Tuning.html#adaptive-replacement-cache

----
Edit: an obviously extreme example - on the *BSD side, not OpenZFS: https://wiki.freebsd.org/ZFSTuningGuide#ARC --> "vfs.zfs.arc_max="40M"" :-)
 
Last edited:
  • Like
Reactions: Johannes S
You must log in or register to reply here.
Top Bottom
Back