Proxmox boot from usb pen drive

[bsilva]

Note: this setup will loose your log files and VM stats on reboot.
As far as I know, VMs are safe, as is the Host configuration (I have been using this setup for almost one year, done several reboots after dist-upgrades, never had an issue).
Not sure about clusters: I have not checked where a cluster setup will put its I/O, and how to keep the cluster config safe.
Note2: I have updated the init script, the latest proxmox versions will not start containers if the /var/lib/vz/root/${vid} do not exist yet.
The new script will create folders 100 to 131, edit if you have more containers.

---

Hi all,

I have spent some time with a peculiar setup and just thought I could share the results with the community.

I have recently bought a nice hp microserver without disks, and I have setup proxmox 2.1 on a usb pen drive (2GB) .
I used the instructions on http://pve.proxmox.com/wiki/Proxmox_ISCSI_installation.
It was done on a ubuntu desktop, replacing the target iscsi volume by the flash drive.

Storage for all (templates, isos, containers, images...) is on a readynas NVX via NFS.

It's all working, I have already a couple of containers in it, great work proxmox guys, it's a really great virtualization solution.

Now, I am trying to keep writes on that flash pen to a minimum.

my /etc/fstab is like: (flash pen partition is labeled "armorica")

[FONT=courier new]
[/FONT]LABEL=armorica /                 ext2         errors=remount-ro,noatime,nodiratime     0 1

none      /tmp                tmpfs        defaults           0    0
none      /var/log            tmpfs        defaults           0    0
none      /var/run            tmpfs        defaults           0    0
none      /var/tmp            tmpfs        defaults           0    0
none      /var/cache/apt/archives    tmpfs        defaults           0    0
none      /var/lib/rrdcached        tmpfs        defaults           0    0
none      /var/lib/vz            tmpfs        defaults           0    0
none      /var/lib/vzctl        tmpfs        defaults           0    0
none      /var/lib/vzquota        tmpfs        defaults           0    0

and a df -h results:

root@armorica:/etc/init.d# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             1.9G  842M  935M  48% /
tmpfs                 2.0G     0  2.0G   0% /lib/init/rw
udev                  2.0G  192K  2.0G   1% /dev
tmpfs                 2.0G   29M  1.9G   2% /dev/shm
none                  2.0G     0  2.0G   0% /tmp
none                  2.0G  308K  2.0G   1% /var/log
none                  2.0G  196K  2.0G   1% /var/run
none                  2.0G     0  2.0G   0% /var/tmp
none                  2.0G     0  2.0G   0% /var/cache/apt/archives
none                  2.0G  360K  2.0G   1% /var/lib/rrdcached
none                  2.0G     0  2.0G   0% /var/lib/vz
none                  2.0G     0  2.0G   0% /var/lib/vzctl
none                  2.0G  4.0K  2.0G   1% /var/lib/vzquota
/dev/fuse              30M   16K   30M   1% /etc/pve
discus:/c/VMBackups   3.7T  777G  2.9T  22% /mnt/pve/VMBackups
discus:/c/isos        3.7T  777G  2.9T  22% /mnt/pve/ISOs
discus:/c/OpenVZTemplates
                      3.7T  777G  2.9T  22% /mnt/pve/Templates
discus:/c/bluesky     3.7T  777G  2.9T  22% /mnt/pve/bluesky

putting the /var/log on tmpfs will break stuff, as some apps (like apache) expect the folder to already exist.
So I have created a small startup script on /etc/init.d (see atthached file), and installed it with:
update-rc.d prepTmpfs.sh start 09 S

So, my question to the proxmox "veterans", is there any other folder you would put on tmpfs?
(or any I did that you wouldn't?)

Best Regards,
Bruno Silva

 

[beliya]

Hey bsilva,You've done some nice work. Did you made some progress on it? Do you have any problems running the Proxmox OS from the flash drive?And did you put some other directories out of it? I can't find any other description on the internet about this, and I am really interested in replacing the hard drives in my servers with flash drivers and running the VMs from NAS NFS shares.I will really appreciate it if you reply back with more info on the subject.Thanks.Regards,Beliya

 

[bsilva]

Hi beliya,

That server is running fine from the same usb pen drive.

The mounts are still the same.

I have created just a couple of additional scripts to ease management:

setup_name.sh: to enable vzctl <cmd> <servername> instead of <ctid>
backup_rootdisk.sh: to backup the usb drive to the NFS backups mountpoint

 

[hotwired007]

where do all the logs write to - have you moved them off the USB too?

 

[bsilva]

Yes, /var/log goes to tmpfs.

This is a small test server, so I do not mind loosing the logs and vm stats on a reboot.

but if I really needed them, I guess I would try to save them somewhere else, or rsync that folder frequently to the NAS or at least sysloging to some other box.

 

[hotwired007]

This is a small test server, so I do not mind loosing the logs and vm stats on a reboot.

if it was a production server i would definately want those elsewhere to diagnose any system failures.

 

[bsilva]

if it was a production server i would definately want those elsewhere to diagnose any system failures.

Agreed 100%. But as all my production boxes come with 2 disks in raid 1 for OS... I haven't needed to solve that problem so far.
I would also recommend for production some form of central monitoring/management.
Some problems may also arise if tmpfs gets full...

The point of this exercise was to avoid more drives wasting watts at my home office... and also just for the fun of doing it

But you are right, I should have added some sort of disclaimer/warning about using this on production. I will add that to the first post.