adding a single new hard disk, what are the options? Change size later on? add remove partitions?

[I_Want2Know]

Hello to you all,

I am playing around with proxmox to see how I can use it. The intention is to run things like NAS software, ip camera server and be as flexible as possible. I did manage to install Proxmox. It seems to work fine for now. Proxmox is installed on a SSD. I just added a Hard drive. What I want is to reserve a few TB from the hard drive for the NAS, however maybe I want to change the partition size later on when the partition is full. Maybe I add a extra camera to the system later on and need more space for that VM. Maybe I'll add a VM and it also needs some storage space on de hard drive, maybe I will remove the VM maybe, maybe etc.

I Think I can best be formating the hard drive as EXT4 as the hardare is not very powerfull I have got 16 gig of ram available at this time, looking at current ram prizes this wil not change that much next years.

So my question is and please correct me if I am wrong. If I format the single disk as EXT4 can I use LVM on top of it to get the flexabillity as I do describe?

I understand a raid system might be much better, however I would rather backup the data at the different location on a usb drive or even better on a sever far away so if something happens to this location like fire or if someone steals the proxmox server I will not lose all of my data. First I would like to get the local server running with the hard drive, later on I would like to see if I can backup the data somewhere far away.

hope you can help, thanx !

 

[bbgeek17]

I Think I can best be formating the hard drive as EXT4 as the hardare is not very powerfull I have got 16 gig of ram available at this time, looking at current ram prizes this wil not change that much next years.

There are three primary ways to provide LOCAL storage to the VM: as virtual images/disks (qcow), via LVM/ZFS flexible volumes, physical passthrough.
By formatting your disk (EXT4, XFS, ZFS, etc) you are selecting virtual image/disk path. You can extend the Image later, create a snapshot, add new Image on the same filesystem later for this or another VM.

LVM is another way to manage your disk. It is a lower level tool than a filesystem. You can place a filesystem on LVM, you cannot place LVM on filesystem. Although, you can place LVM on a QCOW that lives on a filesystem. You can still extend, snapshot, and add new disks.

RAID is an even lower level disk management layer. With a single disk, RAID (redundant array of independent diskS) should play no part in your decision making.

You will likely see no meaningful difference in your particular configuration between LVM and QCOW. Pick one and experiment. If you have a valid backup plan, you can always start from scratch. I'd say go with LVM or ZFS.

Good luck

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[I_Want2Know]

Thanx bbgeek17!

There are three primary ways to provide LOCAL storage to the VM: as virtual images/disks (qcow), via LVM/ZFS flexible volumes, physical passthrough.
By formatting your disk (EXT4, XFS, ZFS, etc) you are selecting virtual image/disk path. You can extend the Image later, create a snapshot, add new Image on the same filesystem later for this or another VM.

LVM is another way to manage your disk. It is a lower level tool than a filesystem. You can place a filesystem on LVM, you cannot place LVM on filesystem. Although, you can place LVM on a QCOW that lives on a filesystem. You can still extend, snapshot, and add new disks.

RAID is an even lower level disk management layer. With a single disk, RAID (redundant array of independent diskS) should play no part in your decision making.

You will likely see no meaningful difference in your particular configuration between LVM and QCOW. Pick one and experiment. If you have a valid backup plan, you can always start from scratch. I'd say go with LVM or ZFS.

Good luck

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

Thanx !

By now I did read trough most of this part of the Proxmox docs:
https://pve.proxmox.com/pve-docs/chapter-pvesm.html

the link above gives a goods overview.

A few years ago I did build a one purpose OMV (OpenMediaVault server, its like a NAS). On this system I did install the OS on the SSD and the bulk storage is on a raid with 2 times a 3.5" hd. There is only OMV on this hardware. The raid setup was fun to do, however my humble opinion is it would have been better if I did not use raid for this system, but did place the 2nd hd in another location e.g. another building. My network is mostly 1gig and a part i s 2.5g. The backups can run in the middle of the night and will take 10 minutes (If I dump a memory card of photos lets say of 64g ) or a few hours or so for the first backup.

So I thought I might take the same aproach as above with Proxmox so lets make a LVM for the SSD and a thin-LVM for the 3.5" hd. This way I can install the OS/VM on the LVM (fast SSD) and I can store the files like the photos and documents on the (slow, but cheaper 3.5" hd). I can always add a Raspbery PI with an USB hd on the network to backup the HD in the Proxmox server.

The best thing would be to store the data in another building though, in case of a fire etc. Can this be done in a safe way when I would add some cloud? I did read something about a zero-knowledge end to end encryption cloud. I cannot do or learn everything in one day but I am thinking it might be wise to encrypt the hard drive directly. The 2nd step would be to add a PI backup server on the LAN and the 3th adding a cloud backup.
Years ago, like more than 10 years ago I did this in a not clever way. I did encrypt the total HD. Cant remember how, I think I did zip it and used a pasword. When I needed the backup, I had to: have twice the amount of space, one for the zip file one for the cache. Run the unzip for a few hours. Just to get the 1k text file I needed.
There are better ways, there must be. So what would be the way to go here? Are there standard ways to encrypt all the files in little pieces so I can download one txt file and decrypt it?

thanx for the help!

 

[Pifouney]

Hello,



It seems you are confusing several concepts that are not the same.

You need to sort out exactly what you want on your Proxmox virtualization server and how you wish to separate your backups.

To avoid the issue you encountered with your backups, you should use multiple backup tools, including one dedicated to files. This type of backup will allow you to easily recover a single file from the backup.

 

[news]

The Proxmox virtualization server is for virtualization of all your real hardware.
so i recommend use ZFS and ZFS Mirror for rpool for your Proxmox VE.
This need at last 2 SSD Disks.
Your data sit then on a second ZFS Pool, i.g. dpool.
This can be a ZFS Mirror with 2 or more HDDs, and then you must - in my option - use a second VDEV1 for ZFS Special device with 2 or more SSDs. Then there will be all ZFS meta Data will be stored.
For ZFS you need cmr hdds and dram cache and plp ssds. No NVME required.

But ZFS is ore can more then you think, zfs snapshot in 1 second. so you can use the zfs snapshot for Backup without 'stop' or 'halt' the lxc or VM.

In these days you can easy use 2.5 GBit/s or 10 GBit/s Network for your Backupstream.
Use your 1 GBit/s only for your Proxmox VE or Proxmox BS (Backup Server) Management access.

Check the ability of ZFS Datasets and what that mean!
This is - in my option - the way to go.

If you have access to a second Building, the you can replicate the data of the 1st Proxmox BS to the 2nd Proxmox BS.
But this will for safety reason only a pull.

 

[news]

Concept for home user.

You can think of a Proxmox VE virtualization server with only the ZFS rpool for the Proxmox VE, many ram and fast or high LAN speed.
And a bare metal NAS Server (ZFS) with many TB of dataspace, for your Proxmox VE Server and your lxc, vm application can direkt access this data.
Read/ Write access can controlled and data space can be share between Applications.