ZFS RAID1 to storage important "standard" files?

[Jokte]

Hello everyone, I'm fairly new to this server, virtualization and backup stuff.

I have a 2 TB hard drive with very important information for me. I am planning to buy two 8TB disks and put them in RAID1. My question is if I can use the Proxmox interface to put them in RAID1 with ZFS and then mount them in a QEMU virtual machine.

My idea would be that Proxmox would do the replication or backup and the virtual machine would have read and write access to one or both disks.

I don't know if my idea is correct or if it can be done. In case it can be done, can you tell me how?
If not, could you tell me how to do something similar to what I want?

Thanks so much!

 

[Dunuin]

My question is if I can use the Proxmox interface to put them in RAID1 with ZFS and then mount them in a QEMU virtual machine.

Yes, you can create it using the webUI. But you can't mount a ZFS Pool in a VM. What you can do is create a virtual disk on that pool (which creates a zvol that will be used as a block device to store that virtual disk) and use that virtual disk wit hthe filesystem of your choice to store your data inside the VM.

My idea would be that Proxmox would do the replication or backup and the virtual machine would have read and write access to one or both disks.

Raid1 will always read and write to/from both disks. They are mirrored. And Raid never replaces a backup. So your backups you should get another server or at least additional disks that are not part of that raid1 pool.

 

[Jokte]

Yes, you can create it using the webUI. But you can't mount a ZFS Pool in a VM. What you can do is create a virtual disk on that pool (which creates a zvol that will be used as a block device to store that virtual disk) and use that virtual disk wit hthe filesystem of your choice to store your data inside the VM.


Raid1 will always read and write to/from both disks. They are mirrored. And Raid never replaces a backup. So your backups you should get another server or at least additional disks that are not part of that raid1 pool.

Thank you very much for explaining everything so well. As soon as I have the disks I will try to configure it and if I have any problem I will tell you.

Thanks for clarifying the backups, I do not work in my day to day with backups and I forget that it is best to be on a different server and place where the data is usually accessed.

 

[Jokte]

Yes, you can create it using the webUI. But you can't mount a ZFS Pool in a VM. What you can do is create a virtual disk on that pool (which creates a zvol that will be used as a block device to store that virtual disk) and use that virtual disk wit hthe filesystem of your choice to store your data inside the VM.


Raid1 will always read and write to/from both disks. They are mirrored. And Raid never replaces a backup. So your backups you should get another server or at least additional disks that are not part of that raid1 pool.

Hi again. I have tried the configuration you have told me with a test disk and the following doubt has arisen, could I access the data easily if I need to recover them by removing the disk from the server and connecting it to a GNU/Linux?

The server is for home use and I am a student, so I can not buy many hard drives or another system to make a backup at the moment.

The idea I have in mind would be something like having a backup on another disk on the same server (since I can not buy another server for backups) so that if the data on the first disk is damaged, I can recover it with the second disk and if I need to remove the disk to read the data from another computer with GNU / Linux, just connect it and mount it.

I think I'm losing or confusing concepts, as well as overstating my needs. I think a valid option for what I want could be to pass one of the two 8TB disks (which I don't have yet) to the VM via its id (/dev/disk/by-id) and in Proxmox do a cron with the rsync command and synchronize the two hard disks. This way I could have a backup on another hard disk, if I delete a file by mistake on the first hard disk I can recover it with the second one and I can take the disk out and easily connect and mount it on another computer.

What is your opinion? What do you recommend?
Sorry for bothering and thank you very much for the help!

 

[Dunuin]

Hi again. I have tried the configuration you have told me with a test disk and the following doubt has arisen, could I access the data easily if I need to recover them by removing the disk from the server and connecting it to a GNU/Linux?

It depends on how you use the disk and how you define "easy". But usually shouldn't be a big problem, as for example, a Ubuntu Live CD should be able to mount filesystems on partitions/LV/zvols.

The server is for home use and I am a student, so I can not buy many hard drives or another system to make a backup at the moment.

Then I would skip the raid1 and use a single disk for data and another disks for backups. If you care about your data you could use ZFS with a single disk, so you still get bit rot detection (but it can't heal corrupted data). Later, when you got some money, you could buy a third disk and easily convert that single disk pool into a mirror for redundancy (and then ZFS could also repair corrupted data).

The idea I have in mind would be something like having a backup on another disk on the same server (since I can not buy another server for backups) so that if the data on the first disk is damaged, I can recover it with the second disk and if I need to remove the disk to read the data from another computer with GNU / Linux, just connect it and mount it.

Yes, that's an option if you can accept losing all your created/edited data since your last backup.

I think I'm losing or confusing concepts, as well as overstating my needs. I think a valid option for what I want could be to pass one of the two 8TB disks (which I don't have yet) to the VM via its id (/dev/disk/by-id) and in Proxmox do a cron with the rsync command and synchronize the two hard disks. This way I could have a backup on another hard disk, if I delete a file by mistake on the first hard disk I can recover it with the second one and I can take the disk out and easily connect and mount it on another computer.

Keep in mind that you can't mount a passthroughed disk in parallel inside a VM and on the host, as this would currupt your data. So both disks would need to be accessible by rsync from inside your VM.

 

[Jokte]

Hi, today I got a hard disk and I configured it with ZFS as a single hard disk. I have created a vmdisk and added it to the virtual machine, I have formatted it as EXT4 but still, I am not able to access it from the virtual machine, it does not show me file system type.
How should I configure it to be able to access and write data from the virtual machine?

ZFS:


VM Disk:

jokte@server:~$ sudo fdisk -l
...
Disk /dev/sdd: 6.92 TiB, 7612829532160 bytes, 14868807680 sectors
Disk model: QEMU HARDDISK
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: B068879C-63F6-7247-8538-C0C87FEDA951

Device Start End Sectors Size Type
/dev/sdd1 2048 14868807646 14868805599 6.9T Linux filesystem

jokte@server:~$ sudo lsblk -f
NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS
loop0 squashfs 4.0 0 100% /snap/core20/1828
loop1 squashfs 4.0 0 100% /snap/core20/1852
loop2 squashfs 4.0 0 100% /snap/lxd/23541
loop3 squashfs 4.0 0 100% /snap/snapd/18596
loop4 squashfs 4.0 0 100% /snap/snapd/18933
loop5 squashfs 4.0 0 100% /snap/lxd/24322
sda
├─sda1 vfat FAT32 FE5E-404F 1G 1% /boot/efi
├─sda2 ext4 1.0 3e6d680a-1e41-4f8b-b4f0-81289432dc12 1.5G 13% /boot
└─sda3 LVM2_member LVM2 001 VotL3E-F2LD-8VO8-9te4-78sM-dRq1-t6pcQS
└─ubuntu--vg-ubuntu--lv ext4 1.0 a0d2d063-6082-4454-ac0f-85c18ae88304 16.9G 35% /
...
sdd
└─sdd1

jokte@server:~$ sudo blkid -p /dev/sdd1
/dev/sdd1: PART_ENTRY_SCHEME="gpt" PART_ENTRY_UUID="beaf3d93-68ec-a246-8e29-ca6c739418aa" PART_ENTRY_TYPE="0fc63daf-8483-4772-8e79-3d69d8477de4" PART_ENTRY_NUMBER="1" PART_ENTRY_OFFSET="2048" PART_ENTRY_SIZE="14868805599" PART_ENTRY_DISK="8:48"

jokte@server:~$ sudo mount /dev/sdd1 /media/DATOS/
mount: /media/DATOS: wrong fs type, bad option, bad superblock on /dev/sdd1, missing codepage or helper program, or other error.