[SOLVED] Browse ZFS filesystem

[TiagoLardaCrianca]

Hi, im completely newbie at Proxmox and Linux, generally speaking. I have a machine (actually two) wich I have installed proxmox and have shared some files over samba (smb) with some windows computers. Works finewhen I browse and share the "local" storage (i can share /var/lib/vz and for example, put some ISO files inside). But i have another ZFS created storage (named "Storage"), with the VMs I created, and they show up in the GUI just fine:



But when i try to access the ZFS partition trough the batch, i cannot browse files at all... i run the command "zfs list" and this shows up:



If i browse the "/Storage" directory, really nothing shows up:


How can I access the VM-disk-XXXX.raw trough the batch, for instance, to backup those files away ?

Thank you in advance,

Tiago.

 

[shrdlicka]

Hi,

the usual way would be to set a mountpoint ... but in this case this wouldn't work. Under /dev/zvol/Storage/vm-100-disk-0* you will find the device. This is not a filesystem that handle files it is a block device you can check what partitions you have got on this block device via:

fdisk -l /dev/zvol/Storage/vm-100-disk-0

For me this looks like that:

root@ella:/home/s # fdisk -l /dev/zvol/zfs/vm-223-disk-0
Disk /dev/zvol/zfs/vm-223-disk-0: 32 GiB, 34359738368 bytes, 67108864 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 8192 bytes
I/O size (minimum/optimal): 8192 bytes / 8192 bytes
Disklabel type: gpt
Disk identifier: EBDEE73B-40A7-4ED5-A2E8-583077B6DF86

Device                          Start      End  Sectors  Size Type
/dev/zvol/zfs/vm-223-disk-0p1    2048    67583    65536   32M EFI System
/dev/zvol/zfs/vm-223-disk-0p2   67584   116735    49152   24M Linux filesystem
/dev/zvol/zfs/vm-223-disk-0p3  116736   641023   524288  256M Linux filesystem
/dev/zvol/zfs/vm-223-disk-0p4  641024   690175    49152   24M Linux filesystem
/dev/zvol/zfs/vm-223-disk-0p5  690176  1214463   524288  256M Linux filesystem
/dev/zvol/zfs/vm-223-disk-0p6 1214464  1230847    16384    8M Linux filesystem
/dev/zvol/zfs/vm-223-disk-0p7 1230848  1427455   196608   96M Linux filesystem

p[1-6] corresponds to the parts:
see this

root@ella:/home/s # ls /dev/zvol/zfs/vm-223-disk-0-p*
/dev/zvol/zfs/vm-223-disk-0-part1  /dev/zvol/zfs/vm-223-disk-0-part3  /dev/zvol/zfs/vm-223-disk-0-part5  /dev/zvol/zfs/vm-223-disk-0-part7
/dev/zvol/zfs/vm-223-disk-0-part2  /dev/zvol/zfs/vm-223-disk-0-part4  /dev/zvol/zfs/vm-223-disk-0-part6  /dev/zvol/zfs/vm-223-disk-0-part8

You can mount one of these devices since they should contain a filesystem.[/icode]

 

[TiagoLardaCrianca]

Hi,

the usual way would be to set a mountpoint ... but in this case this wouldn't work. Under /dev/zvol/Storage/vm-100-disk-0* you will find the device. This is not a filesystem that handle files it is a block device you can check what partitions you have got on this block device via:

fdisk -l /dev/zvol/Storage/vm-100-disk-0

For me this looks like that:

root@ella:/home/s # fdisk -l /dev/zvol/zfs/vm-223-disk-0
Disk /dev/zvol/zfs/vm-223-disk-0: 32 GiB, 34359738368 bytes, 67108864 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 8192 bytes
I/O size (minimum/optimal): 8192 bytes / 8192 bytes
Disklabel type: gpt
Disk identifier: EBDEE73B-40A7-4ED5-A2E8-583077B6DF86

Device                          Start      End  Sectors  Size Type
/dev/zvol/zfs/vm-223-disk-0p1    2048    67583    65536   32M EFI System
/dev/zvol/zfs/vm-223-disk-0p2   67584   116735    49152   24M Linux filesystem
/dev/zvol/zfs/vm-223-disk-0p3  116736   641023   524288  256M Linux filesystem
/dev/zvol/zfs/vm-223-disk-0p4  641024   690175    49152   24M Linux filesystem
/dev/zvol/zfs/vm-223-disk-0p5  690176  1214463   524288  256M Linux filesystem
/dev/zvol/zfs/vm-223-disk-0p6 1214464  1230847    16384    8M Linux filesystem
/dev/zvol/zfs/vm-223-disk-0p7 1230848  1427455   196608   96M Linux filesystem

p[1-6] corresponds to the parts:
see this

root@ella:/home/s # ls /dev/zvol/zfs/vm-223-disk-0-p*
/dev/zvol/zfs/vm-223-disk-0-part1  /dev/zvol/zfs/vm-223-disk-0-part3  /dev/zvol/zfs/vm-223-disk-0-part5  /dev/zvol/zfs/vm-223-disk-0-part7
/dev/zvol/zfs/vm-223-disk-0-part2  /dev/zvol/zfs/vm-223-disk-0-part4  /dev/zvol/zfs/vm-223-disk-0-part6  /dev/zvol/zfs/vm-223-disk-0-part8

You can mount one of these devices since they should contain a filesystem.[/icode]

Perfect, thank you!



Now, the last question would be if i could backup a whole zfs block into a file, like a vhdx file... (with partitions and so on)? Im sorry about the cascated questions, i have to learn more about ZFS and linux in general... but you already helped a lot.

Thanks. Cheers!

 

[Dunuin]

Why don't you just backup the VM? Then you got the virtual disks + VM config files packed in an archive.
Another option would be to use the "zfs send ..." command and pipe the stream into a file. It could then be restored by piping the file into the "zfs recv" command. And the dd command of cause also is an option.

And you shouldn't mount a zvol when the VM is running or you might corrupt your data.

 

[TiagoLardaCrianca]

Why don't you just backup the VM? Then you got the virtual disks + VM config files packed in an archive.
Another option would be to use the "zfs send ..." command and pipe the stream into a file. It could then be restored by piping the file into the "zfs recv" command. And the dd command of cause also is an option.

And you shouldn't mount a zvol when the VM is running or you might corrupt your data.

Thank you for the reply, and I'll also take care when mounting a zfs using that method, for not mounting when the vm is running.

I wanted to use the backup feature, since that is what ive wanted from the start, i just couldnt find it to save it on another device on the network or in a usb drive attached to the proxmox server, still have to read how to save those backups in a different device, since the only choice ive got from the backup is the "local" storage. (cant fit 1.4TB vm there).




For instance, ive got a usb drive plugged in, but it doesnt show up on the disk list (would be /dev/sdd ). I dont yet understand proxmox very well, i have to read more...

 

[Dunuin]

If you want Proxmox VE to make use of any USB disk or NAs Share you will have to create a storage first pointing to that share or disk. See "Datacenter -> Storage -> Add -> NFS/CIFS/Directory". And to be able to store backups on a storage you have to set the storages content type to include "Vzdump Backups".

 

[TiagoLardaCrianca]

If you want Proxmox VE to make use of any USB disk or NAs Share you will have to create a storage first pointing to that share or disk. See "Datacenter -> Storage -> Add -> NFS/CIFS/Directory". And to be able to store backups on a storage you have to set the storages content type to include "Vzdump Backups".

Ok figured out why the drive wasent showing.. it was formatted as exFAT.. already installed the packages on proxmox and followed your instructions...



Great! I have got backups! Your help was undoubdfully excellent! Can't thank you enought!

Cheers mate!

 

[Dunuin]

In case of directory storages pointing to a mountpoint you should also set the "is_mountpoint" option for that storage which can only be done using the CLI.
So in case you manually mount that USB HDD or mount it using fstab you also should run the command pvesm set backup_usb --is_mountpoint yes once. Without that PVE won't check if the USB-Disk is mounted or not and will always write your backup to that path. And in case it is not mounted doing backups would write all the backups to your "local" storage filling it up until PVE stops working.

 

[TiagoLardaCrianca]

In case of directory storages pointing to a mountpoint you should also set the "is_mountpoint" option for that storage which can only be done using the CLI.
So in case you manually mount that USB HDD or mount it using fstab you also should run the command pvesm set backup_usb --is_mountpoint yes once. Without that PVE won't check if the USB-Disk is mounted or not and will always write your backup to that path. And in case it is not mounted doing backups would write all the backups to your "local" storage filling it up until PVE stops working.

Valuable information, thank you.. and yes, i mounted manually using the "mount /dev/sdd /mnt/XXX"... i really have to study linux and PVE better to know these details... i am used to windows, where things/drivers/awareness are completely different there... but time will prove me linux will be a better (flexible) choice thanks once again and i appreciate all the time you dispended responding my questions

 

[Dunuin]

and yes, i mounted manually using the "mount /dev/sdd /mnt/XXX"

Thats fine but remember to do it every time after rebooting your server. Linux won't mount it automatially if you don't create a fstab entry for it or using a mounting script like autofs.

 

[Dunuin]

And in case you want to automount it, it would be better not to use "/dev/sdd1" because sometimes your USB-disk might be "/dev/sdd1", sometimes "/dev/sde1" or "/dev/sdf1" and so on. If you want to make sure you are mounting the correct disk you should mount it by a unique ID like a UUID, WNN or by something like "/dev/disk/by-id/usb-YourdiskManufacturer-YourDiskmodel-YourDiskSerial-part1" (you can get the names with the command ls -la /dev/disk/by-id).

 

[TiagoLardaCrianca]

Thats fine but remember to do it every time after rebooting your server. Linux won't mount it automatially if you don't create a fstab entry for it or using a mounting script like autofs.

Ill check into fstab syntax and functioning... i still got many questions about how the mounts happen, if several/different devices, after being on fstab list, will be able to use the same mountpoint, or if i can create a fixed mountpoint for each one of them.. that would be of great use for using different usb backup drives with different - while that, reading the fstab help page... "You can define the filesystems in the /etc/fstab configuration in three different ways: by kernel naming descriptors, by UUID, or by labels." ... well, i'm loving linux!

 

[TiagoLardaCrianca]

And in case you want to automount it, it would be better not to use "/dev/sdd1" because sometimes your USB-disk might be "/dev/sdd1", sometimes "/dev/sde1" or "/dev/sdf1" and so on. If you want to make sure you are mounting the correct disk you should mount it by a unique ID like a UUID, WNN or by something like "/dev/disk/by-id/usb-YourdiskManufacturer-YourDiskmodel-YourDiskSerial-part1" (you can get the names with the command ls -la /dev/disk/by-id).

nailed it!

 

[Dunuin]

What you should keep in mind when using USB-disks with fstab is, that PVE won't boot if a USB-disk is added in the fstab but then not plugged in while booting the server. If you don't plan to keep the USB-Disks permanently attached to the PVE server it would be better to use autofs or something similar. With that, a service will monitor if a new USB-disks are plugged in and then automount them. So it no problem to attach the disks after boot or to boot without them attached. See the autofs article I linked above.

 

[TiagoLardaCrianca]

I didn't know about the boot.. thought that after the bootdisk defined on the bios, it would boot every time. Yes, i was planning to plug them in just for the backups and then unmount take them off again, so, according to you, autofs would be a better way to do this.. ok, ill dive into it. Monitoring the plugged-in state would be a very useful in this situation, of course. Thank you!