why not an ZFS over NFS
- Thread starter demoze
- Start date
A few things making this impossible:
1) NFS does not understand and support ZFS file attributes and ACL
2) NFS does not support atomic writes which brakes basic and fundamental security and integrity in ZFS
3) NFS does not support partitions
Add to the above that it will be plain stupid since it contradict and brake everything which ZFS stands for.
Thank for your response @dietmar and @mir, ZFS over iscsi work for me, however I like the simplicity of NFS. Well, may be, my english skill is somewhat confuse. Tell me if I'am still wrong.
What I means is when we add an NFS storage and this storage is provide by a ZFS box, would it be technically possible to let proxmox remotely create a zfs dataset inside the pool on this box to store the vm image instead of just create the vm image on the pool export by NFS.
Eg:
So assuming /mnt/tank is a zfs pool on the zfs box and vm100-disk1.raw the vm 100 image file.
/mnt/tank1/images/100/vm100-disk1.raw
Then
/mnt/tank1/images/100/ will be a zfs dataset for vm 100 image vm100-disk1.raw
/mnt/tank1/images/100/vm100-disk1.raw
This way we could have snapshot. on NFS. Or am I totally wrong again
What I means is when we add an NFS storage and this storage is provide by a ZFS box, would it be technically possible to let proxmox remotely create a zfs dataset inside the pool on this box to store the vm image instead of just create the vm image on the pool export by NFS.
Eg:
So assuming /mnt/tank is a zfs pool on the zfs box and vm100-disk1.raw the vm 100 image file.
/mnt/tank1/images/100/vm100-disk1.raw
Then
/mnt/tank1/images/100/ will be a zfs dataset for vm 100 image vm100-disk1.raw
/mnt/tank1/images/100/vm100-disk1.raw
This way we could have snapshot. on NFS. Or am I totally wrong again
Last edited:
This is the way I export NFS shares from my ZFS box to proxmox:
Code:
zfs get all /vMotion/nfs
NAME PROPERTY VALUE SOURCE
vMotion/nfs type filesystem -
vMotion/nfs creation Sun Jun 9 17:52 2013 -
vMotion/nfs used 52.5G -
vMotion/nfs available 1.16T -
vMotion/nfs referenced 52.5G -
vMotion/nfs compressratio 2.67x -
vMotion/nfs mounted yes -
vMotion/nfs quota none default
vMotion/nfs reservation none default
vMotion/nfs recordsize 128K default
vMotion/nfs mountpoint /vMotion/nfs default
vMotion/nfs sharenfs rw=@172.16.2.0/24,rw=@10.0.1.0/24,rw=@10.0.2.0/24,root=@172.16.2.0/24,root=@10.0.1.0/24,root=@10.0.2.0/24 local
vMotion/nfs checksum on default
vMotion/nfs compression lz4 inherited from vMotion
vMotion/nfs atime off inherited from vMotion
vMotion/nfs devices on default
vMotion/nfs exec on default
vMotion/nfs setuid on default
vMotion/nfs readonly off default
vMotion/nfs zoned off default
vMotion/nfs snapdir hidden default
vMotion/nfs aclmode passthrough local
vMotion/nfs aclinherit passthrough-x local
vMotion/nfs canmount on default
vMotion/nfs xattr on default
vMotion/nfs copies 1 default
vMotion/nfs version 5 -
vMotion/nfs utf8only off -
vMotion/nfs normalization none -
vMotion/nfs casesensitivity sensitive -
vMotion/nfs vscan off default
vMotion/nfs nbmand off default
vMotion/nfs sharesmb off default
vMotion/nfs refquota none default
vMotion/nfs refreservation none default
vMotion/nfs primarycache all default
vMotion/nfs secondarycache all default
vMotion/nfs usedbysnapshots 0 -
vMotion/nfs usedbydataset 52.5G -
vMotion/nfs usedbychildren 0 -
vMotion/nfs usedbyrefreservation 0 -
vMotion/nfs logbias latency default
vMotion/nfs dedup off default
vMotion/nfs mlslabel none default
vMotion/nfs sync standard local
vMotion/nfs refcompressratio 2.67x -
vMotion/nfs written 52.5G -
vMotion/nfs logicalused 140G -
vMotion/nfs logicalreferenced 140G -
vMotion/nfs filesystem_limit none default
vMotion/nfs snapshot_limit none default
vMotion/nfs filesystem_count none default
vMotion/nfs snapshot_count none default
vMotion/nfs redundant_metadata all default
Hi demoze,
I can see the point, but it is very hard to do that (in such a way that it works great).
iSCSI is block storage which is normally not shared (or at least should'nt be shared if non-clustered filesystem is used) and on the other side we have NFS which is a shared file based storage. If you want to administrate the ZFS directly from remote, you need to have an API for that, which currently does not exist. You can implement a direct SSH access to the ZFS server to execute all the commands, but normally, you do not allow someone to SSH directly into you ZFS storage box.
Best,
LnxBil
I can see the point, but it is very hard to do that (in such a way that it works great).
iSCSI is block storage which is normally not shared (or at least should'nt be shared if non-clustered filesystem is used) and on the other side we have NFS which is a shared file based storage. If you want to administrate the ZFS directly from remote, you need to have an API for that, which currently does not exist. You can implement a direct SSH access to the ZFS server to execute all the commands, but normally, you do not allow someone to SSH directly into you ZFS storage box.
Best,
LnxBil
Hi lnxBill,
This is exactly how proxmox manage zfs over iscsi storage, by running zfs command over ssh.
If you mount an zfs zpool from your zfs box, and you create zfs dataset inside the same pool (one for each vm) it become possible to manage each dataset with quota and snapshot. And this could be done the same way as "zfs over iscsi" with some additional coding on the nfs storage plugin or building a new one.
I've done some investigation in the perl code base. I still need to understand various part and how some component are relate to each other, but thing seems to be doable.
Thank
This is exactly how proxmox manage zfs over iscsi storage, by running zfs command over ssh.
If you mount an zfs zpool from your zfs box, and you create zfs dataset inside the same pool (one for each vm) it become possible to manage each dataset with quota and snapshot. And this could be done the same way as "zfs over iscsi" with some additional coding on the nfs storage plugin or building a new one.
I've done some investigation in the perl code base. I still need to understand various part and how some component are relate to each other, but thing seems to be doable.
Thank
Last edited: