I have two physical drives on the proxmox host. I want to share them with FreeNas and configure them there, then let the other containers access them from there. How would I set that up?
How do I share physical drives to Freenas VM?
- Thread starter Ghostbroker
- Start date
Here's the procedure I used to pass physical disks to my OpenMediaVault VMs:
https://pve.proxmox.com/wiki/Physical_disk_to_kvm
https://pve.proxmox.com/wiki/Physical_disk_to_kvm
I'd discourage you from running FreeNAS as a VM. While possible, it's not without risks (data loss and/or corruption) and you can accomplish most media sharing tasks by using SAMBA on the promox host with ZFS being run natively on the host OS.
Just my 2cents
Just my 2cents
bobmc, would you please elaborate on why there are risks running FreeNAS as a VM? Data loss or corruption?
Would this really be an issue if you pass the disks directly to the VM? Providing FreeNAS raid manager direct access to disks....
What if you pass an entire HBA (if possible) to the FreeNAS VM?
FreeNas allows for an easy way to manage an NAS vs having to use CLI to achieve the same things.....
Would this really be an issue if you pass the disks directly to the VM? Providing FreeNAS raid manager direct access to disks....
What if you pass an entire HBA (if possible) to the FreeNAS VM?
FreeNas allows for an easy way to manage an NAS vs having to use CLI to achieve the same things.....
Nothing about that snippet from freenas.org discourages running FreeNAS in a hypervisor. In fact, it exists precisely to describe how to do it safely. Pass your disks directly through to the VM and bypass the Hypervisor.
Note that the procedure referenced by @Belokan does not do a true pass through. This is a Virtio simulation of pass through, when the disk driver exists in the host OS and KVM is presenting a virtual version of it using Virtio. Not only does not not avoid the risks freenas.org warns about, it will likely perform poorly.
What you want is a true PCI pass through of the entire Disk controller to the guest VM. This is generally possible in KVM - unfortunately I can't give you a pointer to the method.
BTW, to get decent performance from your NAS, you probably also want to use a separate NIC and doe a PCI pass through of that too, though a SRiov pass through of a vport on your NIC is probably good enough.
Note that the procedure referenced by @Belokan does not do a true pass through. This is a Virtio simulation of pass through, when the disk driver exists in the host OS and KVM is presenting a virtual version of it using Virtio. Not only does not not avoid the risks freenas.org warns about, it will likely perform poorly.
What you want is a true PCI pass through of the entire Disk controller to the guest VM. This is generally possible in KVM - unfortunately I can't give you a pointer to the method.
BTW, to get decent performance from your NAS, you probably also want to use a separate NIC and doe a PCI pass through of that too, though a SRiov pass through of a vport on your NIC is probably good enough.
May I ask why you need to run FreeNAS for?
The reason I'm asking is because I was a fan of OpenMediaVault and have managed to run it as VM on proxmox but lately I prefer deploying LXC containers for every service I want. Here is an example:
https://imgur.com/a/s2RDZYd
The reason I'm asking is because I was a fan of OpenMediaVault and have managed to run it as VM on proxmox but lately I prefer deploying LXC containers for every service I want. Here is an example:
https://imgur.com/a/s2RDZYd
Last edited:
I see this Freenas quote all the time that to run ZFS you need to give it direct access to disks, but is this really a requirement of ZFS or a statement from freenas/IXsystems ?
I have ran Freenas virtual on top an enterprise raid controller with cache and BBU. This presents one single disk to Freenas. My raid controller does all of the monitoring for the underlying disks (smart checks, health, etc....). The controller provides notifications when things go wrong with the disk. A nice benefit is the easy swapping and rebuilding of the array......so simple.... just pull out the cage with the bad disk and insert new... raid rebuild happens automatically !!!!
Now in this scenario, I loose the raid function and disk information function built into ZFS. However, I still keep the snapshot function, replication function and other great qualities of ZFS. I let the raid controller do what it is designed for ..... manage the disks underneath and provide me all the disk information + alerts.
I know that plenty will disagree and tell me this is foolish and ZFS is not designed to function this way, but in a real enterprise environment..... I bet you find this kind of setup more often than not.
Again just speaking from my experience and maybe sparking a little debate
Possibly completely wrong and any large organization using ZFS only uses it directly.
PigLover - your suggestion of passing through HBA of the data disks and using SR-IOV for the NIC is a great option as well to use FreeNAs and retain all of the ZFS functions. (Granted I have had trouble getting SR-IOV to work with proxmox during my limited testing ...... It fails after host reboot..... I blame my lack of knowledge)
I have ran Freenas virtual on top an enterprise raid controller with cache and BBU. This presents one single disk to Freenas. My raid controller does all of the monitoring for the underlying disks (smart checks, health, etc....). The controller provides notifications when things go wrong with the disk. A nice benefit is the easy swapping and rebuilding of the array......so simple.... just pull out the cage with the bad disk and insert new... raid rebuild happens automatically !!!!
Now in this scenario, I loose the raid function and disk information function built into ZFS. However, I still keep the snapshot function, replication function and other great qualities of ZFS. I let the raid controller do what it is designed for ..... manage the disks underneath and provide me all the disk information + alerts.
I know that plenty will disagree and tell me this is foolish and ZFS is not designed to function this way, but in a real enterprise environment..... I bet you find this kind of setup more often than not.
Again just speaking from my experience and maybe sparking a little debate
Possibly completely wrong and any large organization using ZFS only uses it directly.PigLover - your suggestion of passing through HBA of the data disks and using SR-IOV for the NIC is a great option as well to use FreeNAs and retain all of the ZFS functions. (Granted I have had trouble getting SR-IOV to work with proxmox during my limited testing ...... It fails after host reboot..... I blame my lack of knowledge)
I still don't get it. Why FreeNAS? What for? When you can run zfs on the proxmox host itself?
You are adding a layer of complexity and degrading performance for no obvious reason.
Tell us what is FreeNAS used for.
On my setup (on the pic above) I have two raid controllers that expose single drives to the os (no hw raid),
zfs does it's magic and all containers get a bind mount to the array for data access.
You are adding a layer of complexity and degrading performance for no obvious reason.
Tell us what is FreeNAS used for.
On my setup (on the pic above) I have two raid controllers that expose single drives to the os (no hw raid),
zfs does it's magic and all containers get a bind mount to the array for data access.
Vassilis - What do you use for just standard network file sharing? User setup and management? File permission control?
- I am not sure what the use case is for the individual starting this thread
I can say in my environment, all my clients pcs (not servers) are windows and need to access files over the network. All of these clients are joined to active directory emulation via Zentyal. I use FreeNas & OMV for their NAS features and ability to lock down file permission and provide an easy CIFS implementation. FreeNAS is also nice for automated snapshot, recovery, replication, etc..... My windows users especially like that you can role back snapshots via windows.
I am just curious how you manage the standard client network shares without a dedicated NAS OS..
Cli ? .....
- I am not sure what the use case is for the individual starting this thread
I can say in my environment, all my clients pcs (not servers) are windows and need to access files over the network. All of these clients are joined to active directory emulation via Zentyal. I use FreeNas & OMV for their NAS features and ability to lock down file permission and provide an easy CIFS implementation. FreeNAS is also nice for automated snapshot, recovery, replication, etc..... My windows users especially like that you can role back snapshots via windows.
I am just curious how you manage the standard client network shares without a dedicated NAS OS..
Cli ? .....
It's not a requirement of ZFS unless you want to take full advantage of all of ZFS's features, like the ability to self-heal the data on the file system.
I know that plenty will disagree and tell me this is foolish and ZFS is not designed to function this way, but in a real enterprise environment..... I bet you find this kind of setup more often than not.
Again just speaking from my experience and maybe sparking a little debate
Actually, I think you're more than likely correct. I mean, I think you're wrong to not only use a RAID controller with ZFS, but actually present ZFS with RAIDed storage. But on the point of how larger businesses use ZFS, if they use it at all, yeah you're probably right. Mainly for the same reason that RAID 5 stuck around as the industry standard loooooooong after people knew it was a terrible idea to use it: Because that's how it's always been done. Not because it's a good idea. But because that's what entrenched sys admins knew, and were comfortable with.
The only time I will ever touch another RAID controller is to replace it with ZFS.
The ZFS advanced checksum is definitely a feature you don't get putting ZFS over hardware raid.
On my FreeNAS box I run the disks through an HBA, but on Proxmox I was thinking of moving to ZFS over my perc raid controller. The advanced checksum is nice, but what I really am interested in is the better snapshot and replication feature of ZFS. I wonder from a performance standpoint whether I would have better IOPS by putting ZFS on top of my hardware raid vs passing the disks directly....
Any way I am digressing from the original post, so I will stop...
On my FreeNAS box I run the disks through an HBA, but on Proxmox I was thinking of moving to ZFS over my perc raid controller. The advanced checksum is nice, but what I really am interested in is the better snapshot and replication feature of ZFS. I wonder from a performance standpoint whether I would have better IOPS by putting ZFS on top of my hardware raid vs passing the disks directly....
Any way I am digressing from the original post, so I will stop...
There would be no need to run FreeNAS in a VM on Proxmox if Proxmox provided an even moderately decent way to manage and present shares. The FreeNAS gui and tools are what people want.
Until this is available people will still want to run a separate NAS platform in a VM.
The "official" position from the Proxmox team is that they supply a virtualization platform and supporting NAS functions this way is not part of their product plans (and some argue doing so would actually conflict with aspects of the core product). This is not an unreasonable position for them to take.
So - unless and until somebody offers something better than "Proxmox can run shares - you just have to manage them all by hand" there will continue to be a demand for this type of VM.
Until this is available people will still want to run a separate NAS platform in a VM.
The "official" position from the Proxmox team is that they supply a virtualization platform and supporting NAS functions this way is not part of their product plans (and some argue doing so would actually conflict with aspects of the core product). This is not an unreasonable position for them to take.
So - unless and until somebody offers something better than "Proxmox can run shares - you just have to manage them all by hand" there will continue to be a demand for this type of VM.
Last edited:
Vshaulsk, you can use webmin for all that you mention.
Since you are familiar with zential you could always move that to a container on proxmox...
There are tons of tools that can be sandboxed each in its own container (more versatile for maintenance/backup/upgrade)
You could also use a windows server vm on the proxmox host to access your underlying storage and do samba shareing/AD/User Management/previous versions etc natively.
Or even export all your data disks via iscsi and have the windows machine do the softraid.
On the other hand, if you absolutelly need to go for FreeNAS or OMV for one reason or the other,
just put them straight on your metal.
Proxmox is a hypervisor (with awesome features for storage, clustering, HA, etc)
It's not its job to do file sharing, user management...
That's for you to figure out how to implement best according to your needs.
That's why I've insisted on asking what you need the NAS for.
How many clients, how much and what kind of data etc...
So we can have a better understanding and propose the best solution...
Since you are familiar with zential you could always move that to a container on proxmox...
There are tons of tools that can be sandboxed each in its own container (more versatile for maintenance/backup/upgrade)
You could also use a windows server vm on the proxmox host to access your underlying storage and do samba shareing/AD/User Management/previous versions etc natively.
Or even export all your data disks via iscsi and have the windows machine do the softraid.
On the other hand, if you absolutelly need to go for FreeNAS or OMV for one reason or the other,
just put them straight on your metal.
Proxmox is a hypervisor (with awesome features for storage, clustering, HA, etc)
It's not its job to do file sharing, user management...
That's for you to figure out how to implement best according to your needs.
That's why I've insisted on asking what you need the NAS for.
How many clients, how much and what kind of data etc...
So we can have a better understanding and propose the best solution...
Last edited:
I completely forgot about webmin and using that solution for storage management.
Yeah in my case I have all the things you mentioned already running. NAS/SAN on bare metal and 4 node proxmox HA cluster. I already virtualize Zentyal, windows server, windows 10 virtual clients with nvme drive passed through to the vm, librenms, webservers + another 20 or so vm's. I have a couple of virtual OMV storage appliances running as well.
The person starting the thread wants to pass disk directly so they can run FreeNAS..... Vassilis pointed out that in theory they would be able to run proxmox with ZFS and use webmin for managing storage and shares. Pretty interesting idea, which I may think about playing around with.
Yeah in my case I have all the things you mentioned already running. NAS/SAN on bare metal and 4 node proxmox HA cluster. I already virtualize Zentyal, windows server, windows 10 virtual clients with nvme drive passed through to the vm, librenms, webservers + another 20 or so vm's. I have a couple of virtual OMV storage appliances running as well.
The person starting the thread wants to pass disk directly so they can run FreeNAS..... Vassilis pointed out that in theory they would be able to run proxmox with ZFS and use webmin for managing storage and shares. Pretty interesting idea, which I may think about playing around with.