[SOLVED] After installing VE I am unable to bind Storage SSD to ZFS

[VMnoob]

Hello Proxmox community!

It's my first Thread on this Forum, I couldn't find any simmilar Thread with search engine, so I started a new one.

Firstly specs of my Hypervisor which is used for learning purpouse:
1x SSD 128GB for VE - it's barely filled up after installation but is it a thing to be concerned with?
2x SSD 1TB for VM's (no RAID hardware enabled)

So... I've done a default installation and the things are set and running. After that I wanted to bundle two 1TB SSD's to a ZFS mirror. I've noticed I have no disks to choose and those two disks appeared as sdb and sdc (Partition mode) on LVM... How can I fix that issue? Can it be done through GUI? Was it a mistake on installation? I've reviewed it and I see no option on Target Harddisk screen.

P.S. My Linux expertise is very low, so if I need to go to shell, can you provide some commands referring to get it on right tracks? It will be much appreciated

 

[apoc]

Have these 1TB SSDs already be in use in another system and maybe some existing partitions on them or are they brand new?

 

[VMnoob]

Have these 1TB SSDs already be in use in another system and maybe some existing partitions on them or are they brand new?

Hi Tburger,

They are reused. Former an NTFS Windows-Mirror disc.

 

[apoc]

Well in this case PVE tries to protect you to do something stupid and does not allow the usage of these disks because there is something on it.
As far as I can say there are 2 options.
1. Do some CLI stuff
2. Attach these disks to a system, remove any partition and after that it should allow you to use these disks from GUI.
What's your choice?

 

[VMnoob]

Okay, that makes sense, I will try to format it from VE shell. It will be quicker I think.

 

[VMnoob]

Thanks Tburger, that did the job! Well, at least partly.
Here's what I've done for both discs:
1) cfdisk /dev/sdaX
2) mkfs.ext3 /dev/sdaX
3) mkdir /media/sdaX
mount /dev/sdaX /media/sdaX
4) edited /etc/fstab:

/dev/sda2 /media/sda2 ext rw 0 0
/dev/sda3 /media/sda3 ext rw 0 0

After that both discs are visible on ZFS I tried to make an mirror and got... an error:
"command '/sbin/zpool create -o 'ashift=12' storage mirror /dev/disk/by-id/ata-SanDisk_SDSSDA-1T00_1946C1802800 /dev/disk/by-id/ata-SanDisk_SDSSDA-1T00_1946C1803133' failed: exit code 1"

 

[VMnoob]

Got it! That was in another thread: https://forum.proxmox.com/threads/create-zfs-fails-gui.49761/
I copy-pasted the command output from error message and added -f and voila!
Tburger thx a lot!

 

[apoc]

I am a little late in responding but you seem OK
Please notice that ZFS consumes a lot of memory sometimes so you should do yourself a favour and limit the arc usage.
You should be able to find that info easily here in the forum (search for "arc limit").
I'd set the limit to 1gb in your case -your pool is not very large

 

[VMnoob]

I am a little late in responding but you seem OK
Please notice that ZFS consumes a lot of memory sometimes so you should do yourself a favour and limit the arc usage.
You should be able to find that info easily here in the forum (search for "arc limit").
I'd set the limit to 1gb in your case -your pool is not very large

Yeah, it's not so big at all... To be honest I've runned on PVE on my search where can I virtualize pfSense... I have now 24GB RAM but it will change in 2 weeks so I get max. of 96GB DDR3, because the VM's which I will be running (for now, ha-ha) will need more RAM and at this point and for now 1TB storage should be fine. I have readed that ZFS arc (caching?) doesn't block RAM usage for VM but it can really eat some chunks of it mantaining highest IOPS. We are now getting off the topic, but I think you know how I could monitor arc behaviour and its influence on VM's. I keep your advise in my mind, thanks once again!

 

[VMnoob]

So just to test "arc limit" according to Tburger advice...
I've created /etc/modprobe.d/zfs.conf and added line:
options zfs zfs_arc_max=1073741824
The difference in RAM usage comes right after reboot - see attached screenshot:
Before: 12,53 GB
After: 2,17GB (reduce 512MB while first VM up and running)

 

[apoc]

just another side-note: you also should set
options zfs zfs_arc_min=1073741824
I have read in some places that sometimes otherwise the wrong value might be pulled (e.g. if min is larger than max)