Can't Create RAIDZ Storage Pool of 5x 4TB drives

RangerX

New Member
Jan 15, 2023
4
0
1
www.tao-of-law.com
Hi,
I'm new to Proxmox, very new in fact. I just built my new file server from an AMD 2950x Threadripper on an ASROCK X399 Fatal1ty Pro Gaming mobo with 64GB DDR4-3000 RAM. My boot drive is a Samsung 960 Pro 512GB NVME, and I have two Samsung 980 Pro 1TB NVMEs I want to use as RAID 0 NVME cache. It has a Gigabyte Aorus 1080 Ti video card for video editing once I also figure out how to use nVidia's GPU boosting VM software in Proxmox.

However, I can't get far enough along to worry about how to set up VM container's or the NVME cache due to the fact that I cannot get Proxmox to even create my initial drive pool. I currently have 5x 4TB HDDs directly attached to the mobo, which has 8 SATA connections. All of the 4TB HDDs SMART reports show them to be healthy and working perfectly. In addition to these 5 4TB HDDs, I also have the remaining three mobo SATA connections filled with three (3) Samsung 860 Pro 512GB 2.5" SSDs. Finally, I have added a new 24 port SATA 16x PCIE card to add new drives as I can afford them. The new card IS installed but there are no drives currently attached to it. The card is NOT a hardware RAID controller, but it is engineered expressly for use as a soft-RAID controller card.

My planned configuration is (was) as follows:

5x 4TB HDDs set as RAIDz (4x are WD HDDS, 1x is Toshiba), which I understand to be the equivalent of RAID5.
2x 980 Pro 1TB NVMEs set as RAID0 and used as NVME cache (once I figure out how to do this that is).
3x 860 Pro 512GB SSDs set as RAID0 for doing video editing and encoding from a VE Docker container running Adobe Premier (again, once I figure out how to do this too).

Eventually the 24 port SATA card will have a full compliment of 18TB HDDs combined into a single large pool using three different pools containing eight (8) HDDs each in RAIDZ. At least, that's the plan.

The drives I currently have installed and am trying to use are shown in the attached picture 1.

The PROBLEM:

After using fdisk to ensure all the 4TB drives were clean and empty of partitions, I attempted to create the RAIDZ pool. However, the pool creation fails with the error shown in the attached picture 2.

Since Proxmox is supposedly capable of using drives of differing sizes and brands, I didn't think it would matter that one of these 4TB drives is a Toshiba while the other four are identical Western Digital drives. And it may very well have nothing to do with this error. Regardless, its a bit difficult for a newbie to Proxmox to understand exactly WHAT caused the error since the error message as displayed provides, to my eyes at least, no real information or indication of what failed or why.

I would REALLY like to get this working so I can know what to do if I encounter a similar problem later when I start adding the much larger capacity 18TB HDD pools. Can anyone offer a solution or a suggestion on where to find one so I can understand this cryptic error message and get this all working correctly?

Thank you all in advance for any suggestions/help you offer. It is appreciated.
 

Attachments

  • Proxmox Error when failing to create RAIDZ pool 0001.png
    Proxmox Error when failing to create RAIDZ pool 0001.png
    42.8 KB · Views: 8
  • Proxmox Error when failing to create RAIDZ pool 0002.png
    Proxmox Error when failing to create RAIDZ pool 0002.png
    26.9 KB · Views: 8
could you post the full task log?
 
I'm not sure if this is the info you want or not (I'm VERY new to Proxmox), butr I hope so.

Under my tasks tab there is a line item for the creation of the RAIDZ pool that shows an Error status. Double-clicking the line item shows me this:


Jan 16 02:41:11Jan 16 02:41:44root@pamUpdate package databaseOK
Jan 16 00:39:58Jan 16 00:39:59root@pamZFS Storage R5-5x-4TB - CreateError: command '/sbin/zpool create -o 'ashift=12' R5-5x-4TB raidz /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0103278 /dev/disk/by-id/ata-WDC_WD40EZRZ-00WN9B0_WD-WCC4E6ATLLF8 /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WMC1F0087665 /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0058151 /dev/disk/by-id/ata-TOSHIBA_HDWE140_85A8K7ZNF58D' failed: exit code 1
Jan 16 00:37:55Jan 16 00:37:56root@pamZFS Storage XXX_5x4TB - CreateError: command '/sbin/zpool create -o 'ashift=12' XXX_5x4TB raidz /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0103278 /dev/disk/by-id/ata-WDC_WD40EZRZ-00WN9B0_WD-WCC4E6ATLLF8 /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WMC1F0087665 /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0058151 /dev/disk/by-id/ata-TOSHIBA_HDWE140_85A8K7ZNF58D' failed: exit code 1
Jan 16 00:29:24Jan 16 00:33:24root@pamShellOK
Jan 16 00:28:56Jan 16 00:29:17root@pamShellOK
Jan 16 00:28:13Jan 16 00:28:29root@pamShellOK
Jan 16 00:11:27Jan 16 00:24:30root@pamShellOK
Jan 16 00:09:54Jan 16 00:09:58root@pamUpdate package databaseOK
Jan 16 00:09:08Jan 16 00:09:11root@pamUpdate package databaseOK
Jan 16 00:08:54Jan 16 00:08:54root@pamStart all VMs and ContainersOK
Jan 16 00:05:04Jan 16 00:08:54root@pamShellOK
Jan 16 00:06:59Jan 16 00:06:59root@pamStop all VMs and ContainersOK
Jan 16 00:03:51Jan 16 00:03:54root@pamUpdate package databaseOK
Jan 16 00:01:41Jan 16 00:01:44root@pamUpdate package databaseError: command 'apt-get update' failed: exit code 100
Jan 15 20:50:57Jan 15 20:50:57root@pamStart all VMs and ContainersOK
Jan 15 16:53:34Jan 15 16:53:34root@pamStop all VMs and ContainersOK
Jan 15 10:31:00Jan 15 10:31:00root@pamStart all VMs and ContainersOK

Logs
TasksCluster log

Start Time
End Time
Node
User name
Description
Status

Jan 16 02:41:11Jan 16 02:41:44pve0root@pamUpdate package databaseOK
Jan 16 00:39:58Jan 16 00:39:59pve0root@pamZFS Storage R5-5x-4TB - CreateError: command '/sbin/zpool create -o 'ashift=12' R5-5x-4TB raidz /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0103278 /dev/disk/by-id/ata-WDC_WD40EZRZ-00WN9B0_WD-WCC4E6ATLLF8 /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WMC1F0087665 /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0058151 /dev/disk/by-id/ata-TOSHIBA_HDWE140_85A8K7ZNF58D' failed: exit code 1
Jan 16 00:37:55Jan 16 00:37:56pve0root@pamZFS Storage XXX_5x4TB - CreateError: command '/sbin/zpool create -o 'ashift=12' XXX_5x4TB raidz /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0103278 /dev/disk/by-id/ata-WDC_WD40EZRZ-00WN9B0_WD-WCC4E6ATLLF8 /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WMC1F0087665 /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0058151 /dev/disk/by-id/ata-TOSHIBA_HDWE140_85A8K7ZNF58D' failed: exit code 1
Jan 16 00:29:24Jan 16 00:33:24pve0root@pamShellOK
Jan 16 00:28:56Jan 16 00:29:17pve0root@pamShellOK
Jan 16 00:28:13Jan 16 00:28:29pve0root@pamShellOK
Jan 16 00:11:27Jan 16 00:24:30pve0root@pamShellOK
Jan 16 00:09:54Jan 16 00:09:58pve0root@pamUpdate package databaseOK
Jan 16 00:09:08Jan 16 00:09:11pve0root@pamUpdate package databaseOK
Jan 16 00:08:54Jan 16 00:08:54pve0root@pamStart all VMs and ContainersOK
Jan 16 00:06:59Jan 16 00:06:59pve0root@pamStop all VMs and ContainersOK
Jan 16 00:05:04Jan 16 00:08:54pve0root@pamShellOK
Jan 16 00:03:51Jan 16 00:03:54pve0root@pamUpdate package databaseOK
Jan 16 00:01:41Jan 16 00:01:44pve0root@pamUpdate package databaseError: command 'apt-get update' failed: exit code 100
Jan 15 20:50:57Jan 15 20:50:57pve0root@pamStart all VMs and ContainersOK
Jan 15 16:53:34Jan 15 16:53:34pve0root@pamStop all VMs and ContainersOK
Jan 15 10:31:00Jan 15 10:31:00pve0root@pamStart all VMs and ContainersOK

Task viewer: ZFS Storage R5/5x/4TB - Create


OutputStatus

Stop
# /sbin/zpool create -o ashift=12 R5-5x-4TB raidz /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0103278 /dev/disk/by-id/ata-WDC_WD40EZRZ-00WN9B0_WD-WCC4E6ATLLF8 /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WMC1F0087665 /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0058151 /dev/disk/by-id/ata-TOSHIBA_HDWE140_85A8K7ZNF58D
invalid vdev specification
use '-f' to override the following errors:
raidz contains devices of different sizes
TASK ERROR: command '/sbin/zpool create -o 'ashift=12' R5-5x-4TB raidz /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0103278 /dev/disk/by-id/ata-WDC_WD40EZRZ-00WN9B0_WD-WCC4E6ATLLF8 /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WMC1F0087665 /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0058151 /dev/disk/by-id/ata-TOSHIBA_HDWE140_85A8K7ZNF58D' failed: exit code 1
 
Code:
invalid vdev specification
use '-f' to override the following errors:
raidz contains devices of different sizes

you will have to create the pool manually (basically using the same command, but with -f added) after verifying that the sizes are different, but close enough that you don't care. please also read https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#sysadmin_zfs_raid_considerations before setting up your pool using raidz!
 
Code:
invalid vdev specification
use '-f' to override the following errors:
raidz contains devices of different sizes

you will have to create the pool manually (basically using the same command, but with -f added) after verifying that the sizes are different, but close enough that you don't care. please also read https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#sysadmin_zfs_raid_considerations before setting up your pool using raidz!
Is this the command that you are referring to when you say "you will have to create the pool manually (basically using the same command, but with -f added"?

/sbin/zpool create -f -o ashift=12 R5-5x-4TB raidz /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0103278 /dev/disk/by-id/ata-WDC_WD40EZRZ-00WN9B0_WD-WCC4E6ATLLF8 /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WMC1F0087665 /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0058151 /dev/disk/by-id/ata-TOSHIBA_HDWE140_85A8K7ZNF58D
 
Last edited:
Is this the command that you are referring to when you say "you will have to create the pool manually (basically using the same command, but with -f added"?

/sbin/zpool create -f -o ashift=12 R5-5x-4TB raidz /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0103278 /dev/disk/by-id/ata-WDC_WD40EZRZ-00WN9B0_WD-WCC4E6ATLLF8 /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WMC1F0087665 /dev/disk/by-id/ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0058151 /dev/disk/by-id/ata-TOSHIBA_HDWE140_85A8K7ZNF58D
Okay, that command worked and made the storage pool, so thank you. Now all I have to do is figure out these things:

1. Does the pool have to be partitioned and formatted?
2. If #1 is "yes," how do I do that?
3. Once it's ready, how do I map a network drive from a different Windows PC to use the pool on Proxmox?

I'll keep reading and video binging. Thank you again for the assist.
 
Okay, that command worked and made the storage pool, so thank you. Now all I have to do is figure out these things:

1. Does the pool have to be partitioned and formatted?
a zpool consists of datasets (either filesystem or volume ones). it makes sense to create different datasets for different use cases, since each dataset can have different properties (quotas, compression, ..) as well as be snapshotted separately. when configured as storage in PVE, sub-datasets will be created for each PVE-managed guest volume on that storage.

I'd recommend at least setting up one dataset for your PVE storage
2. If #1 is "yes," how do I do that?
zfs create POOL/DATASET, e.g. "zfs create r5-5x-4TB/storage". see "man zfs-create" and "man zfsprops".
3. Once it's ready, how do I map a network drive from a different Windows PC to use the pool on Proxmox?
if you want to make a dataset available for other hosts over the network, NFS or CIFS are the usual solutions. but that is not really related to PVE ;)