MDADM create with NVMe failed - Floating point exception

sdssdice

Member
Nov 24, 2015
14
0
21
Good day everybody,

we would like to subsequently set up a software raid 1 on our Proxmox system with mdadm.
Proxmox itself is currently installed on an NVMe with ext4, which is now to be mirrored on another NVMe.

However, the following error occurs.
"Floating point exception"

root@pr-001:~# mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/nvme0n1p1 /dev/nvme1n1p1 mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90 Continue creating array? y mdadm: Defaulting to version 1.2 metadata Floating point exception

I have already done a wipe on the second NVMe and taken over the partition table again.
Code:
sfdisk --dump /dev/nvme0n1 | sfdisk /dev/nvme1n1

Versions:
Proxmox: 7.1-7
mdadm: 4.1


Unfortunately I couldn't find anything concrete about this, can someone help with this problem?
 

LnxBil

Famous Member
Feb 21, 2015
6,481
824
163
Saarland, Germany
Are you currently booting from /dev/nvme0n1p1? If so, you cannot do that with 1.2 metadata, as it is described in the output.

Normally you install with a single-disk mdadm raid and expand it afterwards. Metadata 1.2 has other data "in between", so that a simple "after-install-change" to mdadm is not possible anymore.

Why do you use mdadm over ZFS?
 

sdssdice

Member
Nov 24, 2015
14
0
21
thanks @LnxBil for your answer.

yes, we use /dev/nvme0n1p1 as boot device. For the Raid 1 setup, according to the output of mdadm, would metadata 0.9 be a big disadvantage for us in comparison?

ext4 is required for our use case, unfortunately it is not possible to create a raid under ext4 during the installation.
 

sdssdice

Member
Nov 24, 2015
14
0
21
What do you mean by disadvantage? You cannot use anything else than 0.9.
I meant generally in comparison 0.9 to 1.2
Yes, this is an unsupported setup. What do you need ext4 for?
We would like to run a Ceph monitor on the Proxmox system. As far as we know, ext4 is best suited for this. ZFS also needs more RAM in comparison. The important thing here is that the Ceph monitor is also on the Proxmox partition. Do you have a good idea for this use-case?
 

LnxBil

Famous Member
Feb 21, 2015
6,481
824
163
Saarland, Germany
We would like to run a Ceph monitor on the Proxmox system. As far as we know, ext4 is best suited for this. ZFS also needs more RAM in comparison. The important thing here is that the Ceph monitor is also on the Proxmox partition. Do you have a good idea for this use-case?
You have at least three nodes with this setup? Do you want to use the NVMe also as OSD? Could you please describe what do you plan to deploy (nodes, #OSD etc).
 

sdssdice

Member
Nov 24, 2015
14
0
21
You have at least three nodes with this setup? Do you want to use the NVMe also as OSD? Could you please describe what do you plan to deploy (nodes, #OSD etc).

Yes, more than 3. These two NVMs drives are only for OS and Ceph MON, we have enough other OSDs separately.
 

LnxBil

Famous Member
Feb 21, 2015
6,481
824
163
Saarland, Germany
Yes, more than 3. These two NVMs drives are only for OS and Ceph MON, we have enough other OSDs separately.
If you go with PVE and want to have support, I suggest using ZFS, which is the only viable and supported option. ZFS itself is much more versatile than ext4 and you can put your monitor on its own dataset. The caching in RAM depends on the size of the data and how much it is read. In general, you can set an upper limit to the memory ZFS uses, this is e.g. not possible with ext4. You can even configure your ZFS dataset with the monitor to only cache metadata and not data itself making it using even less ram. But caching stuff is always good, so that is not advised.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!