Need help setting up extra storage.

[Irnda]

Hello,
Recently we got a new server and installed Proxmox on it. It has two nvme disks ( 2 x 500GB ).
The problem starts when we attempt to use the whole storage.
While with the second disk we had zero problems, with the main disk ( which is partitioned as the OS is set up on it), we are not able to use it.

gyazo dot com/a50085e5ba79ae6889ae5351610fc48d.png

nvme1n1 is working flawlessly, we're using it as lvm storage. However, we're not able to do the same with nvme0n1.

Here is some information so that maybe any of you can help us out as we're desperate .

fdisk -l
Disk /dev/nvme0n1: 419.2 GiB, 450098159616 bytes, 879097968 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 16969A16-0242-48B7-A6DC-AB1AD687C990

Device         Start       End   Sectors   Size Type
/dev/nvme0n1p1  2048 879097934 879095887 419.2G Linux LVM


Disk /dev/nvme1n1: 419.2 GiB, 450098159616 bytes, 879097968 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

With the case of nvme1n1 we only had to, pvcreate , vgcreate using the disk directly and it worked perfectly.
However, the moment we attempt to do the same with nvme0n1, it doesn't work.

pvcreate /dev/nvme0n1
  Can't open /dev/nvme0n1 exclusively.  Mounted filesystem?

So the option would be creating a partition on nvme0n1 with the space left.

gyazo dot  com/b3990f1d7a6768ece1113998fa76263d.png

However, when we do pvcreate, vgcreate ... the created partition only has 500M available storage which is what confuses me the most as when we cfdisk , it appears as 420G.

Any clue what could we be doing wrong?
Thank you in advance!

 

[oguz]

hi,

you can't create pvs on a disk device that's currently mounted/in use by your system (you can also see this in the first screenshot, it's mounted)

maybe nvme0n1 is the device you installed proxmox on in the first place?

see this link for more information[0]

[0]: https://www.thegeekdiary.com/lvm-error-cant-open-devsdx-exclusively-mounted-filesystem/

 

[Irnda]

That is correct, proxmox is installed on that disk.
The link that you provided, at first I thought would be the resolution, however

# pvscan
# vgscan
# lvscan

Them all end up returning the disk that we're already using, however, the problematic disk does never appear.

df -hP
Filesystem      Size  Used Avail Use% Mounted on
udev             63G     0   63G   0% /dev
tmpfs            13G  162M   13G   2% /run
/dev/nvme0n1p2   20G   13G  5.9G  68% /
tmpfs            63G   43M   63G   1% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            63G     0   63G   0% /sys/fs/cgroup
/dev/fuse        30M   20K   30M   1% /etc/pve

Maybe, is there any way we can expand local storage to use what's left on the disk?

 

[oguz]

hi,

all end up returning the disk that we're already using, however, the problematic disk does never appear

can you show me the output from `pvs -a` and `vgs -a`

also /etc/pve/storage.cfg

Maybe, is there any way we can expand local storage to use what's left on the disk?

you can use the lvm tools such as `lvextend`

 

[Irnda]

hi,



can you show me the output from `pvs -a` and `vgs -a`

also /etc/pve/storage.cfg



you can use the lvm tools such as `lvextend`

  PV             VG       Fmt  Attr PSize   PFree
  /dev/nvme0n1                 ---       0      0
  /dev/nvme0n1p2               ---       0      0
  /dev/nvme0n1p3               ---       0      0
  /dev/nvme1n1   fstorage lvm2 a--  419.18g 14.18g

vgs -a
  VG       #PV #LV #SN Attr   VSize   VFree
  fstorage   1   8   0 wz--n- 419.18g 14.18g

storage file:
dir: local
        path /var/lib/vz
        content vztmpl,images,snippets,rootdir,iso
        maxfiles 0

lvm: StorageFinal
        vgname fstorage
        content rootdir,images
        shared 0

Thanks in advance for your time, if there is no way for us to create the new partition I guess we can try lvextend , however, I will wait for your confirmation as I do not want to mess up even more

 

[Irnda]

Hello,
I still need help with this as extending the local storage does not seem possible.
Is it possible to get further assistance, please?

 

[oguz]

hi,

how did you try to extend the storage?

 

[Irnda]

hi,

how did you try to extend the storage?

lvextend -L400G /var/lib/vz

 

[Irnda]

Main problem is that the LVM disk present on my system is the second disk which is already used by other vm's.
The disk that we now want to use is the one that was used for installation, I believe about 80 out of 500G are used by installation and few ISO's that are on the local storage.
I found no way myself to implement that unused storage as lvm disk, however, I would not mind if local storage was expanded, anything to get rid of this headache really.
Thanks again for your help

 

[oguz]

/var/lib/vz is a directory type storage, so it can't be extended with lvextend.

i thought you wanted to extend your other storage, which is lvm.

it is a different process to extend the 'local' storage, if that's what you want.

can you send the output of:

pvesm status

 

[Irnda]

/var/lib/vz is a directory type storage, so it can't be extended with lvextend.

i thought you wanted to extend your other storage, which is lvm.

it is a different process to extend the 'local' storage, if that's what you want.

can you send the output of:

pvesm status

Name                Type     Status           Total            Used       Available        %
StorageFinal         lvm     active       439545856       424673280        14872576   96.62%
local                dir     active        20026236        12862972         6122932   64.23%

Please refer to my previous post, I think I made it more clear, sorry for not being like that since the beginning.

 

[oguz]

ok, i see what you mean now.

send the output of `lsblk /dev/nvme0n1`

if the p3 is empty, you should be able to create a new lvm there

 

[Irnda]

ok, i see what you mean now.

send the output of `lsblk /dev/nvme0n1`

if the p3 is empty, you should be able to create a new lvm there

NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
nvme0n1     259:0    0 419.2G  0 disk
├─nvme0n1p2 259:4    0  19.5G  0 part /
└─nvme0n1p3 259:5    0  1023M  0 part [SWAP]

 

[oguz]

so i really don't know how you did this (where is p1?), but if you have space at the end of the disk, do the following:

1. delete swap (p3)
2. extend p2 ext4 system to size you wish
3. add swap at the end if you wish

or:

1. delete swap (p3)
2. create a new partition of size you wish
3. put lvm or lvm-thin on it
4. add swap at the end if you wish

you should be able to use any partitioning tool (and the lvm tools if you want lvm) to do it, just search for guides on stackoverflow or similar.

 

[Irnda]

so i really don't know how you did this (where is p1?), but if you have space at the end of the disk, do the following:

1. delete swap (p3)
2. extend p2 ext4 system to size you wish
3. add swap at the end if you wish

or:

1. delete swap (p3)
2. create a new partition of size you wish
3. put lvm or lvm-thin on it
4. add swap at the end if you wish

you should be able to use any partitioning tool (and the lvm tools if you want lvm) to do it, just search for guides on stackoverflow or similar.

fdisk -l /dev/nvme0n1
Disk /dev/nvme0n1: 419.2 GiB, 450098159616 bytes, 879097968 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 16969A16-0242-48B7-A6DC-AB1AD687C990

Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 879097934 879095887 419.2G Linux filesystem

There is p1, it is impossible for me to use it as storage or use its space at all.
Whenever I try to use it, the system says its mounted or busy by the system.

 

[Irnda]

Hello,
Is there anything we can do about it? I tried researching as much as possible and I can't find any similar problem