In a bit of a panic mode here so I will try to put in everything I can think of and sorry if I missed a thread or any other needed info, please link me to it or let me know and I will follow up.
Been running a Proxmox server for a few years now and finally needed to upgrade my OS HDD... I cloned a 250GB SSD onto a 1TB SSD, got the drive up and running and expanded as well as expanding one of my VM's. I then left the state...
Today I got some calls that my services were down and when I jump on the machine I had read only errors on the VM and had to repair... Spent some time on this and decided I have docker containers so I would just clone the drive and reinstall the OS... Thats when I hit the wall...
ERROR:
update VM 102: -virtio3 local-lvm:250
WARNING: Remaining free space in metadata of thin pool pve/data is too low (100.00% >= 94.74%). Resize is recommended.
TASK ERROR: lvcreate 'pve/vm-102-disk-1' error: Cannot create new thin volume, free space in thin pool pve/data reached threshold.
So I went looking and found:
root@pve:~# pvs
PV VG Fmt Attr PSize PFree
/dev/sda1 Prox-Backup lvm2 a-- <3.64t <1.58t
/dev/sdb1 Raid-6 lvm2 a-- <25.47t <1.45t
/dev/sdc3 pve lvm2 a-- <931.26g 0
root@pve:~# lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
data pve twi-aotzM- <865.11g 32.44 100.00
root pve -wi-ao---- 58.00g
swap pve -wi-ao---- 8.00g
root@pve:~# lvdisplay /dev/pve/data
--- Logical volume ---
LV Name data
VG Name pve
LV UUID
LV Write Access read/write
LV Pool metadata data_tmeta
LV Pool data data_tdata
LV Status available
# open 3
LV Size <865.11 GiB
Allocated pool data 32.44%
Allocated metadata 100.00%
Current LE 221468
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:4
root@pve:~# vgdisplay pve
--- Volume group ---
VG Name pve
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 54
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 5
Open LV 4
Max PV 0
Cur PV 1
Act PV 1
VG Size <931.26 GiB
PE Size 4.00 MiB
Total PE 238402
Alloc PE / Size 238402 / <931.26 GiB
Free PE / Size 0 / 0
So I went to googling and I found a few threads to educate me some on LVM Metadata (always learning so please feel free to poing out anything I missed PLEASE)... I found the following thread: Increase lvm thin what about metadata
Which I BELEIEVE leads me to the fact that the only way to fix this is to re-create the LVM in order to fix the metadata issue. This is where the questions and problems actually start...
Since I am not in the same state as the server I have no physical access to it, so is there a way to recreate this without losing all my data?
Or...
Do I need to buy and setup another 1TB HDD (I actaully have a clone of the server local to me so all chipset and such would be the same) and ship it and have someone install it and start copying over my VM's and setting's?
ANY help here would be greatly appreciated...
Been running a Proxmox server for a few years now and finally needed to upgrade my OS HDD... I cloned a 250GB SSD onto a 1TB SSD, got the drive up and running and expanded as well as expanding one of my VM's. I then left the state...
Today I got some calls that my services were down and when I jump on the machine I had read only errors on the VM and had to repair... Spent some time on this and decided I have docker containers so I would just clone the drive and reinstall the OS... Thats when I hit the wall...
ERROR:
update VM 102: -virtio3 local-lvm:250
WARNING: Remaining free space in metadata of thin pool pve/data is too low (100.00% >= 94.74%). Resize is recommended.
TASK ERROR: lvcreate 'pve/vm-102-disk-1' error: Cannot create new thin volume, free space in thin pool pve/data reached threshold.
So I went looking and found:
root@pve:~# pvs
PV VG Fmt Attr PSize PFree
/dev/sda1 Prox-Backup lvm2 a-- <3.64t <1.58t
/dev/sdb1 Raid-6 lvm2 a-- <25.47t <1.45t
/dev/sdc3 pve lvm2 a-- <931.26g 0
root@pve:~# lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
data pve twi-aotzM- <865.11g 32.44 100.00
root pve -wi-ao---- 58.00g
swap pve -wi-ao---- 8.00g
root@pve:~# lvdisplay /dev/pve/data
--- Logical volume ---
LV Name data
VG Name pve
LV UUID
LV Write Access read/write
LV Pool metadata data_tmeta
LV Pool data data_tdata
LV Status available
# open 3
LV Size <865.11 GiB
Allocated pool data 32.44%
Allocated metadata 100.00%
Current LE 221468
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:4
root@pve:~# vgdisplay pve
--- Volume group ---
VG Name pve
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 54
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 5
Open LV 4
Max PV 0
Cur PV 1
Act PV 1
VG Size <931.26 GiB
PE Size 4.00 MiB
Total PE 238402
Alloc PE / Size 238402 / <931.26 GiB
Free PE / Size 0 / 0
So I went to googling and I found a few threads to educate me some on LVM Metadata (always learning so please feel free to poing out anything I missed PLEASE)... I found the following thread: Increase lvm thin what about metadata
Which I BELEIEVE leads me to the fact that the only way to fix this is to re-create the LVM in order to fix the metadata issue. This is where the questions and problems actually start...
Since I am not in the same state as the server I have no physical access to it, so is there a way to recreate this without losing all my data?
Or...
Do I need to buy and setup another 1TB HDD (I actaully have a clone of the server local to me so all chipset and such would be the same) and ship it and have someone install it and start copying over my VM's and setting's?
ANY help here would be greatly appreciated...