Hey guys, I could really use some fresh thoughts on this. I have a Proxmox VE server running on an intel machine in a data center where I have no access to the console screen. This server has been running great for about a year and has randomly stopped responding. Before the hard disk was becoming to full and I'd have to SSH the machine to remove some files to allow it to boot again. I went to navigate my machine the other day and there was no response from the website. So I checked the Proxmox WEB GUI and response there either. I loaded up SSH and tried that as well no response. I rebooted the machine and tried many hours later no luck. I got Sysrcd 4.3.1 and GRML 2014.11 for recovery for software to try to recover it.
Could my GRUB configuration be corrupted?
Proxmox VE runs off of the 120GB SSD and I host the machines off the 2TB drive.
I am able to mount the root directory through GRML unlike Sysrcd was saying bad super block(due to LVM_member?).
/dev/sda1 Seems to be boot, where the grub and kernel files are located.
/dev/mon0-vg/root being the root but when I mounted it was read-only.
Any and all help would be appreciated I've been stuck for days.
Could my GRUB configuration be corrupted?
Code:
root@s154670 ~ # fdisk -l :(
Disk /dev/sda: 119.2 GiB, 128035676160 bytes, 250069680 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0de70c91
Device Boot Start End Sectors Size Id Type
/dev/sda1 * 2048 499711 497664 243M 83 Linux
/dev/sda2 501758 250068991 249567234 119G 5 Extended
/dev/sda5 501760 250068991 249567232 119G 8e Linux LVM
Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/loop0: 416.7 MiB, 436916224 bytes, 853352 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/mon2--vg-vm--101--disk--0: 1.8 TiB, 1905891737600 bytes, 3722444800 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 9027EBE6-5D8B-4C03-A561-EB3C518F0ED2
Device Start End Sectors Size Type
/dev/mapper/mon2--vg-vm--101--disk--0p1 34 32767 32734 16M Microsoft reserved
/dev/mapper/mon2--vg-vm--101--disk--0p2 32768 3722440703 3722407936 1.8T Microsoft basic data
Partition 2 does not start on physical sector boundary.
Disk /dev/mapper/mon0--vg-root: 103.3 GiB, 110901592064 bytes, 216604672 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/mon0--vg-swap_1: 15.7 GiB, 16873684992 bytes, 32956416 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Code:
root@s154670 ~ # pvdisplay
--- Physical volume ---
PV Name /dev/sdb
VG Name mon2-vg
PV Size 1.82 TiB / not usable 1.09 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 476932
Free PE 22532
Allocated PE 454400
PV UUID 3faaCw-3LoU-IxjT-CdMC-zJJQ-Hhss-pL5mml
--- Physical volume ---
PV Name /dev/sda5
VG Name mon0-vg
PV Size 119.00 GiB / not usable 3.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 30464
Free PE 0
Allocated PE 30464
PV UUID QegEmk-zu0q-jBgQ-leXX-yBEj-3nAq-Y5ncQN
Code:
root@s154670 ~ # vgchange -ay
1 logical volume(s) in volume group "mon2-vg" now active
2 logical volume(s) in volume group "mon0-vg" now active
root@s154670 ~ # lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cp y%Sync Convert
root mon0-vg -wi-a----- 103.29g
swap_1 mon0-vg -wi-a----- 15.71g
vm-101-disk-0 mon2-vg -wi-a----- 1.73t
root@s154670 ~ # mount /dev/mon0-vg/root /mnt -o ro,userProxmox VE runs off of the 120GB SSD and I host the machines off the 2TB drive.
I am able to mount the root directory through GRML unlike Sysrcd was saying bad super block(due to LVM_member?).
/dev/sda1 Seems to be boot, where the grub and kernel files are located.
/dev/mon0-vg/root being the root but when I mounted it was read-only.
Any and all help would be appreciated I've been stuck for days.