[SOLVED] need help to recover files from proxmox drives

[sunnyboy]

Hi,
one of my proxmox servers died - no chance to get into bios.
I want to recover some files and configurations from the disks by attaching them via USB to another Proxmox server.
Now I focus 2 challenges.

1.) get conf files of the VMs and CTs. I could mount root lvm (/dev/pve/root), but how to get access to /dev/pve/... normally mounted on /dev/fuse

pvs
  PV         VG   Fmt  Attr PSize    PFree
  /dev/sdb   VG01 lvm2 a--   232.88g 232.88g
  /dev/sdd3  pve  lvm2 a--  <372.11g  15.99g <=== boot disk of defect server

lvdisplay pve
  --- Logical volume ---
  LV Path                /dev/pve/swap
  LV Name                swap
  VG Name                pve
...
  --- Logical volume ---
  LV Path                /dev/pve/root
  LV Name                root
  VG Name                pve
... 

  --- Logical volume ---
  LV Name                data
  VG Name                pve

mount /dev/pve/root ~/pve_root

...and then???

2.) get some data from a CT container file located on a ZFS disk:
I attached the ssd with ZFS filesystem on it via usb 3.0 adapter. But how can I access the ZFS filesystem and the files on it???
Is there a kind of fschk for ZFS drives?

[  111.755943] scsi 5:0:0:0: Direct-Access     CT2000MX 500SSD1          0    PQ: 0 ANSI: 6
[  111.757146] sd 5:0:0:0: Attached scsi generic sg3 type 0
[  111.757609] sd 5:0:0:0: [sdd] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB)
[  111.758287] sd 5:0:0:0: [sdd] 4096-byte physical blocks
[  111.758959] sd 5:0:0:0: [sdd] Write Protect is off
[  111.759506] sd 5:0:0:0: [sdd] Mode Sense: 43 00 00 00
[  111.759681] sd 5:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[  111.761018] sd 5:0:0:0: [sdd] Optimal transfer size 33553920 bytes not a multiple of physical block size (4096 bytes)
[  111.807488]  sdd: sdd1 sdd9
[  111.849837] sd 5:0:0:0: [sdd] Attached SCSI disk

And I got this mail when trying to access the drive:

The number of I/O errors associated with a ZFS device exceeded
acceptable levels. ZFS has marked the device as faulted.

 impact: Fault tolerance of the pool may be compromised.
    eid: 19
  class: statechange
  state: FAULTED
   host: pve02
   time: 2022-03-10 17:40:54+0100
  vpath: /dev/disk/by-id/ata-CT2000MX500SSD1_2153E5F9E54E-part1
  vphys: pci-0000:00:17.0-ata-1.0
  vguid: 0x87F3B0A7339D81C6
  devid: ata-CT2000MX500SSD1_2153E5F9E54E-part1
   pool: 0x0AAA3FF933C43624

Help would be appreciated.
Thanks, Dieter

 

[Dunuin]

Hi,
one of my proxmox servers died - no chance to get into bios.
I want to recover some files and configurations from the disks by attaching them via USB to another Proxmox server.
Now I focus 2 challenges.

1.) get conf files of the VMs and CTs. I could mount root lvm (/dev/pve/root), but how to get access to /dev/pve/... normally mounted on /dev/fuse

pvs
  PV         VG   Fmt  Attr PSize    PFree
  /dev/sdb   VG01 lvm2 a--   232.88g 232.88g
  /dev/sdd3  pve  lvm2 a--  <372.11g  15.99g <=== boot disk of defect server

lvdisplay pve
  --- Logical volume ---
  LV Path                /dev/pve/swap
  LV Name                swap
  VG Name                pve
...
  --- Logical volume ---
  LV Path                /dev/pve/root
  LV Name                root
  VG Name                pve
...

  --- Logical volume ---
  LV Name                data
  VG Name                pve

mount /dev/pve/root ~/pve_root

...and then???

/etc/pve is the mountpoint of the pmxcfs filesystem which will be mounted by the pve-cluster service. Aa far as I know it will read the configs stored in a SQLite DB and present it as files in that mountpoint using fuse. See here: https://pve.proxmox.com/wiki/Proxmox_Cluster_File_System_(pmxcfs)

Seen several people asking here how to get to the old configs but as far as I know no one actually got it working.

2.) get some data from a CT container file located on a ZFS disk:
I attached the ssd with ZFS filesystem on it via usb 3.0 adapter. But how can I access the ZFS filesystem and the files on it???
Is there a kind of fschk for ZFS drives?

[  111.755943] scsi 5:0:0:0: Direct-Access     CT2000MX 500SSD1          0    PQ: 0 ANSI: 6
[  111.757146] sd 5:0:0:0: Attached scsi generic sg3 type 0
[  111.757609] sd 5:0:0:0: [sdd] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB)
[  111.758287] sd 5:0:0:0: [sdd] 4096-byte physical blocks
[  111.758959] sd 5:0:0:0: [sdd] Write Protect is off
[  111.759506] sd 5:0:0:0: [sdd] Mode Sense: 43 00 00 00
[  111.759681] sd 5:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[  111.761018] sd 5:0:0:0: [sdd] Optimal transfer size 33553920 bytes not a multiple of physical block size (4096 bytes)
[  111.807488]  sdd: sdd1 sdd9
[  111.849837] sd 5:0:0:0: [sdd] Attached SCSI disk

Typically LXCs use ZFS datasets. So first I would import the pool (zpool import yourPool). Then you can mount the datasets ("zfs mount" command) if they aren'T already mounted.

 

[sunnyboy]

Thanks. Glad my hardware expert got my testserver back to live.
Took immediately backups to prevent such a disaster in future