[SOLVED] Stuck with weird external USB drive

C

charleslcso

Member
Oct 1, 2022
37
1
13
The external 8TB USB drive is kinda flaky...

Now I am stuck in this:

Code: 
root@360g9:~# ls -l /mnt/tosh-8tb-dl360/
ls: reading directory '/mnt/tosh-8tb-dl360/': Input/output error
total 0

Code: 
root@360g9:~# umount /mnt/tosh-8tb-dl360
umount: /mnt/tosh-8tb-dl360: target is busy.

Code: 
root@360g9:~# eject /dev/sde
eject: /dev/sde: not found mountpoint or device with the given name

Code: 
cat /var/log/syslog | grep sde
Dec 29 00:40:02 360g9 kernel: [8786957.926386] EXT4-fs error (device sde): __ext4_find_entry:1682: inode #2: comm rsync: reading directory lblock 0
Dec 29 00:40:02 360g9 kernel: [8786958.370784] EXT4-fs error (device sde): __ext4_find_entry:1682: inode #2: comm rsync: reading directory lblock 0
Dec 29 00:40:02 360g9 kernel: [8786958.371167] EXT4-fs error (device sde): __ext4_find_entry:1682: inode #2: comm rsync: reading directory lblock 0
Dec 29 00:40:02 360g9 kernel: [8786958.371437] EXT4-fs error (device sde): __ext4_find_entry:1682: inode #2: comm rsync: reading directory lblock 0
Dec 29 00:40:02 360g9 kernel: [8786958.371679] EXT4-fs error (device sde): __ext4_find_entry:1682: inode #2: comm rsync: reading directory lblock 0
Dec 29 00:40:02 360g9 kernel: [8786958.371912] EXT4-fs error (device sde): __ext4_find_entry:1682: inode #2: comm rsync: reading directory lblock 0
Dec 29 00:40:02 360g9 kernel: [8786958.372132] EXT4-fs error (device sde): __ext4_find_entry:1682: inode #2: comm rsync: reading directory lblock 0
Dec 29 00:40:02 360g9 kernel: [8786958.372347] EXT4-fs error (device sde): __ext4_find_entry:1682: inode #2: comm rsync: reading directory lblock 0
Dec 29 00:40:02 360g9 kernel: [8786958.372549] EXT4-fs error (device sde): __ext4_find_entry:1682: inode #2: comm rsync: reading directory lblock 0
Dec 29 00:40:02 360g9 kernel: [8786958.372752] EXT4-fs error (device sde): __ext4_find_entry:1682: inode #2: comm rsync: reading directory lblock 0
Dec 29 00:40:07 360g9 kernel: [8786962.948793] EXT4-fs error (device sde): __ext4_find_entry:1682: inode #144179202: comm rsync: reading directory lblock 0
Dec 29 00:40:07 360g9 kernel: [8786962.949896] EXT4-fs error (device sde): __ext4_find_entry:1682: inode #144179202: comm rsync: reading directory lblock 0
Dec 29 00:40:07 360g9 kernel: [8786962.950431] EXT4-fs error (device sde): __ext4_find_entry:1682: inode #144179202: comm rsync: reading directory lblock 0
Dec 29 00:40:07 360g9 kernel: [8786962.950727] EXT4-fs error (device sde): __ext4_find_entry:1682: inode #144179202: comm rsync: reading directory lblock 0
Dec 29 00:40:07 360g9 kernel: [8786962.951034] EXT4-fs error (device sde): __ext4_find_entry:1682: inode #144179202: comm rsync: reading directory lblock 0
Dec 29 00:40:07 360g9 kernel: [8786962.951290] EXT4-fs error (device sde): __ext4_find_entry:1682: inode #144179202: comm rsync: reading directory lblock 0

Bash: 
root@360g9:~# lvs -a
  LV              VG               Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  data            pve              twi-aotz-- 702.98g             0.00   0.26                          
  [data_tdata]    pve              Twi-ao---- 702.98g                                                  
  [data_tmeta]    pve              ewi-ao----  <7.18g                                                  
  [lvol0_pmspare] pve              ewi-------  <7.18g                                                  
  root            pve              -wi-ao----  96.00g                                                  
  swap            pve              -wi-ao----   8.00g                                                  
  vm-100-disk-0   pve1000-980Pro-1 -wi-a-----  20.00g                                                  
  vm-121-disk-0   pve1000-980Pro-1 -wi-ao----  20.00g                                                  
  vm-122-disk-0   pve1000-980Pro-1 -wi-ao----  20.00g                                                  
  vm-101-disk-0   pve2000-p31-1    -wi-ao----  20.00g                                                  
  vm-102-disk-0   pve2000-p31-1    -wi-ao---- 330.00g                                                  
  vm-103-disk-0   pve2000-p31-1    -wi-ao----  20.00g                                                  
  vm-104-disk-0   pve2000-p31-1    -wi-ao---- 400.00g                                                  
  vm-105-disk-0   pve2000-p31-1    -wi-ao----  20.00g                                                  
  vm-106-disk-0   pve2000-p31-1    -wi-a-----  20.00g                                                  
  vm-107-disk-0   pve2000-p31-1    -wi-ao----  20.00g                                                  
  vm-108-disk-0   pve2000-p31-1    -wi-ao----  20.00g                                                  
  vm-110-disk-0   pve2000-p31-1    -wi-a-----  20.00g                                                  
  vm-111-disk-0   pve2000-p31-1    -wi-a-----  20.00g                                                  
  vm-112-disk-0   pve2000-p31-1    -wi-ao----  20.00g                                                  
  vm-113-disk-0   pve500           -wi-a-----  20.00g                                                  
  vm-114-disk-0   pve500           -wi-ao----  20.00g                                                  
  vm-115-disk-0   pve500           -wi-ao----  20.00g                                                  
  vm-116-disk-0   pve500           -wi-ao----  20.00g                                                  
  vm-118-disk-0   pve500           -wi-ao----  20.00g                                                  
  vm-119-disk-0   pve500           -wi-ao----  20.00g                                                  
  vm-120-disk-0   pve500           -wi-ao----  20.00g                                                  
  vm-123-disk-0   pve500           -wi------- 300.00g                                                  
  vm-109-disk-0   pve900-2         -wi-ao----  20.00g                                                  
  vm-123-disk-0   pve900-2         -wi-ao---- 818.00g                                                  
  vm-124-disk-0   pve900-3         -wi-ao----  20.00g                                                  
  vm-125-disk-0   pve900-3         -wi-a-----  20.00g                                                  
root@360g9:~# vgs -a
  VG               #PV #LV #SN Attr   VSize    VFree  
  pve                1   3   0 wz--n- <837.33g  <16.00g
  pve1000-980Pro-1   1   3   0 wz--n-  931.51g  871.51g
  pve2000-p31-1      1  11   0 wz--n-   <1.82t <953.02g
  pve500             1   8   0 wz--n- <465.73g  <25.73g
  pve900-2           1   2   0 wz--n- <838.33g  336.00m
  pve900-3           1   2   0 wz--n- <838.33g <798.33g
root@360g9:~# lsblk
NAME                                  MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                                     8:0    0 838.3G  0 disk
├─sda1                                  8:1    0  1007K  0 part
├─sda2                                  8:2    0     1G  0 part
└─sda3                                  8:3    0 837.3G  0 part
  ├─pve-swap                          253:9    0     8G  0 lvm  [SWAP]
  ├─pve-root                          253:10   0    96G  0 lvm  /
  ├─pve-data_tmeta                    253:11   0   7.2G  0 lvm
  │ └─pve-data-tpool                  253:14   0   703G  0 lvm
  │   └─pve-data                      253:15   0   703G  1 lvm
  └─pve-data_tdata                    253:12   0   703G  0 lvm
    └─pve-data-tpool                  253:14   0   703G  0 lvm
      └─pve-data                      253:15   0   703G  1 lvm
sdb                                     8:16   0 465.7G  0 disk
├─pve500-vm--116--disk--0             253:0    0    20G  0 lvm
├─pve500-vm--118--disk--0             253:17   0    20G  0 lvm
├─pve500-vm--113--disk--0             253:19   0    20G  0 lvm
├─pve500-vm--115--disk--0             253:21   0    20G  0 lvm
├─pve500-vm--114--disk--0             253:23   0    20G  0 lvm
├─pve500-vm--120--disk--0             253:24   0    20G  0 lvm
└─pve500-vm--119--disk--0             253:25   0    20G  0 lvm
sdc                                     8:32   0 838.3G  0 disk
├─pve900--2-vm--109--disk--0          253:13   0    20G  0 lvm
└─pve900--2-vm--123--disk--0          253:29   0   818G  0 lvm
sdd                                     8:48   0 838.3G  0 disk
├─pve900--3-vm--124--disk--0          253:30   0    20G  0 lvm
└─pve900--3-vm--125--disk--0          253:31   0    20G  0 lvm
nvme1n1                               259:0    0 931.5G  0 disk
├─pve1000--980Pro--1-vm--100--disk--0 253:22   0    20G  0 lvm
├─pve1000--980Pro--1-vm--121--disk--0 253:27   0    20G  0 lvm
└─pve1000--980Pro--1-vm--122--disk--0 253:28   0    20G  0 lvm
nvme0n1                               259:1    0   1.8T  0 disk
├─pve2000--p31--1-vm--102--disk--0    253:1    0   330G  0 lvm
├─pve2000--p31--1-vm--103--disk--0    253:2    0    20G  0 lvm
├─pve2000--p31--1-vm--104--disk--0    253:3    0   400G  0 lvm
├─pve2000--p31--1-vm--107--disk--0    253:4    0    20G  0 lvm
├─pve2000--p31--1-vm--106--disk--0    253:5    0    20G  0 lvm
├─pve2000--p31--1-vm--108--disk--0    253:6    0    20G  0 lvm
├─pve2000--p31--1-vm--105--disk--0    253:7    0    20G  0 lvm
├─pve2000--p31--1-vm--101--disk--0    253:8    0    20G  0 lvm
├─pve2000--p31--1-vm--110--disk--0    253:16   0    20G  0 lvm
├─pve2000--p31--1-vm--111--disk--0    253:18   0    20G  0 lvm
└─pve2000--p31--1-vm--112--disk--0    253:20   0    20G  0 lvm


Code: 
Filesystem           Inodes IUsed IFree IUse% Mounted on
udev                    32M   743   32M    1% /dev
tmpfs                   32M  1.6K   32M    1% /run
/dev/mapper/pve-root   6.0M  102K  6.0M    2% /
tmpfs                   32M   106   32M    1% /dev/shm
tmpfs                   32M    46   32M    1% /run/lock
/dev/fuse              256K    60  256K    1% /etc/pve

/dev/sde               233M  2.4M  231M    2% /mnt/tosh-8tb-dl360

tmpfs                  6.3M    18  6.3M    1% /run/user/0

Code: 
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=131955276k,nr_inodes=32988819,mode=755,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=26398024k,mode=755,inode64)
/dev/mapper/pve-root on / type ext4 (rw,relatime,errors=remount-ro,stripe=64)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,inode64)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k,inode64)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=25655)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
lxcfs on /var/lib/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
/dev/fuse on /etc/pve type fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other)

/dev/sde on /mnt/tosh-8tb-dl360 type ext4 (ro,relatime)

tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=26398020k,nr_inodes=6599505,mode=700,inode64)
tracefs on /sys/kernel/debug/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime)



I can't umount it and re-mount it...I don't want to restart PVE 7.4-19...

Please help.
 
Last edited:
Assuming your usb disk is a toshiba 8TB hdd ..., it shows I/O error which could be in first a filesystem error or a disk error but as it isn't shown anymore in lsblk so the disk is dead. What you can do is "umount -l /mnt/tosh-8tb-dl360" which should remove the disk from pve, see df.
Then you can unplug the usb drive and you need a new one to restore from backup what was onto. No need to reboot the pve host.
 
Last edited:
  • Like
Reactions: Kingneutron
Wow, thanks alot! The -l option saved me.

Code: 
root@360g9:/mnt/tosh-8tb-dl360# umount -l /mnt/tosh-8tb-dl360 
root@360g9:/mnt/tosh-8tb-dl360# ls -l
ls: reading directory '.': Input/output error
total 0
root@360g9:/mnt/tosh-8tb-dl360# cd ..
root@360g9:/mnt# umount -l /mnt/tosh-8tb-dl360 
umount: /mnt/tosh-8tb-dl360: not mounted.
root@360g9:/mnt# mount /dev/sdf /mnt/tosh-8tb-dl360/
root@360g9:/mnt# ls -l tosh-8tb-dl360/
total 464

I really don't know... the drive or the external case will die after a few days of backup jobs.... the drive seems to be back on /dev/sdf after I unplugged it and plugged it into another port.

Thanks again!
 
I want to write a script to check the validity of the FS and the drive is properly connected.

What approach should I use? Simply ls and lsblk?
 
It appears that the drive is using ext4, so to check the FS you should use:
Code: 
fsck.ext4 /dev/sdXY

# replacing XY with the correct drive/partition
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom