HDD never spin down

Okay I am a complete idiot - I myself called smartctl all the time when checking for the power state of the drives.... Even though I did it with -n standby. blktrace is now more informative. Empty for ~half a minute, then tons of this (notice, still some smartctl in there):

Code: 
 8,0   14      106     1.375297503 529144  A   R 2576 + 16 <- (8,1) 528
  8,0   14      107     1.375298570 529144  Q   R 2576 + 16 [vdev_open]
  8,0   14      108     1.375303895 529144  G   R 2576 + 16 [vdev_open]
  8,0   14      109     1.375304606 529144  P   N [vdev_open]
  8,0   14      110     1.375305410 529144  U   N [vdev_open] 1
  8,0   14      111     1.375306201 529144  I   R 2576 + 16 [vdev_open]
  8,0   14      112     1.375314385 529144  D   R 2576 + 16 [vdev_open]
  8,0   14      113     1.375329125 529144  A   R 23437751312 + 16 <- (8,1) 23437749264
  8,0   14      114     1.375329489 529144  Q   R 23437751312 + 16 [vdev_open]
  8,0   14      115     1.375330761 529144  G   R 23437751312 + 16 [vdev_open]
  8,0   14      116     1.375331135 529144  P   N [vdev_open]
  8,0   14      117     1.375331435 529144  U   N [vdev_open] 1
  8,0   14      118     1.375331751 529144  I   R 23437751312 + 16 [vdev_open]
  8,0   14      119     1.375334091 529144  D   R 23437751312 + 16 [vdev_open]
  8,0   14      120     1.375348787 529144  A   R 23437751824 + 16 <- (8,1) 23437749776
  8,0   14      121     1.375349117 529144  Q   R 23437751824 + 16 [vdev_open]
  8,0   14      122     1.375350479 529144  G   R 23437751824 + 16 [vdev_open]
  8,0   14      123     1.375350694 529144  P   N [vdev_open]
  8,0   14      124     1.375350966 529144  U   N [vdev_open] 1
  8,0   14      125     1.375351200 529144  I   R 23437751824 + 16 [vdev_open]
  8,0   14      126     1.375353402 529144  D   R 23437751824 + 16 [vdev_open]
  8,0   17      483     1.356961752     0  C   R [0]
  8,0   17      484     1.369894306     0  C   R [0]
  8,0   17      485     1.382719285     0  C   R [0]
  8,0   17      486     1.383300382     0  C   R 2576 + 16 [0]
  8,0    5       43     1.382827948 529109  D   R 64 [smartctl]
  8,0    5       44     1.417369709 529109  D   R 18 [smartctl]
  8,0    8      520     1.383329614  2330  A   W 2576 + 16 <- (8,1) 528
  8,0    8      521     1.383330184  2330  Q   W 2576 + 16 [z_null_iss]
  8,0    8      522     1.383332873  2330  G   W 2576 + 16 [z_null_iss]

Any way to check the powerstate now (apart from checking physically on the drive LEDs)?

For the record, I found out by doing this:
ps -axjf | grep -i -B 20 smartctl
This includes enough lines to see what initially spawned smartctl.
 
Last edited:
I found that global_filter-ing the devices in
Code: 
/etc/lvm/lvm.conf
also prevented the devices (which are OSD's in my case) from coming back up and in when the system reboots. I found simply using
Code: 
hdparm -S 242 /dev/sdX
was sufficient to get my HDDs to spindown after a minute of inactivity, and found this persisted across reboots.

TL;DR:
Code: 
hdparm -S 242 /dev/sdX
to get an HDD at /dev/sdX to spindown after 1 hour.
 
Last edited:
wassupluke said:
I found that global_filter-ing the devices in
Code: 
/etc/lvm/lvm.conf
also prevented the devices (which are OSD's in my case) from coming back up and in when the system reboots. I found simply using
Code: 
hdparm -S 242 /dev/sdX
was sufficient to get my HDDs to spindown after a minute of inactivity, and found this persisted across reboots.

TL;DR:
Code: 
hdparm -S 242 /dev/sdX
to get an HDD at /dev/sdX to spindown after 1 hour.
Click to expand...
i.e. Is this all you had to do to get them to spin down in proxmox?
 
Giggling3999 said:
i.e. Is this all you had to do to get them to spin down in proxmox?
Click to expand...
That's all I did and it seemed to work. I referenced this great documentation for using
Code: 
hdparm -S
. I had them spin down after 60 seconds but then they'd spin down when I was part way through navigating the cephfs I have as my backup storage and it made it take longer to navigate around waiting for them to spin back up, so I extended the up time to an hour and that seems to be fine so far.
 
Spinning down wasn't really the problem I had - It was them constantly spinning back up with proxmox

I think I had to disable proxmox monitoring + SMART
 
Giggling3999 said:
Can you post your filter list, please?

I am going to try this again, tonight....I have to use hd-idle as that's the only method to spin down my disks
Click to expand...
In that comment you referenced I was saying I removed the /dev/sd* devices from the global flags because if I had them filtered out they wouldn't start when the system rebooted.
Code: 
global_filter=["r|/dev/zd.*|","r|/dev/rbd.*|"]
 
So....

I am still unable to get my ZFS from being written to - Regardless of shutting down all containers/vm. I disabled pvestatd and smartmontools.

When I had spin down working in a VM - I was able to see that zfs get -H -o value written would consistently give me 0 bytes....In fact I have modified auto-snap to not snap if it is zero. I was finding in the VM that snaps (even though I have special device) would wake the HDD.

For now I've disabled auto-snaps - As I can see a consistent amount of kb is being written to each zfs

Are there any other process in proxmox that could be writing to these zfs directories ? I'm having a hard time tracking down a process
 
OK so for anyone following this in the future; snaps DO wake the HDD...This I was already aware. If anyone is interested, I modified zfs-auto-snapshot script a long time back, to only snap if changes were present (To be honest, this is quite nice for tidiness, anyway)

Code: 
do_snapshots () # properties, flags, snapname, oldglob, [targets...]
{
    local PROPS="$1"
    local FLAGS="$2"
    local NAME="$3"
    local GLOB="$4"
    local TARGETS="$5"
    local KEEP=''
    local RUNSNAP=1
    local WRITTEN

    # global DESTRUCTION_COUNT
    # global SNAPSHOT_COUNT
    # global WARNING_COUNT
    # global SNAPSHOTS_OLD

    for ii in $TARGETS
    do
        ##THIS IS MY MODIFICATION
        #CHECK IF BEEN WRITTEN AND ONLY PROCEED IF YES)
        WRITTEN=$(zfs get -H -o value written "${ii}")
        if [ "${WRITTEN}" = "0" ]
        then
            if echo "$NAME" | grep -q "frequent"
            then
                echo "${ii} has no changes, ignoring (frequent)."
                continue
            fi
        fi


It seems though, that the problem of random writing to the ZFS, was due to me sharing .raw files between containers. I have just changed my whole setup and happened to stumble upon that this was a bad idea. I set up again with directory sharing, rebooted...And bam.....Seems to no longer be writing - So I got lucky.

The other steps are outlined in this post -

Set atime off for ZFS
disable/add exception smartmon
disable/add exception pvestatd

Cheers!
 
You must log in or register to reply here.
Top Bottom