HDD never spin down

C

charlot rodolphe

New Member
Oct 31, 2018
1
0
1
31
Hi,

I have a Proxmox server (5.2.1) with the config below :
- Xeon E5 2650 V2
- 16Gb Ram ECC
- 120 GB SSD (KINGSTON SV300S3)
- 2x2To HDD (Hitachi HUS72403 and TOSHIBA DT01ACA3)
The proxmox server is intalled on the SSD, the containers too.

My two hard-drives are configured in RAID 1 with this thread :
doc.ubuntu-fr.org/raid_logiciel

It's mounted on my proxmox server, and shared through NFS with my 4 containers.

Everything is working smoothly but hard drives never spin down.
When i try a : hdparm -y /dev/sd[bc]
It lasts for a few seconds then hard drives spin again

I tried to stop the NFS server and stop all the containers but there are always 4096 bytes read or write every 1 or 2 seconds and I don't know where i comes from.

They are driving me crazy because I sleep next to it :)
Any idea that could help me understand where I did something wrong ?

PS : Sorry for my english

Attachments :
Result of
- dstat -D sdb,sdc
- mdadm --detail /dev/md0
- iostat -d 10 /dev/x
- iotop
capture d'écran.png capture d'écran.png capture d'écran_1.png capture d'écran_2.png capture d'écran_3.png
 
Hello,

is there a known solution for hdd sleep now?
I want to use proxmox (install it on a ssd) and create a zfs pool (with 4 disks) and share it via SMB to my Windwos systems.
The ZFS Pool is used maybe 4 or 5 times a week, so it is necessary for me to send my disks from the ZFS pool to sleep when they are idle.

as far as i know the pvestatd keeps the disks awake because it scans them every few seconds.

I really want to use Proxmox but as long as there is no real solution for this its a pain to use it.
Proxmox needs a function to exclude disks from pvestatd.

Or is there any solution to get disk sleep working?
 
the filesystem is ext4 ?

you can this to see what access the disk :

Hi

Another option is to enable hard drive access dumping to syslog.

That may tell you what is accessing the particular drive. I used it myself may moons ago and found it useful.

To turn it on...

Code:
echo 1 | sudo tee /proc/sys/vm/block_dump
Click to expand...

and to turn it off

Code:
echo 0 | sudo tee /proc/sys/vm/block_dump
Click to expand...

You can then check syslog to try to determine what is writing to the drive.


BECAREFUL --> the log file grows very fast.


If you have ext4 filesystem, the system chek the health and wake up the disk.

No problem with ext2
 
Hello Dark26,

Your answer is kinda "off topic".
I never talked about ext, I was pretty clear that its a ZFS volume. And there is no reason to watch at logfiles because I already have identified pvestad as the problem.

I would like to hear a answer from a staff member who knows what im talkin about.

Thanks
 
Hello, I found a way to keep unused disk standby.
I see all hdd become active when I use lvm commands (like lvdisplay), and pvestatd will call lvm functions to get the status of default local-lvm storage.
so I edit the /etc/lvm/lvm.conf, append a new filter rule in
devices{
global_filter []​
}​

now it is
global_filter = [ "r|/dev/zd.*|", "r|/dev/mapper/pve-.*|", "r|/dev/sdb.*|" ]​

and now pvestatd will not awake my /dev/sdb
 
Last edited:
  • Like
Reactions: wbk and AlexanderT
Hello Tinker Xie,

thanks a lot. Can you please explain the functions you are using?
i assume that I have to add "r|/dev/sdc.*|","r|/dev/sdd.*|","r|/dev/sde.*|" when i have 3 other disks i want to keep in standby mode?

Greetings

To the Proxmox Staff Members:
Are you planning to integrate a function in version 6.0 to prevent discs from waking up? Like exclude disk from beeing read by pvestatd or exclude them from lvm commands?
I know that only a few users would use this, but as I found on the forums and other tech sites it is something that people would use.
It would be greate if you could have a look on it, I mean in times like these its worth to have a look at our environment and with this simple funktion you could contribute on energy saving - in my case the discs are spinning 24/7 but I use them only 4 - 5 times a week. YOU could help people to save money on energy and cooling just by adding some lines of code.
 
  • Like
Reactions: sanchO, isasmendi, herm and 3 others
martin_bk said:
To the Proxmox Staff Members:
Are you planning to integrate a function in version 6.0 to prevent discs from waking up? Like exclude disk from beeing read by pvestatd or exclude them from lvm commands?
I know that only a few users would use this, but as I found on the forums and other tech sites it is something that people would use.
It would be greate if you could have a look on it, I mean in times like these its worth to have a look at our environment and with this simple funktion you could contribute on energy saving - in my case the discs are spinning 24/7 but I use them only 4 - 5 times a week. YOU could help people to save money on energy and cooling just by adding some lines of code.
Click to expand...

Yep, that would be great, in a period when we should try to be "as green as possible".
This functionality, used by thousands of proxmox setups, would mean that GigaWatt's of electricity would be saved per year.
 
  • Like
Reactions: sanchO
I'm also interested in disk spin down, I'm using this functions since years now on my old NAS, and I'm struggling now with setting it up in proxmox.
 
Same.
Using global_filter is not a solution, sometimes we need this LVM outside of proxmox, so we can't just hide them from lvm configuration.

We must have a way to ask pve to do not vgscan everything
 
I also find this particularly useful to save energy.
 
Last edited:
Hello, I do not have time to write a nice post but here are my evernote notes:

spindown / sleep disks
you can edit /etc/lvm/lvm.conf to exclude your drives. Just reject them via global filter.
eg.: dont scan sda and sdb:
global_filter = [ "r|/dev/sda.*|", "r|/dev/sdb.*|" ,"r|/dev/zd.*|", "r|/dev/mapper/pve-.*|" "r|/dev/mapper/.*-(vm|base)--[0-9]+--disk--[0-9]+|"]
other: https://forum.proxmox.com/threads/pvestatd-doesnt-let-hdds-go-to-sleep.29727/
other: https://forum.proxmox.com/threads/upgrade-from-5-x-to-6-x-harddrives-dont-sleep-no-more.69672/
blktrace -d /dev/sdc -o - | blkparse -i - which showed immediately that lvs and vgs where responsible.
I excluded the drive in lvm.conf, and it is working now. Code:
sleep 300 && hdparm -C /dev/sdc
/dev/sdc:
drive state is: standby

Edited LVM.conf:

Filter by-id
global_filter = ["r|/dev/disk/by-id/ata-ST8000DM004-2CX188_ZCT0D032*|","r|/dev/sdd*|", "r|/dev/zd.*|", "r|/dev/mapper/pve-.*|" "r|/dev/mapper/.*-(vm|base)--[0-9]+--disk--[0-9]+|"]

[]add /mnt/pve/BackupsDS718 to sleep DS718 ?

To manualy spindown SDA:
hdparm -y /dev/sda
OKKKKK but not yet automatic (doesn't spindown after XX minutes).

Trying this: (not working)
Added at: /etc/hdparm.conf
/dev/sda {
spindown_time = 60
apm = 127
}

This Works: (For Sata 8TB datastore Windows) (disable scan in lvm.conf before !)
  • hdparm -S 12 /dev/sdc
    • /dev/sdc:
    • setting standby to 12 (1 minutes)
  • or
  • hdparm -S 240 /dev/sdc
    • /dev/sdc:
    • setting standby to 240 (1 minutes)
    • [] But does it stay after a reboot ???
  • If not, bulletproof version: create a bash file: /root/scripts/cron_hdparm.sh
#!/bin/sh #sleep [B]Sata 8TB Datatstore W2019[/B] echo \/-----------------------------------\/ date /usr/sbin/hdparm -C /dev/disk/by-id/ata-ST8000DM004-2CX188_ZCT0D032 /usr/sbin/hdparm -S 240 /dev/disk/by-id/ata-ST8000DM004-2CX188_ZCT0D032 #sleep 10 #hdparm -C /dev/disk/by-id/ata-ST8000DM004-2CX188_ZCT0D032
  • Then Cron it twice a day !
crontab -e #6h and 18h each day 0 6,18 * * * /root/scripts/cron_hdparm.sh >> /root/scripts/cron_hdparm.log 2>&1

infos config sleep hdparm:
==> spindown_time: see https://blog.bravi.org/?p=134
0 = disabled
1..240 = multiples of 5 seconds (5 seconds to 20 minutes)
241..251 = 1..11 x 30 mins 252 = 21 mins
253 = vendor defined (8..12 hours)
254 = reserved
255 = 21 mins + 15 secs

I will not make any response or support, take my note as they are, hope it helps, sorry, no time here.
If you try and it works and you have time, please give your experience below.

Save energy ! ;-)
 
Last edited:
  • Like
Reactions: wbk
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back