Search results

  1. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    By upcoming I meant just the next version. As soon as the patch is merged to master I was planning to compile ZFS from source and make upgrade as right now I'm still unable to do any real backups and I need to resolve this ASAP. Is there any danger in doing that or what would be your...
  2. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    Actual problem seems to be described here: https://github.com/zfsonlinux/zfs/issues/6171 @manu It seems as it could be fixed in upcoming 0.7.6 (maybe) so I would ask Proxmox staff to upgrade to this version ASAP because current version of ZFS is definitely not production ready.
  3. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    Ok back to some another tests Command: rsync -r --progress /DP1/backups/ /DT3btrfs/backups/ Will cause system to start hanging (as usually) within 10 seconds after launch, I have to kill it pretty quickly. sdd & sde are main data SSD (where data are copied from) sdf & sdg are system SSD sda...
  4. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    @czechsys @manu I do not know why I didn't though about it before but I had idea to actually test speed from inside container VM/CT and what ever test I do from inside of container they are exactly what I would assume speed should be on this system: # dd if=/dev/zero of=/tmp/test bs=1G count=10...
  5. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    FIO on main storage SSD: # full write pass - there is definitely problem - server slowing down fio --name=writefile --size=10G --filesize=10G --filename=/DP1/test --bs=1M --nrfiles=1 --sync=0 --randrepeat=0 --rw=write --refill_buffers --end_fsync=1 --iodepth=200 --ioengine=libaio writefile...
  6. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    Even setting backup/copy to lowest possible priority and limiting speed does not help: nice -n 19 ionice -c2 -n7 rsync --bwlimit=51200 -r --progress /DP1/backups/ /DT3btrfs/backups/ Setting: vm.dirty_ratio = 5 vm.dirty_background_ ratio = 1 Seems to help slightly but it may as well be random...
  7. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    I thing that I'm experiencing same unresolved problem: https://forum.proxmox.com/threads/kvm-guests-freeze-hung-tasks-during-backup-restore-migrate.34362/
  8. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    After many more tests and many more tuned values I'd given up on idea that It is FreeNAS problem completely, nothing I did on FreeNAS side changed the behavior. Today I added two 500 GB spinning HDD to the Proxmox server and put them to RAID 1 (ZFS). And surprise surprise problem is still there...
  9. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    Thank you for tips, yes I read large portion of FreeNAS documentation and I have multiple FreeNAS boxes, it's true that only this one does not have ZIL on SSD, but I already tested transfers with disabled ZIL on given dataset to rule it out, and I also had NFS set to async. And still why It...
  10. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    I never thought about testing SSD because when I initially build the server and tested them they worked great: dd if=/dev/zero of=/DP1/dump/test bs=1G count=2 2+0 records in 2+0 records out 2147483648 bytes (2.1 GB, 2.0 GiB) copied, 1.26704 s, 1.7 GB/s But as there is larger amount of data: dd...
  11. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    I'm running 6 x CT and 2 x VM. What would be best way to do benchmark so It would give us proper/all informations? Just try to write bunch of data with dd? Normally Server Load is around 1,5-2 so I didn't yet consider that there would be problem. Thank you!
  12. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    This is what happens during last two test transfers, system hanged so that graphs are not showing at all. The one that I mentioned in last post was around 13:00 and spike there is clearly visible, but I manage to kill it before it hang system as well.
  13. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    I did: echo 1073741824 >> /sys/module/zfs/parameters/zfs_arc_min And It seems that dips are little lower (during dips instead of 2-10MB/s transfer speed I now get 25MB/s) system still feels very sluggish after few minutes of transfer. Only thing that I noticed that wasn't there before (because...
  14. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    When I do ps faxl | grep " D " Result is: 1 0 6613 2 20 0 0 0 io_sch D ? 0:02 \_ [txg_sync] 0 0 27891 6353 20 0 12788 936 pipe_w S+ pts/6 0:00 \_ grep D But not always, txg_sync could be related to...
  15. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    I do not have SLOG but I have 6x 4 TB disk in RAID 10 (3 groups with 2 discs) And according to usage graphs in Freenas they do not have problem to sustain the load, load drops down immediately as transfer starts loosing speed. The disc load doesn't go to more than 70 % and neither discs...
  16. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    I will test that (ps faxl). Yes I have lot of hang messages, sorry about not mentioned it earlier I totally forgot that I have that screenshot: It reports folowing hang tasks (more than 120s): kthreadd khungepaged kswapd z_wr_iss txg_sync ksmtuned usbhid-ups cfs-loop (it seems like all over the...
  17. J

    [KNOWN ZFS PROBLEM] Freezing while high IO (only on host - not guest)

    EDIT: As I find out that high IO will only result in freezing while performed on host, not guests please skip to post #19 as there are new findings. I'm basically reposting issue that I have with backups to FreeNAS from this post...
  18. J

    Moving lxc image

    Now I tried: pct clone 102 101 -experimental true -storage vmContainer (101 is not used) and got error: 400 Parameter verification failed. full: missing property - 'storage' requiers this property What am I missing now?
  19. J

    Moving lxc image

    So simply coping container to different location and changing config file of CT works - to certain degree. Newly copied container will not aper in GUI, and in general Proxmox doesn't seam to know about this movement at all. There is command "pct clone" but I can't find documentation for what...
  20. J

    Moving lxc image

    To simplify, I need to move container from zfs /DP1 to /DP1/vmContainer...

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!