vms lags during vm cloning

C

chalan

Member
Mar 16, 2015
119
4
16
during vm migration all vms totaly lagging... ssh very slow, some of the vms didnt works well... cpu usage during clone show about 10percent usage, but IO delay 28percent... is it normal on a raid 10 zfs proxmox Virtual Environment 5.0-30 with 16GB RAM????

i'v had proxmox v3 with raid 1 also 16gb ram also lagging problem so i buy new server with 4x 1TBs disk but it seems to be the same, its very disaapointig...
 
Last edited:
i dont understand this formula

So the formula is: total_ram - 1 GB - expected_GB_for_vm/ct = zfs_arc_max; zfs_arc_max >= 4 GB.

i have 16GB so 16GB - 1GB - 8GB = 7GB so how to set the zfs arc?
 
ok i did:

Code: 
root@pve-klenova:~# cat /sys/module/zfs/parameters/zfs_arc_max
7516192768
root@pve-klenova:~# cat /sys/module/zfs/parameters/zfs_arc_min
4294967296
root@pve-klenova:~# free -h
              total        used        free      shared  buff/cache   available
Mem:            15G        3,9G         11G         51M        123M         11G
Swap:          8,0G          0B        8,0G

will see if something changed
 
well i dont know but it seems totaly bad


Code: 
root@pve-klenova:~# pveperf
CPU BOGOMIPS:      38401.52
REGEX/SECOND:      450140
HD SIZE:           745.21 GB (rpool/ROOT/pve-1)
FSYNCS/SECOND:     58.78
DNS EXT:           29.48 ms
DNS INT:           18.56 ms (elson.sk)
 
If you want additional ZFS tips & tweaks, gkovacs has an excellent write-up on his ZFS settings here:
https://forum.proxmox.com/threads/single-server-storage-recommendations.35148/#post-172410
Otherwise if everything's running fine, I'd leave well enough alone. And if you do change anything, notate everything you change, and the previous settings/post settings of the config files. That way anything unexpected, you can always go back without reinstalling the whole server.
 
I don't know if syntax makes a difference, what happens to fsync rate if you do:
#pveperf /rpool/
 
here the results, please see, its a shame i think:

Code: 
root@pve-klenova:~# zpool status
  pool: public
 state: ONLINE
  scan: scrub repaired 0 in 0h0m with 0 errors on Sun Oct  8 00:24:02 2017
config:

    NAME                        STATE     READ WRITE CKSUM
    public                      ONLINE       0     0     0
      ata-MB0500EBZQA_Z1M0EHYH  ONLINE       0     0     0
      ata-MB0500EBZQA_Z1M0EGEJ  ONLINE       0     0     0

errors: No known data errors

  pool: rpool
 state: ONLINE
  scan: scrub repaired 0 in 6h18m with 0 errors on Sun Oct  8 06:42:22 2017
config:

    NAME                                                STATE     READ WRITE CKSUM
    rpool                                               ONLINE       0     0     0
      mirror-0                                          ONLINE       0     0     0
        ata-WDC_WD10EFRX-68PJCN0_WD-WCC4J2021886-part2  ONLINE       0     0     0
        ata-WDC_WD10EFRX-68JCSN0_WD-WMC1U6546808-part2  ONLINE       0     0     0
      mirror-1                                          ONLINE       0     0     0
        ata-WDC_WD10EFRX-68FYTN0_WD-WCC4J2AK75T9        ONLINE       0     0     0
        ata-WDC_WD10EFRX-68FYTN0_WD-WCC4J1JE0SFR        ONLINE       0     0     0

errors: No known data errors
root@pve-klenova:~# zfs list
NAME                       USED  AVAIL  REFER  MOUNTPOINT
public                     312K   899G    96K  /public
rpool                     1,13T   645G    96K  /rpool
rpool/ROOT                99,9G   645G    96K  /rpool/ROOT
rpool/ROOT/pve-1          99,9G   645G  99,9G  /
rpool/data                1,02T   645G    96K  /rpool/data
rpool/data/vm-200-disk-1  1003G   645G  1003G  -
rpool/data/vm-201-disk-1  2,32G   645G  2,32G  -
rpool/data/vm-201-disk-2   856M   645G   856M  -
rpool/data/vm-211-disk-1  37,8G   645G  37,8G  -
rpool/swap                8,50G   647G  6,83G  -
root@pve-klenova:~# pveperf /rpool
CPU BOGOMIPS:      38401.52
REGEX/SECOND:      455315
HD SIZE:           745.02 GB (rpool/ROOT/pve-1)
FSYNCS/SECOND:     67.95
DNS EXT:           72.10 ms
DNS INT:           21.66 ms (elson.sk)
root@pve-klenova:~# pveperf /public
CPU BOGOMIPS:      38401.52
REGEX/SECOND:      457415
HD SIZE:           899.00 GB (public)
FSYNCS/SECOND:     139.99
DNS EXT:           106.16 ms
DNS INT:           27.25 ms (elson.sk)
 
fsync for pool /public/ looks about right. fsync for pool /rpool/ looks terrible. REGEX/SECOND for both seem lower than normal.
Can you check ashift settings for pool /rpool/ and also pool /public/
Check ashift settings:

#zdb -C rpool | grep ashift
and
#zdb -C public | grep ashift
The recommended setting is "12" IIRC
 
hello

Code: 
root@pve-klenova:~# zdb -C rpool | grep ashift
                ashift: 12
                ashift: 12
root@pve-klenova:~# zdb -C public | grep ashift
                ashift: 12
                ashift: 12
 
Is the /public/ ZFS pool local storage? What happens if you temporarily remove (offline) /public/ ZFS pool, pull the drives out, reboot, and then re-run pveperf with just the 4 disks in raid10? Do you remember running pveperf before adding /public/?, and if it was better?
 
/public is local storage... 2x sata2 500GB drives... i dont remeber how it was before i add /public, but i dont think it was better...
 
i have removed the two sata2 500GB drives, so only the 4x 1TB WD RED remain and pveperf was absolutly the same...
 
@chalan If this is a test server, can you backup your VMs, and then setup WD RED drives on a different system? Perhaps your server's controller is not optimized with standard SATA drives. Perhaps it's tuned for HW RAID/SAS combo. Only way to find out is put these drives on different, non-supermicro motherboard system. Or if possible, install a 4 Port SATA III PCIe 2.0 controller into your existing server, and run WD RED off that.
 
@chalan here are my numbers for one of my HP test servers. This is a fresh setup 1 drive for boot, 4 drives for rpool, and no other SSD for ZFS cache:

XeonX3470 (5204 CPU passmarks)/16GB DDR3/HPZ200 motherboard
2 WDRE4 7200rpm (new 3.5") + 2 Seagate 7200RPM (old 2.5")

rpool ashift=12 zfs sync=standard

CPU BOGOMIPS: 23465.12
REGEX/SECOND: 1854135
HD SIZE: 449.50 GB (rpool)
FSYNCS/SECOND: 119.16
DNS EXT: 61.26 ms
DNS INT: 37.43 ms (lan)

rpool ashift=12 zfs sync=disabled

CPU BOGOMIPS: 23465.12
REGEX/SECOND: 1807325
HD SIZE: 449.50 GB (rpool)
FSYNCS/SECOND: 22731.42
DNS EXT: 47.67 ms
DNS INT: 41.92 ms (lan)
 
GadgetPig said:
@chalan If this is a test server, can you backup your VMs, and then setup WD RED drives on a different system? Perhaps your server's controller is not optimized with standard SATA drives. Perhaps it's tuned for HW RAID/SAS combo. Only way to find out is put these drives on different, non-supermicro motherboard system. Or if possible, install a 4 Port SATA III PCIe 2.0 controller into your existing server, and run WD RED off that.
Click to expand...

cau you please advise me a good 4port SATA3 PCie 2.0 controller? or post link please, thank you...
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back