nfs performance

Discussion in 'Proxmox VE: Installation and configuration' started by ChappIO, Jun 8, 2018.

  1. ChappIO

    ChappIO New Member

    Joined:
    Apr 1, 2018
    Messages:
    8
    Likes Received:
    0
    I have been testing different ways to set up my NAS storage for proxmox but I am finding odd results. Maybe you can help me clarify them.

    Testing on local ssd I am finding ~500MB/s writes
    Testing through iSCSI target ~100MB/s writes
    Testing through NFS storage ~1.8GB/s writes

    I find it weird to see the high speeds on NFS storage since I just have a bonded gigabit network. Is there a lot of caching happening or someting?

    Also, do you have some tips? I have many compute heavy vms which don't need long-lasting storage and a few vms which require a lot of storage (some for slow storage, some for databases).
     
  2. mailinglists

    mailinglists Active Member

    Joined:
    Mar 14, 2012
    Messages:
    387
    Likes Received:
    34
    Probably a lot of caching.
    Probably bad testing methods.

    How did you test? You provide too little detail about testing and the environment for anyone to meaningfully comment on your results.
     
  3. ChappIO

    ChappIO New Member

    Joined:
    Apr 1, 2018
    Messages:
    8
    Likes Received:
    0
    I just dd'd /dev/zero to disk in a vm. Probably a bad way to test this. Could you point me in a better direction for testing?

    I am just trying to figure out what the pros and cons are of the various ways of storing disk images for my vms.

    My setup:

    2 compute nodes with limited local storage
    1 NAS
    1 network node which does ssl termination

    I run mostly stateless compute services in the vms but also a handful of database/storage vms. I want to move (or at least backup) the storage of those vms to the NAS so that I have a central place which contains all important data.

    I have never used iscsi before but I see some people recommending this so I figured I'd try it out. But would there be any benefits?
     
  4. r.jochum

    r.jochum Member

    Joined:
    Mar 26, 2018
    Messages:
    88
    Likes Received:
    13
    Bonnie++ is nice to do real tests on NFS.
     
    fireon likes this.
  5. ChappIO

    ChappIO New Member

    Joined:
    Apr 1, 2018
    Messages:
    8
    Likes Received:
    0
    Thanks for the tip!
    I ran bonnie++ on every configuration (not at the same time of course).
    As far as I can see, nfs provides higher performance all round.

    Did I draw the wrong conclusion? Is there a benefit of using iSCSI instead of NFS?

    2018-06-08-192631_1684x1351_scrot(1).jpg
     
  6. David Pilibosian

    David Pilibosian New Member

    Joined:
    Apr 11, 2017
    Messages:
    5
    Likes Received:
    0
    What kind of NAS are you testing to? If it's a FreeNAS and I recall correctly iSCSI does sync writes by default and NFS doesn't. Without a ZIL SSD you're going to see a pretty massive performance difference.
     
  7. ChappIO

    ChappIO New Member

    Joined:
    Apr 1, 2018
    Messages:
    8
    Likes Received:
    0
    This is a Synology Rackstation RS818+ with mostly default settings. It contains 3 WD Red (4TB) drives using their SHR configuration.
     
  8. mailinglists

    mailinglists Active Member

    Joined:
    Mar 14, 2012
    Messages:
    387
    Likes Received:
    34
    I would go with NFS, due to easier debugging and maintenance.
    However if your data is important, make sure NFS is not mounted / exported with async.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice