Search results

I added the following to my ceph.conf and restarted. debug lockdep = 0/0 debug context = 0/0 debug crush = 0/0 debug buffer = 0/0 debug timer = 0/0 debug journaler = 0/0 debug osd = 0/0 debug optracker = 0/0 debug...

Some interesting discussion here: large reads become 512 byte reads on qemu-kvm rbd http://lists.ceph.com/pipermail/ceph-users-ceph.com/2014-November/045033.html Ceph performance - 10 times slower http://lists.ceph.com/pipermail/ceph-users-ceph.com/2014-November/044796.html

Thats with SSD for main disk spirit? Will those tuning settings be appropriate for a spinner as well? Nodes are also proxmox servers - would those settings interfere with VM hosting?

No, I disabled the ZFS log. It was getting very little use, its possible it was actually slowing write performance. The ZFS cache does seem to be helping though, reads are actually faster than the disks transfer rate. Destructive tests unfortunately, so I can't really run it on the current...

Hi Spirit, thanks for the reply. I experimented with ceph on ext4/xfs a few weeks back and got similar results. I switched to gluster on zfs for the flexibility that zfs gives, amazing pool managemnent and a superior SSD log/cache. When I saw that ceph could be installed on it I gave it a...

A common post on the forums it seems, but my case is unique! :) probably not really ... 3 proxmox/ceph nodes, one just a nuc that is used for quorum purposes, no OSD's or VM's Underlying filesystem is ZFS, so using journal dio = 0 2 OSD's on two nodes - 3TB Western Digital Reds - SSD for...

Just spamming my summary post from the list: Thought I'd do a quick summary of my results - very subjective really, so take with a pinch of salt. As I suspected, raw disk becnhmarks, either from within the VM (Crystal DiskMark) or on the host (dd, bonnie++) while interesting, arent' a very...

FIFO initially, then LRU - LRU felt like it gave bettere results. Not sure unfortunately, it was a concern. Also wether the backups woul "wear out" the SSD I did try this with ext4, using the one SSD. 10GB for the Journal, 50GB for EnhanceIO (read/write) Seemed quite fast, but generated...

In a test VM, Read performance is good, 200-300MB/s after cache is loaded. Even with writeback cache write performance is abysmal, 25 MB/s, when I was getting 35MB/s for uncached XFS, 120MB/s for cached EXT4

Found the problem - zfs does not support O_DIRECT. I had to set the kvm cachemode to Write Through

Well things looked promising but ended up somewhat disappointing. Setup to zfs pools (1 disk, 1 cache, 1 log) and mount on each node, created a glusterfs store on top of them, added as proxmox storage quick dd test seemed ok. Copied a vm image over to the new storage - fails to start with...

Not using containers - plus with kernel 3.10 containers aren't even an option :)

Beyond our budget and time at the moment. I'm testing glusterfs on top of zfs for shared storage.

Thanks. It updated my grub which was surprising. I'll let you know how it goes.

Good point, thanks. You talked me into it, I'll give it a spin. Do you use the debian repo from http://zfsonlinux.org/debian.html? Have you found the write cache (ZIL) worthwhile?

Tempting, but the memory requirements seem high - 3Gb for my 3TB disks, this on a server that only has 32GB

And seeing as we are discussing ext4 :) ext4 or xfs for VM image storage? I've been testing a lot lately and ext4 just seems to give better read/write performance, both sustained and random - especially with a external SSD journal. This is with large spindles, no raid (3TB WD Red) You can use...

Its a good question, ext4 is well out of its dev phase and is the default for most distro's now. And given how common it is to use an SSD for boot now, ext3 as a defaul is positively dangerous.

Hmmph - my previous post was actually written yesterday but I neglected to press the post button :( I'll try it with ext4 and an external journal today. The gluster people heavily push xfs as the underlying filesystem, but every test I have done in my usecase has it slower. And while you can...

I gave it a spin today. Installation - trivally easy. Install build-essential, dkms, git and the kernel headers. Instrtuctiosn from source: https://github.com/stec-inc/EnhanceIO/blob/master/Install.txt Setup by far the easier and less error prone compared to dm-cache. I love that it sets...