Hello,
I have @home a small 3 nodes cluster running PVE 4.4.12. It's based on 2x NUC with core i5 and 16GB and formerly, a virtual node based on a virtualbox instance on a NAS just used for Quorum purpose.
As I've planed to replace one of my NAS (Syno) by an HP µserver Gen8 I've installed PVE on it too in order to get rid of the virtual node and virtualize the NAS instead.
The HP boots on a dedicated SSD and I've configured a R5 based on 4x2TB which in mounted under /var/lib/vz and acts as a local storage.
When I "bench" the local storage, using rsync --progress or scp, I'm able to write from local SDD to R5 at an average of ~250MB/s and from a remote client at an average of ~110MB/s (limited by the 1GB connection).
I've created an OpenMediaVault3 VM localy with a 16GB Virtio/qcow2 for the OS and a extra 4.5TB Virtio/qcow2 for the DATA:
bootdisk: virtio0
cores: 2
ide2: none,media=cdrom
memory: 4096
name: vmomv1
net0: virtio=7E:6F:E9:E8:3B: D0,bridge=vmbr0
net1: virtio=B6:F1:4E: D1:A7:61,bridge=vmbr1
numa: 0
onboot: 1
ostype: l26
scsihw: virtio-scsi-pci
smbios1: uuid=4f89b895-e7a0-46ee-a95f-6a441a116191
sockets: 1
virtio0: local:114/vm-114-disk-1.qcow2,size=16G
virtio1: local:114/vm-114-disk-2.qcow2,backup=0,size=4608G
When I write files to this VM (I've made tests with a basic Jessie VM too), being using smb, nfs or scp/rsync, throughput starts around 80MB/s for few seconds and then drops to few MB/s, sometime stalled, then increase to 50 MB/s and so on ... Average is about 15MB/s for my tests based to a single 2GB file.
During that period, Proxmox shows IO Delay about 1 to 10% on the GUI.
If someone could help me to explain/twink/analyze this behavior I'll be really grateful !
PS: I've tried adding RAM and vCPU to the guest, tried several kernels, disk emulation (ide/scsi/virtio) and caching options with no luck ...
Thanks in advance for your help
Olivier
PS: Reads are always fine with an average of 110MB/s (1GB network)
I have @home a small 3 nodes cluster running PVE 4.4.12. It's based on 2x NUC with core i5 and 16GB and formerly, a virtual node based on a virtualbox instance on a NAS just used for Quorum purpose.
As I've planed to replace one of my NAS (Syno) by an HP µserver Gen8 I've installed PVE on it too in order to get rid of the virtual node and virtualize the NAS instead.
The HP boots on a dedicated SSD and I've configured a R5 based on 4x2TB which in mounted under /var/lib/vz and acts as a local storage.
When I "bench" the local storage, using rsync --progress or scp, I'm able to write from local SDD to R5 at an average of ~250MB/s and from a remote client at an average of ~110MB/s (limited by the 1GB connection).
I've created an OpenMediaVault3 VM localy with a 16GB Virtio/qcow2 for the OS and a extra 4.5TB Virtio/qcow2 for the DATA:
bootdisk: virtio0
cores: 2
ide2: none,media=cdrom
memory: 4096
name: vmomv1
net0: virtio=7E:6F:E9:E8:3B: D0,bridge=vmbr0
net1: virtio=B6:F1:4E: D1:A7:61,bridge=vmbr1
numa: 0
onboot: 1
ostype: l26
scsihw: virtio-scsi-pci
smbios1: uuid=4f89b895-e7a0-46ee-a95f-6a441a116191
sockets: 1
virtio0: local:114/vm-114-disk-1.qcow2,size=16G
virtio1: local:114/vm-114-disk-2.qcow2,backup=0,size=4608G
When I write files to this VM (I've made tests with a basic Jessie VM too), being using smb, nfs or scp/rsync, throughput starts around 80MB/s for few seconds and then drops to few MB/s, sometime stalled, then increase to 50 MB/s and so on ... Average is about 15MB/s for my tests based to a single 2GB file.
During that period, Proxmox shows IO Delay about 1 to 10% on the GUI.
If someone could help me to explain/twink/analyze this behavior I'll be really grateful !
PS: I've tried adding RAM and vCPU to the guest, tried several kernels, disk emulation (ide/scsi/virtio) and caching options with no luck ...
Thanks in advance for your help
Olivier
PS: Reads are always fine with an average of 110MB/s (1GB network)
Last edited: