Search results

We have a cluster on which I would like to implement per VM firewalling. We enabled firewalling on the datacentre, node and guest basis and finally set the VM's network interface to also use the function. Reviewing iptables shows packets matching the guest's IN chain but nothing in that chain...

We have a 3 x HP ProLiant DL380 G7 node cluster which is working perfectly: kvm1: HP ProLiant DL380 G7 (583914-B21) BIOS: 05/05/2011 kvm2: HP ProLiant DL380 G7 (583914-B21) BIOS: 12/01/2010 kvm3: HP ProLiant DL380 G7 (583914-B21) BIOS: 12/01/2010 Running Ceph 12.2.7, two NICs in a LACP bond...

The following is most probably a little crude but essentially collects concurrent storage IO statistics from cluster nodes and replaces 'rbdX' references with Ceph RBD image names. Useful in finding guests which generate allot of disk IO: #!/bin/sh hosts='kvm1a kvm1b kvm1c kvm1d kvm1e kvm1f'...

We're running 4.15.18-1-pve on pure Intel servers (two integrated 10GbE NICs and two 10Gbps SFP+ add-on module NICs): Everything's working as expected. We're running active/backup bond interfaces using OVS and disable GRO on the VM traffic NICs: /etc/rc.local ethtool -K eth0 gro off; ethtool...

Herewith another simpler example where eth0 and eth1 are LACP members and the LAG is an untagged member of VLAN 1 with other VLANs trunked through: /etc/network/interfaces auto lo iface lo inet loopback allow-vmbr0 bond0 iface bond0 inet manual ovs_bridge vmbr0 ovs_type OVSBond...

Herewith before and after network configuration files that may help you. eth0 and eth1 are untagged members of VLAN 1 (Management), eth2 and eth3 are untagged members of VLAN 33 (Ceph). NB: VM's network configuration should generally be unchanged, except that you would leave the tag undefined...

If you're new, you may prefer to benchmark your guests directly. Microsoft's diskspd application is actually pretty good. Have a look here for sample commands: https://forum.proxmox.com/threads/ceph-bluestore-crash-compressed-erasure-coded-pool.43678/#post-209893

Hi Alwin, We manage erasure coded and compressed erasure coded Ceph pools via the CLI and subsequently manually edit the VM configuration files. It would be nice if the GUI obtained information on the images (rbd info <pool>/<image>) to know that data is located in an alternate pool. The...

Apologies, I don't recall seeing your post. I've posted the commands I used in another topic: https://forum.proxmox.com/threads/created-an-erasure-code-pool-in-ceph-but-cannot-work-with-it-in-proxmox.45099/#post-215986

Looks like you simply have to enable overwriting. Herewith my notes from when I setup an erasure coded pool (8 months of production use) and a compressed erasure coded pool (3 months of production use): NB: We run the pool with a min_size of 4 (3 data and 2 parity shards) and subsequently...

We ordered 2.5 inch HGST NVMe drives to fit our Intel servers but they also manufacturer PCIe drives with even higher throughput. My FIO command was slightly different to yours, from benchmarking prior to adding them to a Ceph cluster: fio --filename=/dev/nvme0n1 --direct=1 --sync=1...

Ceph and Proxmox would need to reach quorum, which means a vote of more than 50%. ie: Impossible with only 4 nodes, when 2 are unavailable. Ceph RBD crush map would also need a min_size of 2, with replication count of 4...

Running virtual firewalls or routers, without requiring many interfaces, is possible by using QinQ VLANs. The different sites, zones or services are delivered to the VLAN aware virtual guest using an outer customer delivery VLAN. This essentially requires Proxmox to pop the outer delivery VLAN...

We have unfortunately started receiving feedback that some guests intermittently have network performance degradation unless we disable GRO on the physical NICs: /etc/rc.local ethtool -K eth0 gro off ethtool -K eth1 gro off These NICs are part of an OVS active/backup bond interface. We...

It's surprisingly stable, the rados benchmarks from the post yesterday everything were with 3 deep scrubs active at the time... This is with OVS on an active/backup 10Gbps bond were we set the active (primary) NIC to a common switch stack member.

My point above, where I stated that our erasure coded pool outperforms a replicated pool, should be taken in context to this thread. We primarily run replicated pools for stability purposes and use an erasure coded pool and compressed erasure coded pool for specific purposes. Simply dumping data...

RADOS bench: rados bench -p ec_nvme 10 write --no-cleanup Total time run: 10.060251 Total writes made: 2187 Write size: 4194304 Object size: 4194304 Bandwidth (MB/sec): 869.561 Stddev Bandwidth: 240.658 Max bandwidth (MB/sec): 1056 Min bandwidth...

IRQs are not balanced evenly when using VirtIO multi-queue. Does anyone know if this can be adjusted? The following KVM multi-queue site references a flow director, anyone know anything about where to start digging?: https://www.linux-kvm.org/page/Multiqueue-optimization queues 2 & 5 receive...

We've added some NVMe drives to our production cluster and get maximal IOPS (100k) and throughput (1GB/s) when benchmarking with 8 threads. Could someone please detail the commands that Proxmox issues when partitioning and formatting drives? We essentially want to partition our drives (3.6TB)...

We've been happy with our erasure coded pools so far, they however require non-rotational media. Our replicated pools use a size of 3 and min_size of 2. Two device failures would hang writes until such time that OSDs are marked out but this yields 300% overhead. Our erasure coded pool uses k=3...