Search results

Use MLAG with both F5 switches and configure LACP 802.3ad LAG both in PVE and in the switches and you will get both links in an active/active setup with failover/failback times tipically under 400ms. Remember to add the LAG to the Ceph vlan. You will lose Ceph quorum because no monitor will...

This is surprising, given the added instructions of v3 and v4 over v2 or v2-AES. Here [1] is the definition of each x86-64 CPU type. Maybe you could try using a custom CPU and manually add each flag and bench your workload to try to find out which one is introducing the performance loss that you...

IMHO something is off with that measurement: it's simply impossible that mechanical disks provide a 8 minute GC for a 2TB datastore, no matter the filesystem. Maybe it's using some cache? I do have some 2-4TB datastores using either single drives + EXT4 or ZFS mirrors and all take way more than...

Does the network of the VMs running inside ESXi work correctly for you?

Yes, you need to disable CSO offloading inside ESXi. Don't really know why just yet, found it after 2 days of trial and error and browsing online docs. The idea came from here [1] and the fact that tcpdump shows lots of crc errors typically related to misbehaving offloading in the drivers...

Sorry, I'm not the OP ;). I too think that something isn't correctly configured in Corosync, host lost quorum and HA made them reboot, hence I asked for the configuration files and other details.

Essentially it's the same procedure as with a major version upgrade, like you posted [1]. Install packages in all nodes, set noout, restart one by one mons, managers, OSD, MDS, while waiting for Ceph status to be ok before restarting the next daemon. Finally, remove the noout flag. PVE webUI...

QEMU guest agent does not keep time in sync like i.e. VMWare tools do. It only syncs time on VM start, after suspend/resume and after a snapshot is taken, either a manual one or from a backup. You need some ntp/sntp client to keep time synced. Most OS do come with some configured by default.

What does happen exactly when you try to access the webUI of ESXi ?

Make vzdump backups of the PBS VM to a Samba or NFS share on that same NAS that hosts you datastore. When the cluster breaks or any other disaster, just install PVE anywhere, restore the backup of the VM and you'll get access to your backups. Still, I would install PBS anywhere else but inside...

Post your /etc/pve/corosync.conf and /etc/network/interfaces of the nodes. Also the current output of corosync-cfgtool -n in each node. Full logs of each node at around the time of the reboot will be useful too. We can't really guess anything without that information.

This post would be much more useful if you provide the exact version of the virt-io ISO and the version of each component (spice vdagent, virtio drivers, etc) that each one installs. That information will be needed for the bug report anyway. Virt-IO drivers are hosted in Github [1], but I'm...

Simply apt update && apt install ceph-common. That will update Ceph packages and their dependencies. If in doubt, after apt update, check the available versions for each Ceph packages with something like apt list --installed | grep ceph.

If you care about your data, buy second hand enterprise drives instead of consumer ones. The performance you are seeing is expected with those drives: once the drive SLC cache is full, writes are very slow. Also, no sure if the overhead/complexity induced by Ceph in your single node setup is...

On /etc/network/interfaces add this under the interface secction: pre-up /usr/sbin/ethtool -s $IFACE speed 2500 duplex full autoneg on

As detailed in the docs, max latency is 10ms [1], although IME being around 5ms max is recomended. Clustering is not supported if latency is over that values. In your use case, I would simply have independent hosts or at least an independent cluster on each location. Then set the value for...

Glad to know its ok now! Add that third host asap, like yesterday. Meanwhile you add the third node with HDD's, if for some reason you have to take down one of the HDD hosts or if it breaks, change the pool to 2/1. That will allow to keep I/O to the pool at the (very high) cost of no redundancy...

As aaron pointed out, check the MTU. I've just noticed that you've set mtu 8972 in the nics for the 10.22.0/24 network, instead of the more typical 9000. In your case, ping -M do -s 8972 {target host} won't work, as you have to substract 28 from the interface MTU. ping -M do -s 8944 {target...

Please, post /etc/pve/ceph.conf and /etc/pve/storage.cfg so we get the full picture of the setup. I would also take a look with tcpdump in the new node to check if that host is getting some reply from the Ceph cluster when trying to access it.

Not exactly: data will remain in one node but PGs will become inactive unless the pool is set to 2/1 replicas (which is something not recomended in any case except for disaster recovery).