Search results

And maybe you are using cephfs to store something

Destroy all osds per node, that would be easiest.

This is okay setup for me, usually we don't have more than 4xdisks per one nvme for caching . If you need more than that we usually recommend adding more nvme caching drives. Yes, if this nvme dies, all osds which are cached on it die, but this is acceptable in CEPH.

I have esxi running under proxmox for more than 5 years, currently version 7x

Thats okay if the backups are right. Who cares. Just demolish ceph cluster and bring it up again.

Can you do the backups of the current VMs?

reinstall ceph. Not whole proxmox, but just ceph inside it. And restore backups from PBS or whatever you have. This is usually procedure for it; systemctl stop ceph-mon.target systemctl stop ceph-mgr.target systemctl stop ceph-mds.target systemctl stop ceph-osd.target rm -rf...

Yeah, usually with Zabbix, nagios has some great scripts: https://github.com/nbuchwitz/check_pve

Easiest way is to look at ceph dashboard inside proxmox.

You didnt specify ssds models, i would like that before any recommendation about ceph.

Why didn;t you just restart it every night at 1?

Use Influx v1 or v2 ,usually it depends on pre-built dashboard on grafana. v3 is lets say opensource, but cripppled, so we are waiting for influx v1 or v2 fork in the future.

As for storage-replication , HA is working yes ,but automatic failover isn't because it is not a HA storage, and as you can see you can lose the data versus regular HA storage(CEPH etc)> .

I'll answer some misconceptions from Johannes, and give some recommendations: 1) CEPH demands 3+ nodes, with more than 100tb data i would recommend probably 5-7-9 nodes. So out of your scope. 2) Clustering doesnt add complexity since it is already included, and adding quorom nodes is just fine...

I usually work with Duplicati on linux/windows workstation. It is really easy gui, with numerous restore options.

Strange and interesting idea. I would use CEPH for that, it seems that it would suit better. i've never seen single node zfs pool with that size on one machine in Proxmox. But if you stick 20x20tb drives in Raidz2/3 i guess you could do it. And replication only works on ZFS, not on LVM...

Thats why i sad, do the backups, and afterwards just delete CEPH and reinstall it.

If you have more monitors do not restart them before backing all up!

Are the backups working? Do all VM/CTs backups if they are.

Just delete that monitor from other nodes, and start clean.