Search results

  1. L

    Windows Server 2022 reports disk errors on ceph volume

    We installed a new Windows server 2022 on a cluster that uses an SSD-based ceph volume. All seems to be going well, when suddenly windows event log reports: "An error was detected on device \Device\Harddisk0\DR0 during a paging operation" It's Windows error # 51 There are other Windows...
  2. L

    [SOLVED] Remote server doesn't deduplicate

    I have setup a remote server in a different city to which I ship all backups using a sync job. The remote PBS datastore however doens't seem to be doing deduplication. The local PBS. Usage : 91.02% (3.97 TB of 4.37 TB) Backup Count CT : 32 Groups, 380 Snapshots Host : 0 Groups, 0...
  3. L

    pvesh and how to list API endpoints

    I have seen a couple of blogs out there that claim one can simply use the pvesh command without any parameters and it will drop into an interactive mode where one can show the calls that can be done and a particular level. It doesn't work like that for me though and the documentation is really...
  4. L

    Remote PBS log shows error, but all processes look completed

    Can anyone see what causes this error? 2023-12-18T13:00:07+02:00: percentage done: 98.18% (54/55 groups) 2023-12-18T13:00:07+02:00: sync group vm/199 2023-12-18T13:00:07+02:00: re-sync snapshot vm/199/2023-11-20T08:36:28Z 2023-12-18T13:00:07+02:00: no data changes 2023-12-18T13:00:07+02:00...
  5. L

    Use API to get storage location for VM's

    I need to extract which storage is assigned to each VM and LXC in our cluster. I can retrieve the total allocation for the boot disk, but can't see an obvious way to get the detail for each storage volume allocated. Some of our VM's have a boot disk on an ceph SSD pool and a logging disk on...
  6. L

    Strange disk behaviour

    We're experiencing a problem with a FreeBSD KVM guest that works 100% on installation, but after a while starts complaining that it can't write to the disk anymore. What we have done so far: Moved the disk image off ceph to a lvm-thin volume Changed the disk from Virtio-SCSI to SATA and also...
  7. L

    [SOLVED] Ballooning memory: How to retrieve the max ram allowed from the guest OS?

    Scenario: Centos Guest OS with 8GB/24GB RAM as min/max allocated. The machine typically uses between 10GB and 12GB of the allowed RAM due to ballooning, but here's a problem: Using free -h shows only 14GB in total available. Can't find anything else that shows the 24GB max allowed. There are...
  8. L

    proxmox-backup-proxy rrd EINVAL error

    I'm getting the error below after something happened (it was not happening before) and not sure that I changed anything deliberately. It prevent the status graphs (rrd, right?) to be displayed on the PBS administration section. Oct 11 22:07:17 pbs3 systemd[1]: Starting...
  9. L

    Can one set PBS priority lower to prevent guest slowdowns?

    I have run into an issue a couple of times in that guest OS's slow down dramatically if the PBS server doesn't perform for whatever reason. Previously I had a network issue, which prevented backups from being written at a reasonable speed and it caused the guest machines being backed up to...
  10. L

    [SOLVED] SDN broken after underlying network change

    We ran into a very nasty issue a few days ago. Background: Systemd generates ridiculously long interface names (see https://manpages.debian.org/bookworm/udev/systemd.link.5.en.html and referenced here https://wiki.debian.org/NetworkInterfaceNames#CUSTOM_SCHEMES_USING_.LINK_FILES) like...
  11. L

    Mobile app noVNC cannot be shifted.

    When viewing a QEMU machine console with noVNC, the options are to either scale the screen locally, or not. When scaled locally, the text is so small that it's not practically usable. Disabling local scaling fixes that, but then the view screen cannot be shifted left / right or up / down, so...
  12. L

    What happens during VM migration?

    I have a FreeBSD 12.3 guest running a poller node and when it gets installed everything runs just fine. We can stop and start the guest too, no problem. The guest uses VirtIO SCSI and uses an ceph RBD image of 120GB. The FreeBSD qemu-guest-agent is installed. If for some reason the VM is...
  13. L

    Change WAL and DB location for running (slow) OSD's

    I need to do something about the horrible performance I get from the HDD pool on a production cluster. (I get around 500KB/s benchmark speeds!). As the disk usage has been increasing, so the performance has been dropping. I'm not sure why this is, since I have a test cluster, which higher...
  14. L

    [SOLVED] How to remove old mds from ceph? (actually slow mds message)

    I had a failed node, which I replaced, but the MDS (for cephfs) that was on that node is still reported in the GUI as slow. How can I remove that? It's not in ceph.conf or storage.conf MDS_SLOW_METADATA_IO 1 MDSs report slow metadata IOs mdssm1(mds.0): 6 slow metadata IOs are blocked > 30 secs...
  15. L

    How to force quorum if the 3rd monitor is down

    I have a situation where a node failed (due to the boot drive failing) and then another node failed (due to RAM failure). There are 7 nodes in the cluster, so things kept running, but eventually there were many writes that could not be redundantly stored and the whole thing ground to a halt...
  16. L

    Replace failed boot drive without trashing ceph OSD's?

    I have a failed boot drive in a 7 node proxmox cluster with ceph. If I replace the drive and do a fresh install, I would need to trash the OSD's attached to that node. If I could somehow recover the OSD's instead it would be great and probably save time too. Is that possible?
  17. L

    ceph thin provisioning for lxc's not working as expected?

    I have an LXC that is provisioned with a 100GB boot drive using ceph RBD storage. However, see the following: ~# df -h Filesystem Size Used Avail Use% Mounted on /dev/rbd10 98G 8.8G 85G 10% / This is in the running container. Checking the disk usage in ceph however, claims...
  18. L

    How to fence off ceph monitor processes?

    I the continuous process of learning about running an pmx environment with ceph, I came across a note regarding ceph performance: "... if running in shared environments, fence off monitor processes." Can someone explain what is meant by this and how does one achieve this? thanks!
  19. L

    [SOLVED] LXC with more cores assigned uses dramatically less CPU. Why?

    I have an interesting situation. An LXC running Power-mail-in-a-box has 4 cores assigned (with 8GB RAM and 100GB NVMe ceph pool storage). The graph below shows the following: The section from 9:32 to around 10:02 is when I only had 4 cores assigned. Before and after that time I had 12 cores...
  20. L

    How to deal with unresponsive lxc and kvm guests in HA context

    We had an interesting situation this morning. For some reason one node in our cluster was not showing as active (green "running" arrows on the guest icon on the tree) and all the LXC's were not responding. We managed to address the issue as quickly as possible by simply resetting the node and...

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!