Search results

  1. C

    High swap despite plenty of available memory

    Hi, I'm running a hyperconverged cluster (Proxmox & Ceph) across three nodes. I've noticed the swap usage is creeping up. Each node has 192GB of RAM and 8GB of swap. Ever since system install I've reduced the swappiness to 10. Looking at what is actually eating the swap, it's a few KVM...
  2. C

    apply rrd journal failed - stream did not contain valid utf-8

    Hi, Our backup servers RAID array hit a bit of an issue this morning and it ultimately caused the PBS to crash. Now I can't view the datastore and my journal is giving the following errors - Jun 24 04:01:52 pbs proxmox-backup-proxy[191]: GET...
  3. C

    Isolated virtual network using SDN

    Hi, I've just upgraded to 8.2 and after a little trouble with networking (interface names changed) I'm up and running and testing out SDN. I'd like to create an isolated subnet 192.168.100.0/24 that uses a specific upstream so that only machines within this subnet can talk to eachother...
  4. C

    Configuring VM - single or multiple sockets?

    Hi, I was just wondering if there is any particular benefit in configuring a VM to use multiple sockets, e.g. 2 sockets, 5 cores, vs assigning it 1 socket, 10 cores? I'm running some benchmarks trying to squeeze as much performance as I can and it seems that 1 socket, 10 cores is a little more...
  5. C

    Low throughput on Ceph interface

    Hi, I've got a three node proxmox cluster with a dedicated Ceph OSD network on a 10GBit link. It's been working great on our providers switching equipment but they've recently had some rack changes and plugged into new switches. During this change, I took the opportunity to update the Proxmox...
  6. C

    Proxmox 8 Migration Issues

    Hi, I'm running a three node hyperconverged cluster. I've been running Proxmox 7 for some time and decided to take the leap to Proxmox 8. Since then, I've noticed some migration issues regarding disks. Below is an example of one of the errors I got when trying to migrate. The migration actually...
  7. C

    Host PVE behind PFSense on same PVE

    I'm doing this for a proof of concept. I've got a single Proxmox server with a single NIC, connected to a router. The gateway of the router as an example is 190.150.165.1 and I have access to IP's 190.150.165.35 and 190.150.165.41-45 (all not real IPs). Is it possible to configure a vmbr0...
  8. C

    Does live restore instantly power the VM on?

    I'd like to do a live restore (as a new VM alongside the existing) but I don't want the VM to instantly power on. Before booting I want to change the VLAN it's sat on so that it doesn't conflict with the existing machine. Is this possible to achieve? Thanks, Chris.
  9. C

    Node maintenance mode UI

    Hi, the new node maintenance mode is a great addition to Proxmox, thank you! Running sudo ha-manager crm-command node-maintenance enable pve01 worked perfectly. However, it'd be really nice to be able to manage maintenance state directly from Datacenter > HA. Is this something that's in the...
  10. C

    [SOLVED] Ceph mgr daemon crash - NoOrchestrator exception

    Hi, I've suddenly started getting crashes on the active Ceph manager node. It looks to be that the reason is caused by NoOrchestrator. We upgraded to Quincy at the weekend, but have never had any orchestrator modules. root@pve01:# ceph crash info...
  11. C

    Ceph - writethrough caching method

    I've been testing various disk caches with Ceph RBD. I'm using the writethrough cache mode and getting very impressive figures. On a Linux machine, I'm running fio tests and have the below config to test random read speeds - [global] name=fio-rand-read filename=fio-rand-read rw=randread bs=4K...
  12. C

    [SOLVED] Issues with iowait after faulty LAG

    Hi, We have a three node setup, each of which is running Proxmox & Ceph. Each node has a four-port 10GBit link. Two ports for Ceph and two ports for public/private networking. There are also two 1GBit ports used purely for Corosync. Link aggregation for Ceph works great, Ceph is nice and...
  13. C

    [SOLVED] Mail headers showing Received: from hostname (localhost.localdomain [127.0.0.1])

    Hi, I've installed and configured PMG. We're relaying email both inbound and outbound via a smarthost. I noticed on the headers that I get the following - Received: from mxserver.example.net (localhost.localdomain [127.0.0.1]) by mxserver.example.net (Proxmox) with ESMTP id 69FD7201062...
  14. C

    [SOLVED] Cannot backup 3.6TB Ceph guest - IOWait through the roof

    I have a 3.6TB guest which was backing up fine (runs on Ceph), upgraded our nodes on Saturday and now I'm having to cancel the backup because it's causing the guest to become unrepsonsive during the process. I can see it looks like it's having to rebuild from scratch, but thought this wouldn't...
  15. C

    Recreated Ceph cluster throwing errors

    When I created my Proxmox cluster, I installed Ceph and did some testing. We had a few issues with Ceph so I followed some guides on uninstalling it from Proxmox. Now we've recreated our cluster and it's working nicely. However, when I reboot one particular node (pve01), it takes 15-20 minutes...
  16. C

    [SOLVED] Ceph gradual growth in VM IOWaits

    Hi, We've installed a 3 node Proxmox cluster running Ceph. On the cluster we have two VM's. When we reboot all PVE nodes, the IOWait of the VM's drops to pretty much nothing. However over time, the IOWait creeps up - there is no load on these VM's. Any idea why this might be? As you can see...
  17. C

    [TUTORIAL] Hardware watchdog at a per-VM level

    From my testing of Proxmox, one frustration I had was that unlike my previous Xen environment, Proxmox does not detect if a VM has panicked/crashed/frozen and as such won't reboot the VM, potentially ending up in hours of downtime until the issue is realised and resolved. After a bit of digging...
  18. C

    CephFS vs VirtIO SCSI Write IOPS

    Hi, I've been testing our Proxmox Ceph cluster and have noticed something interesting. I've been running fio benchmarks against a CephFS mount and within a VM using VirtIO SCSI. CephFS on /mnt/pve/cephfs - root@pve03:/mnt/pve/cephfs# fio --name=random-write --ioengine=posixaio --rw=randwrite...
  19. C

    Proxmox Ceph - higher iowait times

    Hi, We're in the process of moving from a cluster that had a networked Dell ScaleIO storage to Proxmox using Ceph. Since moving across a couple of our VM's we've noticed quite an increase in iowait. Is this something that is typical of Ceph due to the nature of it's replication? For reference...
  20. C

    [SOLVED] Deleting user 'delete user failed: cannot update tfa config'

    I've setup a new user from Datacenter > Permissions > Users However on creating the user and attempting to set the password I get change password failed: user 'Chris' does not exist (500). If I then attempt to delete the user I get the below: delete user failed: cannot update tfa config...