Search results

  1. B

    Test, nested Ceph cluster SSD latency up to 2000ms

    I configured a nested Proxmox Ceph cluster (3 nodes) for testing on my Proxmox server. 3 nodes, plenty of RAM, CPU power etc. I used 3 good SAS SSDs , 1 per virtual machine. Currently there is nothing else running on this Proxmox serer. All networking works fine , I have 2 x 10Gbps ports and...
  2. B

    [DMA Read NO_PASID] Request device [00:02.0] fault addr 0xcce25000 [fault reason 0x05] PTE Write access is not set

    Experienced it today with my 10years old R720 server taht I use for testing. DMAR DRHD Request device [03:00.0] fault addr ... fault reason PTE read access is not set. I had to replace failing Perc H710P mini and took laying around H310 mini, the server recognized it and did not make a fuss...
  3. B

    PBS backing up relatively big virtual machines

    I am using PBS to backup about 300 systems running on PVE then sync it to a backup location to another PBS. It works GREAT , have been doing it for last year with daily and weekly backups, with no issues. This VMs are relatively small in size, the cumulative size is about 13-14TB. I have a VM...
  4. B

    Adding a NIC post install advice

    Adding network cards should not matter in your case. You can run your VMs from the 1Gpps interfaces for now. Then you will add NICs and simply change the network settings on a VM. As for the cluster , 1 Gbps is more then enough.
  5. B

    Strange network behavior on one port in a bond

    The issue has been fixed. The Ceph cluster network even with 4 nodes needs to use hash 3+4.
  6. B

    VNC argument/port question

    I have the VNC working with args: -vnc 0.0.0.0:xx added to the config file. What happens if the VM is migrated to another node with another VM using the same port ? Is there a wider range of ports that can be used instead of 59xx or perhaps extend that range so I can add it to more VMs from...
  7. B

    Strange network behavior on one port in a bond

    Is there anything else that can be checked ? I cannot do iperf for now. I was able to apply changes to a description of an interface which restarted the network on node1. Now the traffic outbound is using the enp4s0f0 and the enp5s0f0 is not passing any traffic out (outbound) , both interfaces...
  8. B

    Strange network behavior on one port in a bond

    Please note , the traffic TO node1 from the switch (Incoming) is fine and working across both interfaces enp4s0f0 and enp5s0f0. The problem is with sending from node1 (Outgoing per nload) , only enp5s0f0 is sending unless it is unplugged and then enp4s0f0 takes over otherwise the enp4s0f0 is...
  9. B

    Strange network behavior on one port in a bond

    Cannot test at the moment as the node is in production with 4 node cluster having about 200 VMs on Ceph storage. Don't really have a maintenance window scheduled for now. It is working on the second slave interface. I looked at the /proc/net/bonding/bond1 across the 4 nodes and they look the...
  10. B

    Strange network behavior on one port in a bond

    Hi, I am testing by uploading the VMs to the storage and generating traffic that way which is when I took the reading from nload, I can see clearly that the presence on all other ports and or servers for outgoing traffic and absence of that traffic on that particular interface in outgoing...
  11. B

    Strange network behavior on one port in a bond

    proxmox-ve: 7.4-1 ceph: 17.2.6-pve1 I have 4 node Ceph cluster using Proxmox as a host. Public and Private networks are separated using 2 x 10Gbps ports each (2 cards per node , 4 ports total). All nodes are setup in exactly the same way. Here is an example of Ceph Private config: auto...
  12. B

    Ceph PG #

    Thank you for you help and warning, I will keep an eye on it.
  13. B

    Ceph PG #

    Thank you , so the 1024 PGs would be preferred being the calculated value with the ceph pg calculator ? I am warming up to the autoscaler , I have it running on a smaller cluster and it is just working I guess. I am just not sure how it is making the adjustments and how they affect the...
  14. B

    Ceph PG #

    Proxmox 7.4.16. I am getting confused by all the numbers. I have 24 OSDs , SSD 1.46TB across 4 nodes, 3 replicas , total size of the pool 12TB and it is going to be 80-85% full. I did the calculation from ceph calc and it gets me 800 , rounded 1024 PG which is also the number that Ceph...
  15. B

    PBS restore command syntax

    I am playing with restore command for PBS and I cannot seem to get it right. I have local LVM storage called "local1-SSD" where I am trying to restore. proxmox-backup-client restore --repository username@pbs@1.1.1.1:Datastore vm/1000/2023-08-13T23:00:39Z drive-virtio0.img.fidx...
  16. B

    Numa not enabled ?

    ok , thank you, will install.
  17. B

    Numa not enabled ?

    I have a test node on community repo which was an upgrage from 6.4 to 7 with just one socket and numa commands work on it but NOT on any of my 6 nodes with two socked CPUs on them.
  18. B

    Numa not enabled ?

    Is there any reason when numa would not be enabled on Proxmox. I did clean install of Pve 7.3 , enterprise repo. When I try to check the numastat I get command not found, any numa command gives me that response. proxmox-ve: 7.3-1 (running kernel: 5.15.85-1-pve) pve-manager: 7.3-6 (running...
  19. B

    Maintenance on a cluster

    ok, thank you I need to jump on it now so I will do expected 1 and then remove after hours to bring total number of nodes to 2 in the remaining 6.4 cluster. Thx
  20. B

    Maintenance on a cluster

    In production environment can this be done at anytime or should I wait till after hours when the load is low ? I am on 6.4 version. Thank you

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!