Search results

  1. A

    [SOLVED] Tabletop review: How to get remote-sync PBS data to the "on-prem" for a rebuild

    Hi Dcapak, Thanks very much for taking the time to reply! I do like the idea of using the external storage for a datastore. Would I need another instance of PBS or can I "remote" back in on itself? I may do some testing with that theory and see if I can make it work. Thanks! -Eric
  2. A

    [SOLVED] Tabletop review: How to get remote-sync PBS data to the "on-prem" for a rebuild

    I'm doing a tabletop review of our off-site backup strategy and have run into a snag. We have an on-prem PBS for frequent (daily/weekly) backups and convenient recoveries (runs as a VM on our proxmox cluster). This works fine, however, for all the reasons... We also have an off-site PBS...
  3. A

    Ceph 17.2 Quincy Available as Preview and Ceph 15.2 Octopus soon to be EOL

    Installed Quincy on home/test cluster last night. Perfectly smooth install, no hiccups, no issues. Everything running smoothly. Perfect instructions as usual from the Proxmox team!
  4. A

    AMD Nested Virtualization not working after 7.1 upgrade

    Just set the hardware CPU type to "host" , seems to be working. Just migrated that running VM to another node on the cluster and it seems to be working even through a migration. We also configured the virtual machine with QEMU 6.1, UEFI, secure boot, a UFI disk, and a vTPM. Now... since we're...
  5. A

    AMD Nested Virtualization not working after 7.1 upgrade

    7.2 Update seems to have resolved this issue for us. Have a guest now running CG/VBS on our AMD-EPYC-ROME cluster.
  6. A

    AMD Nested Virtualization not working after 7.1 upgrade

    Going to give this thread a nudge for some increased attention. 6 node cluster of matching 2113S-WTRT w/7402P CPU's. We are attempting to enable credential guard / virtualization based security for our windows server guests. This is dependent on Hyper-V working on the guest. I believe the...
  7. A

    Poor write performance on ceph backed virtual disks.

    Hi AngryAdm, Random 4K reads at Que depth 1 will always be pretty slow. That is heavily impacted by the network latency combined with drive access latency / service time.
  8. A

    kernel panic: BUG: unable to handle page fault for address: 0000000000008000

    Similar issues on our cluster happening at random. Started a few weeks ago. Have had a node freeze twice in the last few weeks, bringing down lots of VM's/services in a production environment. We have not made any changes to underlying hardware or BIOS config recently. Jul 28 10:31:13 px3...
  9. A

    Poor write performance on ceph backed virtual disks.

    We moved forward with the install of some NVME DB/WAL drives based on Ingo S's post. We are using the Micron 7300 Pro M.2 2TB drives for this and have 437G per 16TB drive assigned as DB/WAL space. Result is about 3% of total space being NVME for DB/WAL. BIG improvement! Now seeing ~150MB/s...
  10. A

    Awfully slow write performance

    I have the same behavior on Windows Server VM's on Proxmox. 30MB/s is about the same performance I get on spinning disks. The actual transfer has the same bad behavior you've described where it "peaks" for a little while then stalls at zero, then peaks, then stalls.... yuk! On linux VM's the...
  11. A

    Poor write performance on ceph backed virtual disks.

    6 X 10Gb 1: Coro1 2: Coro2 3: CephP 4: CephC 5: Network Trunks 6: Unused Write performance from Windows guests is limited to approximately the write-sync performance of drives in the pool. Other guests do slightly better. Ceph bench shows results similar to expected bare drive performance...
  12. A

    Poor write performance on ceph backed virtual disks.

    Hi Ingo, Our DB/WAL is directly on the spinners on both my home cluster and work cluster. Write performance on my home cluster with far less hardware power seems to perform better. Odd eh? I'm willing to try the dedicated WAL/DB disk. The servers have multiple M.2 slots on the motherboards...
  13. A

    Poor write performance on ceph backed virtual disks.

    Production cluster is still on Nautilus. I did a bunch of testing at home to fine a "best config" to try for the production cluster. Best performance I can get on a windows guest seems to be krbd, virtio block, iothread, writeback, and then configuring windows to disable write cache buffer...
  14. A

    Poor write performance on ceph backed virtual disks.

    Write back starts off fast... like 150MB/s as reported in ceph logs and VM summary graphs, but within a few minutes drops to 25MB/s. It also has a very nasty problem. When I cancel a file copy mid-way, windows "cancels" it, but there's a crap ton of data still waiting to be flushed on the...
  15. A

    Poor write performance on ceph backed virtual disks.

    I just set the virtual disk to direct sync mode to "test a theory." Big surprise here: 8MB/s. So how do we get the VM to respect the actual cache settings?
  16. A

    Poor write performance on ceph backed virtual disks.

    Hello mmidgett, Any file copy operation or samba share file copy operation suffers the severe performance bottleneck when writing to the spinning pool. When I copy a file from the spinning pool to the SSD pool, I also get about 100MB/s in non-cache mode just like you, and windows "behaves"...
  17. A

    Poor write performance on ceph backed virtual disks.

    I turned on krdb for this pool, then shut down and booted the 2 VMs with virtual disks in this pool. Performance appears to have improved about 60%. So my instance of PBS is writing ~80MB/s instead of ~50MB/s to this pool, and the Windows File server is now moving at a scorching 8MB/s instead...
  18. A

    Poor write performance on ceph backed virtual disks.

    Hello! We're getting 5MB/s write speed on the production cluster now. That's on a 10Gb network. The SSD pool on this cluster rebalances and recovers at speeds of 500-2000MB/s. This is not a network or CPU issue. With max backfills and recovery_max_active cranked up a bit the spinning disk...
  19. A

    Poor write performance on ceph backed virtual disks.

    Tried a few things.... Enabled the autoscaler, which shrunk the number of pgs quite a bit. Performance dropped a bit after rebalancing. Gave the autoscaler some information about how large the pool is likely to be down the road, and it grew the number of pgs quite a bit. Performance dropped...