Search results

  1. What does this mean? corosync[3587892]: [KNET ] link: host: 2 link: 0 is down

    Update: So one of two things made this go away: 1. Rebooting all the nodes. 2. Migrating a VM off of one of the nodes. When doing the live migration of the VM..it was going absurdly slowly, like 56k over a 10GbE link with no other traffic. After migrating that VM and rebooting, all is well...
  2. What does this mean? corosync[3587892]: [KNET ] link: host: 2 link: 0 is down

    Right I did think that Ceph was just saturating the link but after the rebalancing stopped and I started seeing very little traffic on the 10g link...that's when I got suspicious. I'll do some digging on how to read that stats output but if you could help me interpret it, that would be helpful...
  3. What does this mean? corosync[3587892]: [KNET ] link: host: 2 link: 0 is down

    We had a power outage yesterday. For unrelated reasons, 4/5 of the nodes did not power up successfully. As I powered them up, I would fix each node, which added maybe 5 minutes to each node's bootup. After this, I started seeing these messages constantly on all nodes but _only_ for the secondary...
  4. [SOLVED] Update 7.0-11 to 7.1-8 Ceph issues

    So this is now solved, I think. According to my error log, it seems like, somehow, an MDS was reporting an older version than 16.2.7. Checking the CephFS section in the Proxmox web UI, I actually had 2 MDSes that were reporting _no_ version string at all. I first did ceph mds fail...
  5. [SOLVED] Update 7.0-11 to 7.1-8 Ceph issues

    I've posted a message to the ceph mailing list here: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/FU6JPZNLY2PVF4ZV7PYP2KDJ4UFSVOR2/
  6. [SOLVED] Update 7.0-11 to 7.1-8 Ceph issues

    Yup, I can confirm. Tried to turn it off this morning and I still get assertion failures. I wonder if I can delete and re-create each mon one at a time...
  7. [SOLVED] Update 7.0-11 to 7.1-8 Ceph issues

    Did you upgrade all of your nodes?
  8. [SOLVED] Update 7.0-11 to 7.1-8 Ceph issues

    I'm having this happen now as well, I think you're supposed to switch it back after the upgrade to the mon db is done but I don't know how to tell when that happens?
  9. Windows 10 VM with GPU Passthrough Chugging after Upgrade to Proxmox 7.0

    For the past few years I've been gaming on a Windows 10 VM with a GTX 1070 Ti. This has been working surprisingly well. After upgrading to Proxmox 7.0, the VM is now chugging seemingly randomly. I can play for an hour or two fine but then suddenly, I'll start getting "skips" for about half...
  10. Confusing Ceph GUI Info when using multiple CephFS volumes

    Created a bug here, please let me know if there's anything else I can do: https://bugzilla.proxmox.com/show_bug.cgi?id=3616
  11. Confusing Ceph GUI Info when using multiple CephFS volumes

    I recently upgraded to Proxmox 7 and Ceph Pacific which brought multiple CephFS support. My goal was to create one FS on my HDD OSDs and one FS on my SSD OSDs so I can balance workloads across the two sets of hardware. I have a "performance" and "capacity" crush rule. Previously, I had 2 RBD...
  12. [SOLVED] Can't migrate VM on Ceph when source node _also_ has local-zfs but VM isn't using it

    Wow ok, so there apparently is indeed a "110/vm-110-disk-0.qcow2" in that directory back from November. I guess it didn't get deleted. I'm assuming it didn't list it because the storage doesn't support images. Moving that disk out of the way allowed the migration to continue. Would you consider...
  13. [SOLVED] Can't migrate VM on Ceph when source node _also_ has local-zfs but VM isn't using it

    Whoops, I forgot to run that as root, here's the actual output: Use of uninitialized value $node in concatenation (.) or string at /usr/share/perl5/PVE/Storage.pm line 145, <DATA> line 755. Use of uninitialized value $node in concatenation (.) or string at /usr/share/perl5/PVE/Storage.pm line...
  14. [SOLVED] Can't migrate VM on Ceph when source node _also_ has local-zfs but VM isn't using it

    Yeah, that's what I was thinking too, it seems very strange. When I run that I get "storage 'local-zfs' does not exist". Is there a way to increase debug output during that migration?
  15. [SOLVED] Can't migrate VM on Ceph when source node _also_ has local-zfs but VM isn't using it

    pveversion -v: proxmox-ve: 6.1-2 (running kernel: 5.3.18-3-pve) pve-manager: 6.1-11 (running version: 6.1-11/f2f18736) pve-kernel-helper: 6.1-9 pve-kernel-5.3: 6.1-6 pve-kernel-5.3.18-3-pve: 5.3.18-3 pve-kernel-5.3.18-2-pve: 5.3.18-2 ceph: 14.2.9-pve1 ceph-fuse: 14.2.9-pve1 corosync...
  16. [SOLVED] Can't migrate VM on Ceph when source node _also_ has local-zfs but VM isn't using it

    I have four nodes setup with Ceph, one of which _also_ has ZFS setup. All of my ISOs, etc. are on ZFS. This particular VM was running on a node _without_ ZFS, just Ceph. I had to take that node offline today so I migrated it to the one _with_ ZFS. After bringing the node back up, I cannot...
  17. Error to migrate VMs or CTs stored in Ceph when one of the nodes doesn't use ZFS

    I'm having a similar issue. I have a single node with a ZFS RAIDZ that is used only by that node for a handfull of VMs and CTs; it's not marked as shared, and only set on that node.. I have 3 additional nodes with all 4 sharing a Ceph RBD pool. I created a VM on this NAS Node during an upgrade...
  18. Guest with Fedora or Centos will not shutdown

    It is indeed an SELinux issue, the following policy solved this for me: module fedora_proxmox 1.0; require { type virt_qemu_ga_t; type systemd_logind_t; class dbus { send_msg }; } allow virt_qemu_ga_t systemd_logind_t:dbus send_msg; allow systemd_logind_t virt_qemu_ga_t:dbus...
  19. Permission denied when trying to create ceph-osd on zvol

    Yeah, everything in there is owned by ceph. Interestingly, so was the block device. I saw similar problems with other kinds of block devices so I'm willing to accept this being a problem with upstream for the moment.
  20. Permission denied when trying to create ceph-osd on zvol

    Ah, I see. You're correct. Since https://pve.proxmox.com/wiki/Ceph_Jewel_to_Luminous was geared towards 4.4, it simply asks you to replace "jewel" with "luminous". The 4.4 to 5.0 upgrade guide at https://pve.proxmox.com/wiki/Upgrade_from_4.x_to_5.0 does tell you to replace the Ceph repos with...

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!