spirit's latest activity
-
spirit replied to the thread Ceph - VM with high IO wait.please don't double post, https://forum.proxmox.com/threads/applying-pve-qemu-kvm-10-2-1-1-may-cause-extremely-high-%E2%80%9Ci-o-delay%E2%80%9D-and-extremely-high-%E2%80%9Ci-o-pressure-stalls%E2%80%9D-patches-in-the-test-repository.182186/page-2... -
spirit replied to the thread Ceph - VM with high IO wait.the thing is that the host itself don't mount the rbd. It's done directly by each qemu process. the librbd is inside qemu, and this is the qemu process which is doing to connections to monitor && osd. (with kbrd it's different, as it's done by...
-
spirit replied to the thread EVPN SDN Multi-tenancy: Something similar to Inter-AS Option A?.if you have a vyos router, the best way is to peer with evpn with other proxmox nodes. (1 peer by vrf. 1 proxmox each zone is a vrf). if your coke tenant is a layer2 behind vyos, vyos should announce their ip/mac inside the evpn zone.
-
spirit replied to the thread Ceph - VM with high IO wait.It's really done when the qemu process is starting when librbd is initialized. (like other tuning that you can do in ceph.conf). It could be possible to pass params to qemu commandline, but qemu rbd driver have limitations of parameters that you...
-
spirit replied to the thread [SOLVED] VXLAN ARP timeouts.I 'm not sure that hook can work with live migration, as the arp entry need to be set after source vm stop (the arp entry need to be flushed from source horst) and before the resume of the target vm. This is a thing that we should implemented...
-
I finally solved problem thank u for help The root was my network devices configurated as stack and corosync was connected to different units, thats caused link flapping, when i connected to 1 unit problem was gone
-
spirit reacted to IsThisThingOn's post in the thread I/O stall while backup to PBS is running and fdatasync failure after backup fails with
Like.
Why? Transmission is encrypted, you don't need Wireguard. I reframe that so it gets easier for a beginner to understand. I know this, because I only learnt that a few days ago :) When the backup is running, and the VM wants to do a write, the... -
spirit replied to the thread I/O stall while backup to PBS is running and fdatasync failure after backup fails.try to enable fleecing option in backup scheduling advanced option. when backup is running, the new writes are going first to the pbs storage and that can give you slowdown if the pbs is slow or have more lantency or have limited bandwidth. The...
-
spirit replied to the thread Ceph - VM with high IO wait.If can really confirm that memory allocator impact librbd latency. you can with a simple fio 4k randread or write. my last qemu patch from 2023 : https://lists.proxmox.com/pipermail/pve-devel/2023-May/056815.html default malloc: 60k iops 4k...
-
spirit replied to the thread live migration: ram_save_setup failed: Input/output error.for hardening, you can do something like DisableForwarding yes Match User root Address <<10.0.10.0/24,10.0.20.0/24>> DisableForwarding no AllowTcpForwarding yes X11Forwarding no AllowAgentForwarding no PermitTunnel no (replace...
-
spirit replied to the thread Corosync link flapping with 3 nodes.So, here, at 22:04, why is your link down ? node reboot ? if not, you have clearly a problem with nic , the cable, or the switch port[/spoiler]
-
spirit replied to the thread Corosync link flapping with 3 nodes.corosync mtu is always lower than real mtu. in my production, I have also 1397 pmtud with 1500 mtu on nic Feb 19 11:05:20 corosync[23618]: [KNET ] pmtud: PMTUD link change for host: 4 link: 0 from 469 to 1397
-
spirit replied to the thread NAS iSCSI LUN and LVM-Shared on top - bad IO while having VM delete ongoing.the current code is my $secure_delete_cmd = sub { my ($lvmpath) = @_; my $stepsize = $scfg->{'saferemove-stepsize'} // 32; $stepsize = $stepsize * 1024 * 1024; my $bdev = abs_path($lvmpath); my...
-
spirit replied to the thread Ceph - VM with high IO wait.Hi Tchaikov, I think that the user setup is hyperconverged with proxmox/ceph on the 3 nodes. So, I think that the rbd client is able to handle it without hook . could you confirm this ? (I have seen other proxmox users doing it in hyperconverged...
-
spirit replied to the thread Corosync link flapping with 3 nodes.it's look like to nic of the 3th is going down/up or flapping. nic driver bug ? maybe bad cable ? do you have any kernel log on the 3th node ? #dmesg ? maybe also try without bonding/lacp with 2 corosync links
-
spirit replied to the thread Proxmox/Ceph - Disable OSD rebalancing.I think what you want is to simply enable noout flag or set mon_osd_down_out_interval to 1hour instead 10min
-
spirit replied to the thread ZFS mirror on 2x Crucial T705 (PCIe 5.0) causing txg_sync hangs under write load – no NVMe errors in dmesg.10year old ceph blog, also apply for zfs https://ceph.io/en/news/blog/2014/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/ https://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/...
-
-
spirit replied to the thread Cluster retransmit issues.About spanning-tree, you should really disable it on your physical switch port of for your proxmox nodes. a spanning tree convergence can happen on host reboot and broke the whole cluster for some second. you don't need change knet_mtu, it's...
-
spirit replied to the thread Ceph - VM with high IO wait.Strange that you also have high memory pressure "PSI some memory". do you have enable numa option on the vm ? you can also look at host numa stat # apt install numactl # numstat and look if you don't have a lot of "numa_miss" vs "numa_hit"...
-
spirit replied to the thread Cluster retransmit issues.can you send corosync log of each node ? (journalctl -u corosync). is the nic for corosync link dedicated ? or do you have vm,storage,backup,...running on it too ? (no bandwidth saturation ? ) no spanning tree on the network ? do you use...
