For the vm firewall, it shouldn't have any impact (as it's done at bridge level).
I really don't known if you have firewall to protect pve host themself.
But, in any case, if you use multiple exit-node, you need to disable reverse path filtering...
best way is to use a vlan aware setup.
auto vmbr1
iface vmbr1 inet manual
bridge-ports bond0
bridge-stp off
bridge-fd 0
vlan-aware yes
auto vmbr1.100
iface vmbr1.100 inet static
address 10.118.65.10/24
gateway 10.118.65.1
for non vlan aware, you can also add in...
after 24h, no crash with 6.8.4-3-pve. (I'm crossing my finger, because it was crashing multiple times by day with 6.8.4-2-pve )
BTW, I have another cluster with 20 nodes with 6.8.4-2-pve without any problem since 3-4weeks, but with ceph-osd running on it. (Os is installed on 2 nvme m2 in raid1...
oh , sorry, I didnt see the bgpcontrollers with the firewall ip as peer.
>>So every node shares only the routes to IP addresses of VMs, that should not be shared by the specific node.
>>
>>Maybe this is caused by the routes, that each node sees itself?
>>pve-green-01 only shows the routes to...
Hi,
here the common crash trace I have with 6.8.4-2-pve , related to ceph osd with dmcrypt. Was able to reproduce 8 times, always same log (I'm going to test 6.8.4-3-pve).
May 6 03:19:17 server1 kernel: [111961.629710] BUG: kernel NULL pointer dereference, address: 0000000000000cd4
May 6...
Hi,
I don't see how your firewall could be able to receive routes, as you don't peers bgp or evpn with your firewall ?
The way it should be done:
if your firewall support evpn: peers evpn controller directly with it, then configure exit-node on the firewall directly
if you firewalll don't...
pool quota has been introduced in 8.1 (for cpu/mem). Storage can attached to pool too. Then you can add permissions to pool. permissions can be done on sdn too. (by zones or by vnets)
Hi, this is interesting. I have also 20 nodes working fine with 6.8.4-2 (without ceph, without osd) for 3 weeks, but 2 nodes with ceph osd are crashing in 24h.
They are lenovovo epyc v3 servers with nvme drivers.
Do you use encryption for your osd ? (I have a trace related to...
don't use consumer ssd . (and qvo are the worst drive of all crappy consumer ssd drivers)
you need a ssd/nvme with supercapacitor for ceph && zfs, to handle the fsync. (or at minimum as a wal/journal drive)
It's possible to do cross-cluster migration (and online) with cli:
"qm remote-migrate <vmid> [<target-vmid>] <target-endpoint> --target-bridge <string> --target-storage <string> [OPTIONS]"
It'll rename the vmid in config, and no need to use temp cephfs.
maybe can you create a pool without replicat , dedicated to fleecing ?
Personnally (I'm also using ceph), I'll look to add some local nvme disks in my nodes dedicated to fleecing.
the only way to do it like vmware distributed vswitch is to use sdn vlan zone, then create 1vnet for each vlan.
The other way is to define tag in vm nic directly.
(you don't need to define ip address for each vlan to get network for your vms, it's only used for host management ips)
ok , so for outside access, you need to define an exit-node. (1 of your proxmox host).
The exit-node is forwarding traffic from the evpn network to the real network (through the exit-node host default gw).
in the reverse direction, you need to add routes to your evpn network. This can be...
if yours vms are in the same subnet on each host, you can simply use a vxlan zone. It's create tunnels between hosts (like a fullmesh vpn/wireguard).
vxlan zone is only for flat l2 network.
if your vms are in differents subnets, you can use the evpn zones. (it's vxlan + integrating routing with...
Hi,
can you make a feature request on bugzilla.proxmox.com ?
Currently, the vlan-aware option has been added as users requested it for special setup (vlan tag on top of vxlan, triple tag in qinq,....).
But the sdn is really made to create 1vnet=1network , because extra feature are coming...
I got also freeze/crash, only on nodes with hyperconverged ceph.
I'm using encrypted nvme osd on theses servers.
epyc v3 with levono server (I post info && log previously in this thread).
do you have any log ?
This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.