Recent content by CodeBreaker

  1. C

    SDN - QinQ and MTU?

    I'm trying to configure SDN QinQ in my Proxomx cluster and I am wondering is my configuration correct and is there anything I can do to increase performance. My goal is to have VMs and containers on a default 1500 MTU so I did the following: I have eno1 & eno2 bonded (LACP (802.3ad)) to bond0...
  2. C

    To separate ceph cluster or not?

    Thanks for the recommendation. Hardware is 3x Dell R620 (4x800GB SAS SSD) & 3x Dell R320 (4x 6TB SAS). The HDD are used for documents, media files, dumps from databases and is used for computers around the office (SMB).
  3. C

    To separate ceph cluster or not?

    The HDD nodes have 3.5" bays and SSD nodes have 2.5" bays so that is a no-go on distributing disks across all 6 nodes. Taking that into consideration how should I distribute Monitors and Managers?
  4. C

    To separate ceph cluster or not?

    I have a 6 node hyper-converged cluster where 3 nodes handle compute with SSD OSDs and other 3 nodes with HDD OSDs (they handle no compute). In the years since the first deployment a few patterns emerged: HDD pool is used exclusively for CephFS SSD pool is used exclusively for RDB (HA VM...
  5. C

    [SOLVED] SDN - weird issues

    Additional information for those that might need it. Because I'm running SDN traffic on its own additional linux vlan (defined in proxmox) an additional 4 bytes is added to the ethernet frame. This exceeds the max MTU defined on the switch. Basically it's the same as running QinQ Zones in...
  6. C

    [SOLVED] SDN - weird issues

    It was the MTU... I had everything on auto and it didn't occur to me that having SDN over VLAN could be the issue. Reduced the MTU on the VMs from default 1500 to 1496 and it works like a charm. Now I need to figure out how to reduce MTU for all VM at once...
  7. C

    [SOLVED] SDN - weird issues

    I'm stumped. I'm migrating my setup to SDN with VLAN zone over vmbr4 with bridge port bond0.2920 VLAN. I didn't touch network config for VMs and CTs except changing interface. At first glanced everything seemed to work. Pinging working correctly, internet working fine etc. After a while I've...
  8. C

    SDN - what do I need?

    My bad, I didn't ask clearly. What I meant was can I have those features (ex, DNS registration) when using just raw VLANs to segmenting network instead of SDN, ie. using other software solutions, proxmox plugins or something? With SDN I could add PowerDNS authoritative server where (from what I...
  9. C

    SDN - what do I need?

    Thanks. I would still need a router (i was thinking VyOS hosted on proxmox) to enable internet access (since I'm moving Mikrotik outside of server network) and to route packets to server network from user network behind Mikrotik router. I was intrigued with other features of SDN, most...
  10. C

    SDN - what do I need?

    This is my current infrastructure: Defined on the Mikrotik are a couple of VLANs for ipmi, proxmox, ceph, and for user networks going across the Mikrotik switch. I have 3 (W)LAN user networks. When a new project is started I need to define a new isolated network and vlan in router and add it...
  11. C

    Ceph replacing drive - no notification of data loss?

    Thanks. Will do that. But, isn't manager suppose to know that there are missing PGs? If I removed disks that are housing all (two) copies of the PG then there is no way to recover them and the system should have an error somewhere?
  12. C

    Ceph replacing drive - no notification of data loss?

    I've removed from different servers. This is what ceph pg dump | grep -v active+clean shows: version 68465 stamp 2022-07-25T22:14:46.049009+0200 last_osdmap_epoch 0 last_pg_scan 0 PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG...
  13. C

    Ceph replacing drive - no notification of data loss?

    I'm replacing HDD's in cluster with a higher capacity drives. Some not that important video data is on a size 2 min 1 CephFS pool (yes, I know, that can lead to data loss). The process that I was doing was: 1. set global flag noout 2. stop and out an OSD then destroy it (from GUI) 3. pull out...
  14. C

    [SOLVED] cephfs storage subdir not working

    I've disabled storage, manually unmount storage on all the nodes, then enabled the cephfs storage back and then it worked.
  15. C

    [SOLVED] cephfs storage subdir not working

    @dcsapak I did not. I've disabled then enabled cephfs storage on datacenter->storage page. I assumed that when doing that the storage gets unmounted. My mistake. Thank you!