I have Proxmox 8
Currently, version 6.8.12-15-pve
The tcp_bbr module is present, but it's not loaded.
The cluster has 3 nodes and uses Ceph. The bond has 10 Gbps network connections each.
Currently, I see that net.ipv4.tcp_congestion_control = cubic is being used.
But to increase throughput and performance, I should switch the nodes to net.ipv4.tcp_congestion_control = bbr
Has anyone encountered this and done this, and is it a good idea? Is there a manual for adding modules to startup and using them?
I'd also like to consult on tuning /etc/sysctl.conf for all my nodes.
Here's the config I'd like to make.
But I have some concerns.
net.core.default_qdisc = fq_codel
net.ipv4.tcp_congestion_control = bbr
net.ipv4.tcp_ecn = 1
# ============================================================
net.ipv4.tcp_sack = 1
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_moderate_rcvbuf = 1
net.ipv4.tcp_mtu_probing = 1
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_rfc1337 = 1
net.ipv4.tcp_fastopen = 3
# ============================================================
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.all.rp_filter = 1
net.ipv4.icmp_echo_ignore_all = 0
# ============================================================
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.lo.disable_ipv6 = 0
# ============================================================
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-arptables = 1
# ============================================================
net.ipv4.tcp_low_latency = 1
# ============================================================
net.core.netdev_budget = 600
net.core.netdev_budget_usecs = 8000
Currently, version 6.8.12-15-pve
The tcp_bbr module is present, but it's not loaded.
The cluster has 3 nodes and uses Ceph. The bond has 10 Gbps network connections each.
Currently, I see that net.ipv4.tcp_congestion_control = cubic is being used.
But to increase throughput and performance, I should switch the nodes to net.ipv4.tcp_congestion_control = bbr
Has anyone encountered this and done this, and is it a good idea? Is there a manual for adding modules to startup and using them?
I'd also like to consult on tuning /etc/sysctl.conf for all my nodes.
Here's the config I'd like to make.
But I have some concerns.
net.core.default_qdisc = fq_codel
net.ipv4.tcp_congestion_control = bbr
net.ipv4.tcp_ecn = 1
# ============================================================
net.ipv4.tcp_sack = 1
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_moderate_rcvbuf = 1
net.ipv4.tcp_mtu_probing = 1
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_rfc1337 = 1
net.ipv4.tcp_fastopen = 3
# ============================================================
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.all.rp_filter = 1
net.ipv4.icmp_echo_ignore_all = 0
# ============================================================
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.lo.disable_ipv6 = 0
# ============================================================
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-arptables = 1
# ============================================================
net.ipv4.tcp_low_latency = 1
# ============================================================
net.core.netdev_budget = 600
net.core.netdev_budget_usecs = 8000