Hello everyone
I am new to Proxmox, coming from a VMware / Hyper-V background.
During evaluation tests I came accross the following issue:
When running a certain workload in a LXC container I get degraded CPU performance compared against a QEMU / Hyper-V VM.
After some research / benchmarking I discovered a similarity to the observed behaviour running the workload as when running a simple 7z benchmark.
So for simplicity I have attached the 7z benchmark results below.
I am running the following configuration:
Virtual Environment 7.1-8
HPE DL325 with AMD EPYC 7402P 24 core / SMT disabled
DDR4 2933Mhz memory
NVMe Storage single disks
LXC / VMs running Ubuntu 21.10
All components are updated on a software level, FW updated in Oktober to the latest versions.
LXC: 6CPU, 12GB RAM
time 7z b
7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=C,Utf16=off,HugeFiles=on,64 bits,24 CPUs AMD EPYC 7402P 24-Core Processor (830F10),ASM,AES-NI)
AMD EPYC 7402P 24-Core Processor (830F10)
CPU Freq: - - - - - 256000000 - - 2048000000
RAM size: 12288 MB, # CPU hardware threads: 24
RAM usage: 5295 MB, # Benchmark threads: 24
Compressing | Decompressing
Dict Speed Usage R/U Rating | Speed Usage R/U Rating
KiB/s % MIPS MIPS | KiB/s % MIPS MIPS
22: 22971 527 4240 22347 | 226605 540 3580 19323
23: 22256 537 4221 22676 | 221023 535 3573 19124
24: 20645 536 4142 22198 | 217017 535 3560 19048
25: 19687 530 4239 22478 | 209757 530 3522 18668
---------------------------------- | ------------------------------
Avr: 533 4210 22425 | 535 3559 19041
Tot: 534 3885 20733
real 2m6.245s
user 10m47.712s
sys 0m7.973s
QEMU VM (same host): 6vCPU, 12GB RAM
time 7z b
7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=C.UTF-8,Utf16=on,HugeFiles=on,64 bits,6 CPUs AMD EPYC 7402P 24-Core Processor (830F10),ASM,AES-NI)
AMD EPYC 7402P 24-Core Processor (830F10)
CPU Freq: - - - - - - - - -
RAM size: 12123 MB, # CPU hardware threads: 6
RAM usage: 1323 MB, # Benchmark threads: 6
Compressing | Decompressing
Dict Speed Usage R/U Rating | Speed Usage R/U Rating
KiB/s % MIPS MIPS | KiB/s % MIPS MIPS
22: 22188 485 4451 21585 | 215319 501 3664 18363
23: 18706 433 4397 19060 | 184480 435 3666 15963
24: 15916 391 4381 17113 | 217377 514 3713 19080
25: 19676 510 4407 22465 | 215293 518 3698 19160
---------------------------------- | ------------------------------
Avr: 455 4409 20056 | 492 3685 18141
Tot: 473 4047 19099
real 0m33.619s
user 2m34.802s
sys 0m2.959s
Windows Hyper-V VM: 6vCPU, 12GB RAM
time 7z b
7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=C.UTF-8,Utf16=on,HugeFiles=on,64 bits,6 CPUs AMD EPYC 7402P 24-Core Processor (830F10),ASM,AES-NI)
AMD EPYC 7402P 24-Core Processor (830F10)
CPU Freq: - - - - - - - - -
RAM size: 11972 MB, # CPU hardware threads: 6
RAM usage: 1323 MB, # Benchmark threads: 6
Compressing | Decompressing
Dict Speed Usage R/U Rating | Speed Usage R/U Rating
KiB/s % MIPS MIPS | KiB/s % MIPS MIPS
22: 23407 511 4460 22771 | 155321 369 3594 13246
23: 22666 531 4349 23094 | 222406 525 3665 19245
24: 20489 508 4340 22030 | 211873 511 3637 18597
25: 20172 498 4627 23033 | 207212 512 3600 18441
---------------------------------- | ------------------------------
Avr: 512 4444 22732 | 479 3624 17382
Tot: 496 4034 20057
real 0m33.796s
user 2m35.747s
sys 0m2.659s
Eventhough the performance results show better results on the LXC, the real time passed was close to 4 times slower. The LXC also sees the whole 24 cores of the host, it's as if the LXC tries to use the other 18 cores of the host CPU wasting time while it never receives the cycles.
This behaviour could be reproduced on a different (older) Intel host also running 7.1-8.
The initialization of the workload the LXC is supposed to run does a synchronization tasks at the beginning, which takes a lot longer on the LXC compared to the VMs, something like 1h vs 5mins. As this is a task which resides heavliy on the CPU I am looking to improve the speeds on the LXC for initial startup.
Any help is appreciated.
I am new to Proxmox, coming from a VMware / Hyper-V background.
During evaluation tests I came accross the following issue:
When running a certain workload in a LXC container I get degraded CPU performance compared against a QEMU / Hyper-V VM.
After some research / benchmarking I discovered a similarity to the observed behaviour running the workload as when running a simple 7z benchmark.
So for simplicity I have attached the 7z benchmark results below.
I am running the following configuration:
Virtual Environment 7.1-8
HPE DL325 with AMD EPYC 7402P 24 core / SMT disabled
DDR4 2933Mhz memory
NVMe Storage single disks
LXC / VMs running Ubuntu 21.10
All components are updated on a software level, FW updated in Oktober to the latest versions.
LXC: 6CPU, 12GB RAM
time 7z b
7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=C,Utf16=off,HugeFiles=on,64 bits,24 CPUs AMD EPYC 7402P 24-Core Processor (830F10),ASM,AES-NI)
AMD EPYC 7402P 24-Core Processor (830F10)
CPU Freq: - - - - - 256000000 - - 2048000000
RAM size: 12288 MB, # CPU hardware threads: 24
RAM usage: 5295 MB, # Benchmark threads: 24
Compressing | Decompressing
Dict Speed Usage R/U Rating | Speed Usage R/U Rating
KiB/s % MIPS MIPS | KiB/s % MIPS MIPS
22: 22971 527 4240 22347 | 226605 540 3580 19323
23: 22256 537 4221 22676 | 221023 535 3573 19124
24: 20645 536 4142 22198 | 217017 535 3560 19048
25: 19687 530 4239 22478 | 209757 530 3522 18668
---------------------------------- | ------------------------------
Avr: 533 4210 22425 | 535 3559 19041
Tot: 534 3885 20733
real 2m6.245s
user 10m47.712s
sys 0m7.973s
QEMU VM (same host): 6vCPU, 12GB RAM
time 7z b
7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=C.UTF-8,Utf16=on,HugeFiles=on,64 bits,6 CPUs AMD EPYC 7402P 24-Core Processor (830F10),ASM,AES-NI)
AMD EPYC 7402P 24-Core Processor (830F10)
CPU Freq: - - - - - - - - -
RAM size: 12123 MB, # CPU hardware threads: 6
RAM usage: 1323 MB, # Benchmark threads: 6
Compressing | Decompressing
Dict Speed Usage R/U Rating | Speed Usage R/U Rating
KiB/s % MIPS MIPS | KiB/s % MIPS MIPS
22: 22188 485 4451 21585 | 215319 501 3664 18363
23: 18706 433 4397 19060 | 184480 435 3666 15963
24: 15916 391 4381 17113 | 217377 514 3713 19080
25: 19676 510 4407 22465 | 215293 518 3698 19160
---------------------------------- | ------------------------------
Avr: 455 4409 20056 | 492 3685 18141
Tot: 473 4047 19099
real 0m33.619s
user 2m34.802s
sys 0m2.959s
Windows Hyper-V VM: 6vCPU, 12GB RAM
time 7z b
7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=C.UTF-8,Utf16=on,HugeFiles=on,64 bits,6 CPUs AMD EPYC 7402P 24-Core Processor (830F10),ASM,AES-NI)
AMD EPYC 7402P 24-Core Processor (830F10)
CPU Freq: - - - - - - - - -
RAM size: 11972 MB, # CPU hardware threads: 6
RAM usage: 1323 MB, # Benchmark threads: 6
Compressing | Decompressing
Dict Speed Usage R/U Rating | Speed Usage R/U Rating
KiB/s % MIPS MIPS | KiB/s % MIPS MIPS
22: 23407 511 4460 22771 | 155321 369 3594 13246
23: 22666 531 4349 23094 | 222406 525 3665 19245
24: 20489 508 4340 22030 | 211873 511 3637 18597
25: 20172 498 4627 23033 | 207212 512 3600 18441
---------------------------------- | ------------------------------
Avr: 512 4444 22732 | 479 3624 17382
Tot: 496 4034 20057
real 0m33.796s
user 2m35.747s
sys 0m2.659s
Eventhough the performance results show better results on the LXC, the real time passed was close to 4 times slower. The LXC also sees the whole 24 cores of the host, it's as if the LXC tries to use the other 18 cores of the host CPU wasting time while it never receives the cycles.
This behaviour could be reproduced on a different (older) Intel host also running 7.1-8.
The initialization of the workload the LXC is supposed to run does a synchronization tasks at the beginning, which takes a lot longer on the LXC compared to the VMs, something like 1h vs 5mins. As this is a task which resides heavliy on the CPU I am looking to improve the speeds on the LXC for initial startup.
Any help is appreciated.