Meltdown and Spectre Linux Kernel fixes

Hi,

I don't know what to do right now about these fixes. A little bit afraid of performances matters after upgrading....
 
Hi,

I don't know what to do right now about these fixes. A little bit afraid of performances matters after upgrading....
The situation is still not clear... In our case, we will just update guest kernel for now that not seem have performance impact but not solve all spectre variant. I will not update HOST for now, waiting to see what we really had to do or what is the better we can do to lower impact performance and increase security.
 
The situation is still not clear... In our case, we will just update guest kernel for now that not seem have performance impact but not solve all spectre variant. I will not update HOST for now, waiting to see what we really had to do or what is the better we can do to lower impact performance and increase security.

you really need to install the host kernel with KPTI if your CPU is affected - otherwise any unprivileged user / process on your host can dump the whole physical memory and become root. there are no ifs and buts here. if your CPU has PCID, the performance impact should not be too big. if your CPU does not support PCID yet, it is time to get newer HW and take the performance hit until you have it.

you have Intel to thank for that mess in particular and I am all for blaming them ;) - but it is still you who has to take action to mitigate it, and you really should do it now.
 
you really need to install the host kernel with KPTI if your CPU is affected - otherwise any unprivileged user / process on your host can dump the whole physical memory and become root. there are no ifs and buts here. if your CPU has PCID, the performance impact should not be too big. if your CPU does not support PCID yet, it is time to get newer HW and take the performance hit until you have it.

you have Intel to thank for that mess in particular and I am all for blaming them ;) - but it is still you who has to take action to mitigate it, and you really should do it now.

Hello Fabian

Also if GUEST have kernel updated, HOST it's so vulnerable? I doubt to update yet HOST because for now, only 1 of 3 variant seem be solved on it.
About PCID, do you really sure that is better to enable it? Online seem proove that's not the case.

Thanks for your help.
 
There's a spectre variant 1 PoC around that lets you read the RAM of the host from a KVM guest. You should at least grab the latest PVE kernel with the fix for that PoC.

Also, there's microcode available already from intel for spectre, but we still need to wait for the kernel updates as well before it makes any sense.

There's a github repo with spectre/meltdown info and a script to check if you're vulnerable here. You might wanna read through the source like I did before running it ;)

https://github.com/speed47/spectre-meltdown-checker

I don't know where you've read that PCID doesn't really help. Any links to that "proof"?

Heres' a pretty good write-up of PCID and meltdown:
http://archive.is/ma8Iw


Hello Fabian

Also if GUEST have kernel updated, HOST it's so vulnerable? I doubt to update yet HOST because for now, only 1 of 3 variant seem be solved on it.
About PCID, do you really sure that is better to enable it? Online seem proove that's not the case.

Thanks for your help.
 
There's a spectre variant 1 PoC around that lets you read the RAM of the host from a KVM guest. You should at least grab the latest PVE kernel with the fix for that PoC.

Also, there's microcode available already from intel for spectre, but we still need to wait for the kernel updates as well before it makes any sense.

There's a github repo with spectre/meltdown info and a script to check if you're vulnerable here. You might wanna read through the source like I did before running it ;)

https://github.com/speed47/spectre-meltdown-checker

I don't know where you've read that PCID doesn't really help. Any links to that "proof"?

Heres' a pretty good write-up of PCID and meltdown:
http://archive.is/ma8Iw

About PCID, it's the same as PTI? or it's an confusion that I have do? About PTI online seem have put graph : https://blog.online.net/2018/01/03/...e-security-flaw-impacting-arm-intel-hardware/
 
Hello Fabian
About PCID, do you really sure that is better to enable it? Online seem proove that's not the case.

Online talk of PTI not PCID.

For PCID, I've not found yet any good bench of fix flaw impact.

The article by arstechnica talks about this but does not provide very detailed proof of these statements.

https://arstechnica.com/gadgets/201...e-and-meltdown-patches-will-hurt-performance/

This makes a difference. In a synthetic benchmark that tests only the cost of switching into the kernel and back again, an unpatched Linux system can switch about 5.2 million times a second. Dual page tables slashes that to 2.2 million a second; dual page tables with PCID gets it back up to 3 million.
 
No, PCID is a CPU feature from 2010 that hasn't really been used until linux kernel 4.14.
Now with meltdown it's actually useable to counteract some of the performance loss.

However, I haven't done any benchmarking but if you're going to reboot your Guests anyway, then make sure to add PCID flag in proxmox under Processors/cpu before you shutdown, start them through proxmox. That way you make sure that the guest can make use of PCID in case it helps.

About PCID, it's the same as PTI? or it's an confusion that I have do? About PTI online seem have put graph : https://blog.online.net/2018/01/03/...e-security-flaw-impacting-arm-intel-hardware/
 
Hello Fabian

Also if GUEST have kernel updated, HOST it's so vulnerable? I doubt to update yet HOST because for now, only 1 of 3 variant seem be solved on it.
About PCID, do you really sure that is better to enable it? Online seem proove that's not the case.

Thanks for your help.

you need to upgrade your host kernel to protect your host from meltdown. period.
 
@fabian
Ok for kernel, we will try to do it ASAP :-S
About PCID, I confirm that this last pve-manager, option is present! I can't really test it because it's an test microserver (Atom c2750) that not have this feature.


pveversion -v
proxmox-ve: 5.1-35 (running kernel: 4.13.13-4-pve)
pve-manager: 5.1-42 (running version: 5.1-42/724a6cb3)
pve-kernel-4.13.4-1-pve: 4.13.4-26
pve-kernel-4.13.13-4-pve: 4.13.13-35
libpve-http-server-perl: 2.0-8
lvm2: 2.02.176-4.1
corosync: 2.4.2-pve3
libqb0: 1.0.1-1
pve-cluster: 5.0-19
qemu-server: 5.0-19
pve-firmware: 2.0-3
libpve-common-perl: 5.0-25
libpve-guest-common-perl: 2.0-14
libpve-access-control: 5.0-7
libpve-storage-perl: 5.0-17
pve-libspice-server1: 0.12.8-3
vncterm: 1.5-3
pve-docs: 5.1-16
pve-qemu-kvm: 2.9.1-5
pve-container: 2.0-18
pve-firewall: 3.0-5
pve-ha-manager: 2.0-4
ksm-control-daemon: 1.2-2
glusterfs-client: 3.8.8-1
lxc-pve: 2.1.1-2
lxcfs: 2.0.8-1
criu: 3.6-2
novnc-pve: 0.6-4
smartmontools: 6.5+svn4324-1
zfsutils-linux: 0.7.3-pve1~bpo9
 
One more things.. PCID seem working only on kernel 4.14... the last proxmox kernel is 4.13.X... this feature is correctly implement in proxmox kernel?
 
This is a question :
Is this the correct way to a update a non subscription server to protect against the Vulns:

1)Enable test repo adding the following line to /etc/apt/source.list
deb http://download.proxmox.com/debian/pve stretch pvetest

2)Issue the following commands on the console:
apt update
apt dist-upgrade

3)Reboot the server

Could you confirm that all the bagage that comes in the test repo is also safe to install?


////////////////////////
The following NEW packages will be installed:
pve-kernel-4.13.13-4-pve pve-xtermjs
The following packages will be upgraded:
libnvpair1linux libpve-common-perl libpve-guest-common-perl libpve-http-server-perl libpve-storage-perl libuutil1linux libzfs2linux
libzpool2linux lxc-pve lxcfs proxmox-ve pve-cluster pve-container pve-docs pve-firewall pve-ha-manager pve-kernel-4.13.4-1-pve
pve-manager pve-qemu-kvm qemu-server spiceterm spl vncterm zfs-initramfs zfsutils-linux
25 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 120 MB of archives.
//////////////////////
 
Results of https://github.com/speed47/spectre-meltdown-checker for kernel 4.4.98-3-pve #1 SMP PVE 4.4.98-103 (Mon, 8 Jan 2018 10:15:44 +0100) x86_64 GNU/Linux PROXMOX
Code:
./spectre-meltdown-checker.sh
Spectre and Meltdown mitigation detection tool v0.29

Checking for vulnerabilities against running kernel Linux 4.4.98-3-pve #1 SMP PVE 4.4.98-103 (Mon, 8 Jan 2018 10:15:44 +0100) x86_64
CPU is Intel(R) Xeon(R) CPU E3-1275 v5 @ 3.60GHz

CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
* Checking count of LFENCE opcodes in kernel:  NO
> STATUS:  VULNERABLE  (only 35 opcodes found, should be >= 70, heuristic to be improved when official patches become available)

CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigation 1
*   Hardware (CPU microcode) support for mitigation:  NO
*   Kernel support for IBRS:  NO
*   IBRS enabled for Kernel space:  NO
*   IBRS enabled for User space:  NO
* Mitigation 2
*   Kernel compiled with retpoline option:  NO
*   Kernel compiled with a retpoline-aware compiler:  NO
> STATUS:  VULNERABLE  (IBRS hardware + kernel support OR kernel with retpoline are needed to mitigate the vulnerability)

CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
* Kernel supports Page Table Isolation (PTI):  YES
* PTI enabled and active:  YES
> STATUS:  NOT VULNERABLE  (PTI mitigates the vulnerability)

A false sense of security is worse than no security at all, see --disclaimer

CENTOS 6 is better at the moment, but it seems that Ubuntu new kernel for spectre will be available soon!
Code:
./spectre-meltdown-checker.sh
Spectre and Meltdown mitigation detection tool v0.29

Checking for vulnerabilities against running kernel Linux 2.6.32-696.18.7.el6.x86_64 #1 SMP Thu Jan 4 17:31:22 UTC 2018 x86_64
CPU is Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz

CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
* Checking count of LFENCE opcodes in kernel:  YES
> STATUS:  NOT VULNERABLE  (84 opcodes found, which is >= 70, heuristic to be improved when official patches become available)

CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigation 1
*   Hardware (CPU microcode) support for mitigation:  NO
*   Kernel support for IBRS:  YES
*   IBRS enabled for Kernel space:  NO
*   IBRS enabled for User space:  NO
* Mitigation 2
*   Kernel compiled with retpoline option:  NO
*   Kernel compiled with a retpoline-aware compiler:  NO
> STATUS:  VULNERABLE  (IBRS hardware + kernel support OR kernel with retpoline are needed to mitigate the vulnerability)

CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
* Kernel supports Page Table Isolation (PTI):  YES
* PTI enabled and active:  YES
> STATUS:  NOT VULNERABLE  (PTI mitigates the vulnerability)

A false sense of security is worse than no security at all, see --disclaime
 
Last edited:
I'm not affiliated with proxmox, but pulling stuff in from testing repo to a production server should be avoided.

We might have to wait for upstream fixes which should be available on monday then wait a day or so for Proxmox to pull it in and test.

https://insights.ubuntu.com/2018/01/12/meltdown-and-spectre-status-update/
Our focus has now shifted to the mitigation of CVE-2017-5753 and CVE-2017-5715 (aka Spectre / Variants 1 & 2). Microcode has been released for Intel processors (see USN-3531-1). Kernel updates will begin with releasing v4.13 for Artful 17.10 on Monday, January 15, 2018, with 16.04 to follow shortly.



This is a question :
Is this the correct way to a update a non subscription server to protect against the Vulns:

1)Enable test repo adding the following line to /etc/apt/source.list
deb http://download.proxmox.com/debian/pve stretch pvetest

2)Issue the following commands on the console:
apt update
apt dist-upgrade

3)Reboot the server

Could you confirm that all the bagage that comes in the test repo is also safe to install?


////////////////////////
The following NEW packages will be installed:
pve-kernel-4.13.13-4-pve pve-xtermjs
The following packages will be upgraded:
libnvpair1linux libpve-common-perl libpve-guest-common-perl libpve-http-server-perl libpve-storage-perl libuutil1linux libzfs2linux
libzpool2linux lxc-pve lxcfs proxmox-ve pve-cluster pve-container pve-docs pve-firewall pve-ha-manager pve-kernel-4.13.4-1-pve
pve-manager pve-qemu-kvm qemu-server spiceterm spl vncterm zfs-initramfs zfsutils-linux
25 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 120 MB of archives.
//////////////////////
 
Hi everyone,

Can you explain why PCID have to be checked for all those different from 'host' cpu type ?
Is that true that if we upgrade Bios and Intel microcode, PTI is not needed ?

Thanks in advanced !
 
Hi everyone,

Can you explain why PCID have to be checked for all those different from 'host' cpu type ?

if you select 'host', the CPU features of your physical CPU will be passed through to the guest. no need to check manually in that case. if you select kvm64 (or another CPU type which does not have the 'pcid' flag by default), you need to check on all hosts where you want to run the VM - because if your actual CPU does not support PCID, you don't want to start a VM with a vCPU that pretends to support it.

Is that true that if we upgrade Bios and Intel microcode, PTI is not needed ?

Thanks in advanced !

the BIOS/microcode updates are related to Spectre, not Meltdown. PTI is the fix for Meltdown. you absolutely need PTI on Intel CPUs, even if you install a microcode update that enables IBRS and IBPB for (potential) Spectre mitigation.
 
  • Like
Reactions: EuroDomenii
Thanks, seems that Dell update bios is quite unstable by now :(
And Stretch Intel microcode update is not available, what we have is an old version 2017 May...
 
Thanks, seems that Dell update bios is quite unstable by now :(
And Stretch Intel microcode update is not available, what we have is an old version 2017 May...

the microcode update does not help unless a patched kernel (needed for host, VMs and containers) and patched qemu (needed for VMs) is available.
 
  • Like
Reactions: chrone

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!