Meltdown and Spectre Linux Kernel fixes

Discussion in 'Proxmox VE: Installation and configuration' started by martin, Jan 7, 2018.

  1. TwiX

    TwiX Member
    Proxmox Subscriber

    Joined:
    Feb 3, 2015
    Messages:
    144
    Likes Received:
    1
    Hi,

    I don't know what to do right now about these fixes. A little bit afraid of performances matters after upgrading....
     
  2. Sebastian2000

    Sebastian2000 Member

    Joined:
    Oct 31, 2017
    Messages:
    80
    Likes Received:
    1
    The situation is still not clear... In our case, we will just update guest kernel for now that not seem have performance impact but not solve all spectre variant. I will not update HOST for now, waiting to see what we really had to do or what is the better we can do to lower impact performance and increase security.
     
  3. fabian

    fabian Proxmox Staff Member
    Staff Member

    Joined:
    Jan 7, 2016
    Messages:
    3,213
    Likes Received:
    498
    you really need to install the host kernel with KPTI if your CPU is affected - otherwise any unprivileged user / process on your host can dump the whole physical memory and become root. there are no ifs and buts here. if your CPU has PCID, the performance impact should not be too big. if your CPU does not support PCID yet, it is time to get newer HW and take the performance hit until you have it.

    you have Intel to thank for that mess in particular and I am all for blaming them ;) - but it is still you who has to take action to mitigate it, and you really should do it now.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    morph027, janssensm and sommarnatt like this.
  4. Sebastian2000

    Sebastian2000 Member

    Joined:
    Oct 31, 2017
    Messages:
    80
    Likes Received:
    1
    Hello Fabian

    Also if GUEST have kernel updated, HOST it's so vulnerable? I doubt to update yet HOST because for now, only 1 of 3 variant seem be solved on it.
    About PCID, do you really sure that is better to enable it? Online seem proove that's not the case.

    Thanks for your help.
     
  5. sommarnatt

    sommarnatt New Member

    Joined:
    Mar 20, 2014
    Messages:
    22
    Likes Received:
    0
    There's a spectre variant 1 PoC around that lets you read the RAM of the host from a KVM guest. You should at least grab the latest PVE kernel with the fix for that PoC.

    Also, there's microcode available already from intel for spectre, but we still need to wait for the kernel updates as well before it makes any sense.

    There's a github repo with spectre/meltdown info and a script to check if you're vulnerable here. You might wanna read through the source like I did before running it ;)

    https://github.com/speed47/spectre-meltdown-checker

    I don't know where you've read that PCID doesn't really help. Any links to that "proof"?

    Heres' a pretty good write-up of PCID and meltdown:
    http://archive.is/ma8Iw


     
  6. Sebastian2000

    Sebastian2000 Member

    Joined:
    Oct 31, 2017
    Messages:
    80
    Likes Received:
    1
    About PCID, it's the same as PTI? or it's an confusion that I have do? About PTI online seem have put graph : https://blog.online.net/2018/01/03/...e-security-flaw-impacting-arm-intel-hardware/
     
  7. stef1777

    stef1777 Member

    Joined:
    Jan 31, 2010
    Messages:
    178
    Likes Received:
    8
    Online talk of PTI not PCID.

    For PCID, I've not found yet any good bench of fix flaw impact.

    The article by arstechnica talks about this but does not provide very detailed proof of these statements.

    https://arstechnica.com/gadgets/201...e-and-meltdown-patches-will-hurt-performance/

     
  8. sommarnatt

    sommarnatt New Member

    Joined:
    Mar 20, 2014
    Messages:
    22
    Likes Received:
    0
    No, PCID is a CPU feature from 2010 that hasn't really been used until linux kernel 4.14.
    Now with meltdown it's actually useable to counteract some of the performance loss.

    However, I haven't done any benchmarking but if you're going to reboot your Guests anyway, then make sure to add PCID flag in proxmox under Processors/cpu before you shutdown, start them through proxmox. That way you make sure that the guest can make use of PCID in case it helps.

     
  9. fabian

    fabian Proxmox Staff Member
    Staff Member

    Joined:
    Jan 7, 2016
    Messages:
    3,213
    Likes Received:
    498
    you need to upgrade your host kernel to protect your host from meltdown. period.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  10. stef1777

    stef1777 Member

    Joined:
    Jan 31, 2010
    Messages:
    178
    Likes Received:
    8
    Concern Haswell et Broadwell processors (all brand servers).
     
  11. Sebastian2000

    Sebastian2000 Member

    Joined:
    Oct 31, 2017
    Messages:
    80
    Likes Received:
    1
    @fabian
    Ok for kernel, we will try to do it ASAP :-S
    About PCID, I confirm that this last pve-manager, option is present! I can't really test it because it's an test microserver (Atom c2750) that not have this feature.


    pveversion -v
    proxmox-ve: 5.1-35 (running kernel: 4.13.13-4-pve)
    pve-manager: 5.1-42 (running version: 5.1-42/724a6cb3)
    pve-kernel-4.13.4-1-pve: 4.13.4-26
    pve-kernel-4.13.13-4-pve: 4.13.13-35
    libpve-http-server-perl: 2.0-8
    lvm2: 2.02.176-4.1
    corosync: 2.4.2-pve3
    libqb0: 1.0.1-1
    pve-cluster: 5.0-19
    qemu-server: 5.0-19
    pve-firmware: 2.0-3
    libpve-common-perl: 5.0-25
    libpve-guest-common-perl: 2.0-14
    libpve-access-control: 5.0-7
    libpve-storage-perl: 5.0-17
    pve-libspice-server1: 0.12.8-3
    vncterm: 1.5-3
    pve-docs: 5.1-16
    pve-qemu-kvm: 2.9.1-5
    pve-container: 2.0-18
    pve-firewall: 3.0-5
    pve-ha-manager: 2.0-4
    ksm-control-daemon: 1.2-2
    glusterfs-client: 3.8.8-1
    lxc-pve: 2.1.1-2
    lxcfs: 2.0.8-1
    criu: 3.6-2
    novnc-pve: 0.6-4
    smartmontools: 6.5+svn4324-1
    zfsutils-linux: 0.7.3-pve1~bpo9
     
  12. Sebastian2000

    Sebastian2000 Member

    Joined:
    Oct 31, 2017
    Messages:
    80
    Likes Received:
    1
    One more things.. PCID seem working only on kernel 4.14... the last proxmox kernel is 4.13.X... this feature is correctly implement in proxmox kernel?
     
  13. stef1777

    stef1777 Member

    Joined:
    Jan 31, 2010
    Messages:
    178
    Likes Received:
    8
  14. rordonez

    rordonez New Member

    Joined:
    Aug 4, 2010
    Messages:
    12
    Likes Received:
    0
    This is a question :
    Is this the correct way to a update a non subscription server to protect against the Vulns:

    1)Enable test repo adding the following line to /etc/apt/source.list
    deb http://download.proxmox.com/debian/pve stretch pvetest

    2)Issue the following commands on the console:
    apt update
    apt dist-upgrade

    3)Reboot the server

    Could you confirm that all the bagage that comes in the test repo is also safe to install?


    ////////////////////////
    The following NEW packages will be installed:
    pve-kernel-4.13.13-4-pve pve-xtermjs
    The following packages will be upgraded:
    libnvpair1linux libpve-common-perl libpve-guest-common-perl libpve-http-server-perl libpve-storage-perl libuutil1linux libzfs2linux
    libzpool2linux lxc-pve lxcfs proxmox-ve pve-cluster pve-container pve-docs pve-firewall pve-ha-manager pve-kernel-4.13.4-1-pve
    pve-manager pve-qemu-kvm qemu-server spiceterm spl vncterm zfs-initramfs zfsutils-linux
    25 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
    Need to get 120 MB of archives.
    //////////////////////
     
  15. EuroDomenii

    EuroDomenii Member
    Proxmox Subscriber

    Joined:
    Sep 30, 2016
    Messages:
    102
    Likes Received:
    15
    Results of https://github.com/speed47/spectre-meltdown-checker for kernel 4.4.98-3-pve #1 SMP PVE 4.4.98-103 (Mon, 8 Jan 2018 10:15:44 +0100) x86_64 GNU/Linux PROXMOX
    Code:
    ./spectre-meltdown-checker.sh
    Spectre and Meltdown mitigation detection tool v0.29
    
    Checking for vulnerabilities against running kernel Linux 4.4.98-3-pve #1 SMP PVE 4.4.98-103 (Mon, 8 Jan 2018 10:15:44 +0100) x86_64
    CPU is Intel(R) Xeon(R) CPU E3-1275 v5 @ 3.60GHz
    
    CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
    * Checking count of LFENCE opcodes in kernel:  NO
    > STATUS:  VULNERABLE  (only 35 opcodes found, should be >= 70, heuristic to be improved when official patches become available)
    
    CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
    * Mitigation 1
    *   Hardware (CPU microcode) support for mitigation:  NO
    *   Kernel support for IBRS:  NO
    *   IBRS enabled for Kernel space:  NO
    *   IBRS enabled for User space:  NO
    * Mitigation 2
    *   Kernel compiled with retpoline option:  NO
    *   Kernel compiled with a retpoline-aware compiler:  NO
    > STATUS:  VULNERABLE  (IBRS hardware + kernel support OR kernel with retpoline are needed to mitigate the vulnerability)
    
    CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
    * Kernel supports Page Table Isolation (PTI):  YES
    * PTI enabled and active:  YES
    > STATUS:  NOT VULNERABLE  (PTI mitigates the vulnerability)
    
    A false sense of security is worse than no security at all, see --disclaimer
    
    CENTOS 6 is better at the moment, but it seems that Ubuntu new kernel for spectre will be available soon!
    Code:
    ./spectre-meltdown-checker.sh
    Spectre and Meltdown mitigation detection tool v0.29
    
    Checking for vulnerabilities against running kernel Linux 2.6.32-696.18.7.el6.x86_64 #1 SMP Thu Jan 4 17:31:22 UTC 2018 x86_64
    CPU is Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
    
    CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
    * Checking count of LFENCE opcodes in kernel:  YES
    > STATUS:  NOT VULNERABLE  (84 opcodes found, which is >= 70, heuristic to be improved when official patches become available)
    
    CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
    * Mitigation 1
    *   Hardware (CPU microcode) support for mitigation:  NO
    *   Kernel support for IBRS:  YES
    *   IBRS enabled for Kernel space:  NO
    *   IBRS enabled for User space:  NO
    * Mitigation 2
    *   Kernel compiled with retpoline option:  NO
    *   Kernel compiled with a retpoline-aware compiler:  NO
    > STATUS:  VULNERABLE  (IBRS hardware + kernel support OR kernel with retpoline are needed to mitigate the vulnerability)
    
    CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
    * Kernel supports Page Table Isolation (PTI):  YES
    * PTI enabled and active:  YES
    > STATUS:  NOT VULNERABLE  (PTI mitigates the vulnerability)
    
    A false sense of security is worse than no security at all, see --disclaime
    
     
    #95 EuroDomenii, Jan 14, 2018
    Last edited: Jan 14, 2018
  16. sommarnatt

    sommarnatt New Member

    Joined:
    Mar 20, 2014
    Messages:
    22
    Likes Received:
    0
    I'm not affiliated with proxmox, but pulling stuff in from testing repo to a production server should be avoided.

    We might have to wait for upstream fixes which should be available on monday then wait a day or so for Proxmox to pull it in and test.

    https://insights.ubuntu.com/2018/01/12/meltdown-and-spectre-status-update/


     
  17. TwiX

    TwiX Member
    Proxmox Subscriber

    Joined:
    Feb 3, 2015
    Messages:
    144
    Likes Received:
    1
    Hi everyone,

    Can you explain why PCID have to be checked for all those different from 'host' cpu type ?
    Is that true that if we upgrade Bios and Intel microcode, PTI is not needed ?

    Thanks in advanced !
     
  18. fabian

    fabian Proxmox Staff Member
    Staff Member

    Joined:
    Jan 7, 2016
    Messages:
    3,213
    Likes Received:
    498
    if you select 'host', the CPU features of your physical CPU will be passed through to the guest. no need to check manually in that case. if you select kvm64 (or another CPU type which does not have the 'pcid' flag by default), you need to check on all hosts where you want to run the VM - because if your actual CPU does not support PCID, you don't want to start a VM with a vCPU that pretends to support it.

    the BIOS/microcode updates are related to Spectre, not Meltdown. PTI is the fix for Meltdown. you absolutely need PTI on Intel CPUs, even if you install a microcode update that enables IBRS and IBPB for (potential) Spectre mitigation.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    EuroDomenii likes this.
  19. TwiX

    TwiX Member
    Proxmox Subscriber

    Joined:
    Feb 3, 2015
    Messages:
    144
    Likes Received:
    1
    Thanks, seems that Dell update bios is quite unstable by now :(
    And Stretch Intel microcode update is not available, what we have is an old version 2017 May...
     
  20. fabian

    fabian Proxmox Staff Member
    Staff Member

    Joined:
    Jan 7, 2016
    Messages:
    3,213
    Likes Received:
    498
    the microcode update does not help unless a patched kernel (needed for host, VMs and containers) and patched qemu (needed for VMs) is available.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    chrone likes this.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice