Proxmox Automatic Updates

[Dexter23]

Hi boys

I want to say if is possible to have automatic updates on proxmox VE with repository no subscription

Thanks

 

[Moayad]

Hi,

Proxmox VE is like Debian distribution, which means you can write a bash script to do the automatic updates or simple cron job with pveupdate and pveupgrade

/usr/bin/pveupdate
/usr/bin/pveupgrade

 

[Dexter23]

Hi

pveupdate and pveupgrade it's the same thing to give apt update and apt upgrade?

 

[Dunuin]

Hi

pveupdate and pveupgrade it's the same thing to give apt update and apt upgrade?

Not sure but you should never use apt upgrade. But a apt update && apt dist-upgrade -y or apt update && apt full-upgrade -y should be fine.
Debian also got the unattended-upgrades package that I use in all my Debian VMs to just autoupgrade security fixes, but not sure if that actually will use apt upgrade or apt dist-upgrade. Its also not that easy to automate PVE host backups, so I still do my PVE/PBS host upgrades manually on a weekly basis.

 

[Dexter23]

ok if there a kernel updates and is required to reboot the node for activate the new kernel, is there a method to automatic shutdown all the VM is "running" and reboot the node in automatic?

 

[Dunuin]

I guess you could write a oneliner that will do this. For the reboot and VM shutdown you just need to run reboot. So you could pipe the output of apt update && apt full-upgrade -y into something like grep to filter out a line that suggests you to reboot your host and if that line is found you could use a if to optionally add a reboot.

But again, I don't think such automatic upgrades should be used on a PVE host.

 

[ArrowOfTruth]

Not sure but you should never use apt upgrade. But a apt update && apt dist-upgrade -y or apt update && apt full-upgrade -y should be fine.
Debian also got the unattended-upgrades package that I use in all my Debian VMs to just autoupgrade security fixes, but not sure if that actually will use apt upgrade or apt dist-upgrade. Its also not that easy to automate PVE host backups, so I still do my PVE/PBS host upgrades manually on a weekly basis.

Sorry to revive an old thread, but why never apt upgrade? I typically run apt-get update -y && apt-get upgrade -y to keep things up to date.

 

[spirit]

Sorry to revive an old thread, but why never apt upgrade? I typically run apt-get update -y && apt-get upgrade -y to keep things up to date.

don't use apt-get upgrade will proxmox ! use apt dist-upgrade.

The difference between both is that "upgrade" only upgrade current installed packages. But if a package (like qemu), need a new dependencie, "upgrade" will not installed it. (so it'll break your proxmox installation).

This is working fine with a "stable" distro like debian, because you never have newer packages in the debian repo until the next debian major version.

Proxmox is more dynamic, some new packages can be needed between minor versions.

Using "dist-upgrade" , upgrade current packages + install new dependencies packages if needed.

 

[ArrowOfTruth]

don't use apt-get upgrade will proxmox ! use apt dist-upgrade.

The difference between both is that "upgrade" only upgrade current installed packages. But if a package (like qemu), need a new dependencie, "upgrade" will not installed it. (so it'll break your proxmox installation).

This is working fine with a "stable" distro like debian, because you never have newer packages in the debian repo until the next debian major version.

Proxmox is more dynamic, some new packages can be needed between minor versions.

Using "dist-upgrade" , upgrade current packages + install new dependencies packages if needed.

Thanks for the info, that makes complete sense. Sounds like dist-upgrade and full-upgrade to the same thing, according to this comment below, but I'll use dist-upgrade since that seems to be the "smartest".

https://askubuntu.com/a/1316448

 

[cjdnad]

Hi,

I have started using unattended-upgrades but only my lxc containers.
Am I safe using this method to update/upgrade them automatically?

There is also an email option in the file
/etc/apt/apt.conf.d/50unattended-upgrades

but they keep going to "/var/mail/root": 10 messages 10 new

with this info from mail:
X-Original-To: root@debian.localdomain
Date: Thu, 14 Jul 2022 17:30:05 +0000 (UTC)
From: MAILER-DAEMON@debian.localdomain (Mail Delivery System)
Subject: Undelivered Mail Returned to Sender

I am stuck on how to set up my email server to go to my personal email for the update notifications. Installed mailx but from there I'm stuck.



Hope someone can chime in soon to assist me.

Thanks in advance

fyi I uncomment these lines on the config;

"origin=Debian,codename=${distro_codename}-updates";
"origin=Debian,codename=${distro_codename}-proposed-updates";
"origin=Debian,codename=${distro_codename},label=Debian";
"origin=Debian,codename=${distro_codename},label=Debian-Security";

​

 

[Dunuin]

Here I always install unattended-upgrades for all VMs and LXCs but not for the PVE host itself and that worked perfectly fine so far. But I wouldn't use it for the PVE host, as you are more likely to break something and you probably don't have host backups that are super up-to-date to restore a PVE installation from right before the upgrade. Also keep in mind that you can't downgrade a PVE installation once upgraded.
And for my VMs/LXCs I only want to auto upgrade security updates and not regular updates. So security holes get closed fast but no updates, that add new features and therefore are more likely to break something, will be auto installed.
And its always a good idea to have daily backups and hourly snapshots. In case something went wrong and the guest won't run anymore, you can restore a snapshot from yesterday or rollback to the hour before the auto upgrade.

Once per week I then manually upgrade small normal upgrades (patch level). And once per month I do the big upgrades (major and minor level upgrades). But right before that I create a backup and right after the upgrade I test everything in short if it still works. And if not I will restore the backup I did some minutes earlier.

I think thats a good workflow for a homelab. For production you might even be more strict and use test systems to verify first that the upgrade is save before rolling out the upgrades to the production systems using ansible.

 

[cjdnad]

Here I always install unattended-upgrades for all VMs and LXCs but not for the PVE host itself and that worked perfectly fine so far. But I wouldn't use it for the PVE host, as you are more likely to break something and you probably don't have host backups that are super up-to-date to restore a PVE installation from right before the upgrade. Also keep in mind that you can't downgrade a PVE installation once upgraded.
And for my VMs/LXCs I only want to auto upgrade security updates and not regular updates. So security holes get closed fast but no updates, that add new features and therefore are more likely to break something, will be auto installed.
And its always a good idea to have daily backups and hourly snapshots. In case something went wrong and the guest won't run anymore, you can restore a snapshot from yesterday or rollback to the hour before the auto upgrade.

Once per week I then manually upgrade small normal upgrades (patch level). And once per month I do the big upgrades (major and minor level upgrades). But right before that I create a backup and right after the upgrade I test everything in short if it still works. And if not I will restore the backup I did some minutes earlier.

I think thats a good workflow for a homelab. For production you might even be more strict and use test systems to verify first that the upgrade is save before rolling out the upgrades to the production systems using ansible.

many thanks - yes I do lxc container backups 3 times a week at least keeping the last 4, to 2 different drives (1 usb mounted hdd and another HDD on a different PC)
The only host upgrades I perform are the manual ones occasionally that appear under the proxmox host 'updates'.
This is just for home use - mqtt/nginx/nodered/duckdns/mariadb/gotify and homeassistant all lxc containers.

Just trying to figure out how to send out the auto update email notifications now. I added my email to the config file /etc/apt/apt.conf.d/50unattended-upgrades but obviously I am missing an important set up.
I also used to get email notifications for the backups but they stopped coming through also.

So still help needed for the email set up for notifications from the containers and the host backups.

Thanks!

 

[Dunuin]

The PVE host uses postfix to send backup notification emails. So you need to setup your postfix server. Same for all your guests. If you want them to send emails you will have to install a email server inside each guest.

But how to setup postfix as a sattelite system really depends on your SMTP provider. Basically each SMTP provider requires a different config. If it doesn't perfectly match, the public SMTP server will just drop your emails and won't relay them.
So best to google for tutorials "postfix tutorial YourMailProviderName".

 

[cjdnad]

The PVE host uses postfix to send backup notification emails. So you need to setup your postfix server. Same for all your guests. If you want them to send emails you will have to install a email server inside each guest.

But how to setup postfix as a sattelite system really depends on your SMTP provider. Basically each SMTP provider requires a dofferent config. If it doesn't peredtly match the public SMTP server will just drop your emails and won't relay them.
So best to google for tutorials "postfix tutorial YourMailProviderName".

great - like this one?

https://forum.proxmox.com/threads/get-postfix-to-send-notifications-email-externally.59940/

Now a daft question are these commands/files on the proxmox host or within each guest(lxc container)?

Thanks

or even gotify instead of email?

 

[Dunuin]

great - like this one?

https://forum.proxmox.com/threads/get-postfix-to-send-notifications-email-externally.59940/

Now a daft question are these commands/files on the proxmox host or within each guest(lxc container)?

I think for the host, but if you want your guests to send emails too, you would have to do the same thing inside each guest.

 

[Dunuin]

I personally only setup postfix on my host because I was too annoyed to set it up for dozens of guests.
Instead I setup filebeat and zabbix-agent inside my guests so I got centralized metrics monitoring using a Zabbix LXC and centralized logging using a Graylog LXC. With that notification mails aren't really needed anymore as I can directly monitor the apt logs for failures and check if upgrades are available through the Graylog and Zabbix webUIs.

 

[cjdnad]

I personally only setup postfix on my host because I was too annoyed to set it up for dozens of guests.
Instead I setup filebeat and zabbix-agent inside my guests so I got centralized metrics monitoring using a Zabbix LXC and centralized logging using a Graylog LXC. With that notification mails aren't really needed anymore as I can directly monitor the apt logs for failures and check if upgrades are available through the Graylog and Zabbix webUIs.

Thanks I'll certainly look into that option also

 

[cjdnad]

struggling with the email configs.

keep getting with tail /var/log/mail.info :
001FBA0E45 414 Thu Jul 21 19:23:20 root@host1.home.com
(delivery temporarily suspended: lost connection with smtp.gmail.com[172.217.192.109] while receiving the initial server greeting)
myemail@gmail.com

been through these tutorials and set up the config files as best I can

https://techlabs.blog/categories/ho...il-notifications-using-postfix-smtp-relayhost
https://forum.proxmox.com/threads/get-postfix-to-send-notifications-email-externally.59940/page-2

 

[cjdnad]

Spent 4 hours on this now with various config settings. No emails coming through
Very frustrating.

Anyone got an easy guide for gotify?