[SOLVED] Today's Kernel / Firmware Update has really messed up my boxes!

[LordRatner]

Note: If you run PBS, it won't work with the overlay file system. So, my suggestion at the moment, stick with VFS. Or, if you're so inclined, move your docker to the host server and avoid all of this fun!

Does it work with fuse-overlayfs?

 

[sirebral]

Yes, I know, yet I've been using it, by default for over a year now on this box, and it's been fine. I don't know of any other way to go about fixing this, other than moving to bare-metal. If you have any recommendations, I'd be happy to explore options.

Thanks!

 

[sirebral]

Does it work with fuse-overlayfs?

It'll work, yet fuse-overlayfs gave me issues with my backups to my Proxmox Backup Server. It showed up as permission errors during the backup, anywhere that I had nested docker. When I tried a restore, none of my Docker containers were functional as it was unable to backup /var/lib/docker/fuse-overlayfs/. So, while it does run, it's presently not a workable solution for my use-case.

 

[lordcheeto]

EDIT: Here's the fix, it's clean for most setups, things like Home Assistant OS may or may not work, still finding out. Ultimately, you'll need to create a file in /etc/docker/ called daemon.json. Inside the file put a line:

storage-driver": "vfs"

Missing the opening quotation mark, but you're a lifesaver! Just wish I found this a few hours ago.

I didn't realize LXC was not recommended for a docker host, I'll have to work on migrating to a VM. Any downsides to a VM over LXC?

 

[sirebral]

Glad to help, issue I have with any VM passthrough is that it's exclusive. I think you'd be best off to just load docker on your Proxmox host. May take some work yet if you can assign a zfs pool that seems like a good option. Other option would be overlay 2, yet I believe they requires EXT4.

 

[curlypdingo]

ok, I changed to vfs and everything came back. This will probably break most docker installs that are running on Proxmox in LXC, so this will hopefully be helpful for people.

If anyone can chime in, is there any way to use a more performant driver with the nested config? I'm about to redo my disks, so I'll probably just setup docker on ZFS on bare metal as that seems to be the better solution, yet I'm wondering if there's a better option for me to use until I have time to do my rebuild...

Thanks for the assist Fabian, you may want to spread this info, considering there will be several others with the same issue, considering it defaults to overlay2 even though it's not actually compatible.

Cheers!

Have registered on the forum just to say this:

THANK YOU!

Just got hit by the same problem, similarly had no idea docker inside LXC was a bad thing; pity really since before this little episode it was a really nice setup...

 

[rakali]

Missing the opening quotation mark, but you're a lifesaver! Just wish I found this a few hours ago.

I didn't realize LXC was not recommended for a docker host, I'll have to work on migrating to a VM. Any downsides to a VM over LXC?

I will migrate to a vm too, but temporarily, to make the vfs work, I needed braces as well, like this:

cat /etc/docker/daemon.json
{
      "storage-driver": "vfs"
}