Packages kept back on upgrade to 6.0 with glusterfs official repo

robynhub

Member
Nov 15, 2011
64
0
6
Hello,

after upgrading i have some packages kept back:

~# apt-get dist-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
pve-qemu-kvm qemu-server spiceterm
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.

seems to be related with the official gluster repo:

# apt-get upgrade pve-qemu-kvm
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
glusterfs-common : Conflicts: libglusterfs0 but 5.5-3 is to be installed
pve-libspice-server1 : Conflicts: libspice-server1
E: Broken packages

~# dpkg -l | grep gluster
ii glusterfs-client 6.4-1 amd64 clustered file-system (client package)
ii glusterfs-common 6.4-1 amd64 GlusterFS common libraries and translator modules


any suggestion how to fix this?
I have enterprise repo and subscription.
 
Last edited:

robynhub

Member
Nov 15, 2011
64
0
6
Downgrading to gluster 5.5 solved the issue but now I can't access to my repo.

I really NEED to upgrade glusterfs to 6.4 with proxmox 6. With the PVE 5.3 aren't any issues.

I just followed the official Glusterfs install guide for Debian buster.

# aptitude dist-upgrade
The following packages will be REMOVED:
libgfchangelog0{u} libgfdb0{u} libglusterfs-dev{u} libpython3.7{u} python3-jwt{u}
The following packages will be upgraded:
glusterfs-client glusterfs-common{b}
2 packages upgraded, 0 newly installed, 5 to remove and 0 not upgraded.
Need to get 8,434 kB of archives. After unpacking 9,181 kB will be freed.
The following packages have unmet dependencies:
glusterfs-common : Conflicts: libglusterfs0 but 5.5-3 is installed
 
Last edited:

Dominic

Proxmox Staff Member
Staff member
Mar 18, 2019
115
17
18
Thank you for reporting!
We are aware of problems with GlusterFS 6 on PVE6. Unfortunately, this is not something we could repair with a small patch. You can follow the exact progress on this problem on Bugzilla.

now I can't access to my repo.
I am not sure about your goals. Is refraining from a converged setup and running a VM as storage server with for example plain Debian as Gluster server an option instead?
 

robynhub

Member
Nov 15, 2011
64
0
6
Hello,

Thank you for your answer. I have this setup:

4 X NAS with Debian 10 and Gluster 6.4 as servers
10 X ProxmoxVE Nodes Enterprise 6.0-2
10 Gbit networking.
about 350 VMs

Before upgrading to PVE 6.0 I had all nodes upgraded to Gluster 6.4 and the Gluster op.version of every share is 60000.
Now with Proxmox 6.0 I had to downgrade the glusterfs clients to 5.5 with clients with op.version of 55000

After a few test i managed to mount a gluster 6.4 share with a 5.5 client but I can't confirm the stability of this solution.
I'm already suprised that the gluster server allowed the mount with a client with an op.version lower than the server!
This should not work by the glusterfs documentation.

I will keep an eye on bugzilla page.
 

malevolent

New Member
Jun 1, 2012
20
1
3
It seems some pve packages are tied to gluster 5.5, so we cannot upgrade gluster through gluster official repositories... and obviously, if you upgrade/install a gluster version higher than 5.5 before upgrading/installing proxmox 6, installation fails.
 

Dominic

Proxmox Staff Member
Staff member
Mar 18, 2019
115
17
18
I'm already suprised that the gluster server allowed the mount with a client with an op.version lower than the server!
This should not work by the glusterfs documentation.
I cannot say much about stability yet, but I installed Gluster 6.4-1 (OpVersion 60000) on 2 Debian 10 VMs and added it as storage on the PVE 6 host running Gluster 5.5-3 (OpVersion 50400) without any problems.
 

Apollon77

New Member
Sep 24, 2018
27
4
3
42
I also run intothis problem now witj gluster 5.8 installed ... PVE 5.4 was accepting this without any problem and my setup is, as also described in the documentation so that I have installed the glusterfs on all my 7 PVE nodes directly ... especially glusterfs 5.5 had big issues with increased network usage that are only solved by 5.6, so reverting my installation back to 5.5 would be the worst option somehow ...

In this case PVE 6 misses a very important feature :-(

It do not help me really, and I expect that true for others too, that it works to connect to a "remote" GlusterFS because all my nodes have it installed locally.

Why glusterfs 5.5 is tied to PVE packages in that way this time? It was working without any issues in PVE 5.4 where it was not tied that hard :-(

Beside the important fact that this should be added asap to the 5-to-6 Upgrade page, because if it would had been there yesterday I would not have started to upgrade.

So my main question is: What are my options now?

Is there a fix/change forseeable in the way how the packages are tied together? Or upgraded to at least 5.6 so that I could think of downgrading (or tied to 6.x so that I could upgrade)

I already upgraded my complete cluster to corosync3 ... how can I revert that?
Or can I have my PVE 5.4 cluster run with corosync3 till a fix comes? Information from other threads let me think this is no good option...

I only have one node upgraded to buster so far, so I need to see what happens if I reboot that host now ... I expect running KVMs is broken because the packages "pve-qemu-kvm qemu-server spiceterm" are not updated ... Else I could re-setup that one on stretch - what would bring me back to the corosync-downgrade question above.

Thank you very much for ideas how to deal with this situation now...

Ingo
 

Apollon77

New Member
Sep 24, 2018
27
4
3
42
Ok, as short update ... I was able to downgrade corosync and will see if my system in general stabilize again (it was a very bad day today with 2 kernel crashes and spontanous reboots and a lot of corosync problems ...), so I now only need to find a way how to handle the one broken host.

But ith this I'm back to 5.4 ... no idea how to continue because of this topic and all those corosync3 issues that I had
 
Last edited:

Dominic

Proxmox Staff Member
Staff member
Mar 18, 2019
115
17
18
Beside the important fact that this should be added asap to the 5-to-6 Upgrade page, because if it would had been there yesterday I would not have started to upgrade.
I've just sent a patch.

The problem is that Gluster 5.5-3 does not conflict with the package libglusterfs0 but more recent versions like 6.4-1 do.
Code:
dpkg -s glusterfs-common | egrep "Version|Conflict"
Please take a look at Dominik's comment in Bugzilla for a more detailed description.
 

Apollon77

New Member
Sep 24, 2018
27
4
3
42
I've just sent a patch.
Patch for the webpage? Thank you!

Thank you very much

PS: On my system the above command shows:

root@pm7:~# dpkg -s glusterfs-common | egrep "Version|Conflict"
Version: 5.8-1
Conflicts: libglusterfs-dev, libglusterfs0
 
Last edited:

Apollon77

New Member
Sep 24, 2018
27
4
3
42
I have send the glusterfs "packager" an email and also pointed to the bug tracker ticket ... let's hope he can support
 

Dominic

Proxmox Staff Member
Staff member
Mar 18, 2019
115
17
18
Last edited:

Dominic

Proxmox Staff Member
Staff member
Mar 18, 2019
115
17
18
Added it to the wiki, hoping it's sufficiently visible.
 

Apollon77

New Member
Sep 24, 2018
27
4
3
42
Maybe add the link to that section as" check jknown upgrade issues if they affect you" in the "preconditions too with a link down to the konwon issues? (so easier for people that follwo checklists ;-) )
 
  • Like
Reactions: Dominic

Dark26

Member
Nov 27, 2017
81
5
8
42
On my own config @home, everyworking fine with gluster v6, but promox was installed over a debian insatllation and i add manually the gluster repository

Code:
deb [arch=amd64] https://download.gluster.org/pub/gluster/glusterfs/6/LATEST/Debian/buster/amd64/apt buster main
before the uprgrade it was stretch

so far so good

Edit :

After verification, it seem that i have the problem too ....
 
Last edited:
  • Like
Reactions: Atila Vasconcelos

Apollon77

New Member
Sep 24, 2018
27
4
3
42
@Dominik But one question is still in my head: In PVE 5 everything works up to current glusterfs packages. They can simply be upgraded and the "gluster-common" and "gluster.client" gets updated too and so PVE is using the newer glusterfs version also as client. As we know this works with all glusterfs package up to 6.x current ones. So also the glusterfs packaging changes have no negative effect here.

But not with PVE 6 ... So beside the glusterfs packaging changes there needs to be a difference on the PVE packaging side! It seems that the dependecies to glusterfs packages are done differently in PVE6 packages ... So I would more think about looking into this because you (=Proxmox) have this side of the packaging under your control!
Simply do it the same as in PVE5 here and the problem should be gone ... Or at least please provide (also in the Bugzilla) information on thos packaging differences and why they are needed and can not be done as earlier.

Thank you
 
  • Like
Reactions: malevolent

malevolent

New Member
Jun 1, 2012
20
1
3
After upgrading all PVE nodes to v6, I realized that gluster 5.5 is damn slow compared with 3.12.15 ... I've readed somewhere it's solved on 5.8 onwards... I had to put critical (DBs) VMs on plain ZFS in order make them work, so I have no redundancy at all besides a simple raidz1

@Dominic would you be son kind to tell us if you have any ETA for the pve dependencies, so we can upgrade gluster (or downgrade it to the old and good 3.12.15)?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!