glusterfs 4.0 released and it has much better support for container (docker). is it possible now proxmox can run lxc on glusterfs 4.0?
lxc rootfs on glusterfs
lxc rootfs on glusterfs
Related to gluster: the pve wiki is wrong i suggest a check
https://forum.proxmox.com/threads/wiki-storage-glusterfs.34204/
https://forum.proxmox.com/threads/wiki-storage-glusterfs.34204/
@Alessandro 123
We can read and it helps nothing if you mix things up. ClusterFS with container as nothing to do with the WIKI.
We can read and it helps nothing if you mix things up. ClusterFS with container as nothing to do with the WIKI.
Thread is related to Gluster and your docs talk about gluster thus i don't see anything bad in my post because even with lxc you still have to resync in case of disk failure.
Hi Wolfgang, when do you think we might see GFS 4.x integration and the ability to use it as an LXC storage backend? If you need some beta testers let us know.
Two gfs volumes I can test are below, gfsper-vol1 comprises SSD only, while gfstri-vol1 comprises magnetic media with an SSD hot tier that's a little under 10% of the cold-tier's size.
Volume Name: gfsper-vol1
Type: Disperse
Volume ID: 90d6b5a7-c5bb-4b6b-a499-f20c84424df7
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x (2 + 1) = 3
Transport-type: tcp
Bricks:
Brick1: pve1-storage:/data/gluster/gvol1/brick1/brick
Brick2: pve2-storage:/data/gluster/gvol1/brick1/brick
Brick3: pve3-storage:/data/gluster/gvol1/brick1/brick
Options Reconfigured:
transport.address-family: inet
performance.readdir-ahead: on
nfs.disable: on
Volume Name: gfstri-vol1
Type: Tier
Volume ID: 30e0ac5f-c6eb-4cc6-9cb1-66c555028f3c
Status: Started
Snapshot Count: 0
Number of Bricks: 4
Transport-type: tcp
Hot Tier :
Hot Tier Type : Replicate
Number of Bricks: 1 x 2 = 2
Brick1: pve3-storage:/data/gluster/gvol2/brick2/brick
Brick2: pve1-storage:/data/gluster/gvol2/brick2/brick
Cold Tier:
Cold Tier Type : Distribute
Number of Bricks: 2
Brick3: pve2-storage:/data/gluster/gvol2/brick2/brick
Brick4: pve3-storage:/data/gluster/gvol3/brick3/brick
Options Reconfigured:
cluster.tier-mode: cache
features.ctr-enabled: on
transport.address-family: inet
performance.readdir-ahead: on
nfs.disable: on
Hi Wolfgang,
Appreciate the reply. If I was to use an NFS export, iSCSI target, or NFS Ganesha as a workaround in the interim (but obviously with a Gluster 3.8.x back end) on either/both of the volumes I described above, do you see performance issues being a problem still?
I'm about to integrate 10 nodes of distributed storage into Proxmox and I'm kind of glad I'm at the step of "ceph or gluster" as it appears I may have to go with the former if you indicate there might be issues on my existing volumes and a much larger deployment (10 nodes, 60TB).
Appreciate the reply. If I was to use an NFS export, iSCSI target, or NFS Ganesha as a workaround in the interim (but obviously with a Gluster 3.8.x back end) on either/both of the volumes I described above, do you see performance issues being a problem still?
I'm about to integrate 10 nodes of distributed storage into Proxmox and I'm kind of glad I'm at the step of "ceph or gluster" as it appears I may have to go with the former if you indicate there might be issues on my existing volumes and a much larger deployment (10 nodes, 60TB).
I made only a few quick tests with glusterfs 4.0 and it looks good and stable.
With 10 nodes I would go with ceph.
Does this mean, that glusterfs would be able to run LXC directly in subdirectory (like ZFS), or i'd still have to use LXC disk images (like dir backend)? It would be nice if it was possible to somehow dynamicaly allocate space like ZFS subvols do, so we don't need any balooning for thin provisioning... i know i can disable disk images and go with plain directory even with directory storage type, but then there's no space limit in that container. Right now i solve this by using ZFS, but it's not a cluster filesystem...