Can't upload to ceph storage

[D.P.]

Hi!
I have Proxmox 4.2 cluster configured (https://pve.proxmox.com/wiki/Proxmox_VE_4.x_Cluster) with ceph storage (http://pve.proxmox.com/wiki/Ceph_Server#Ceph_Pools).
Proxmox version is
root@prox1:/etc/pve# pveversion
pve-manager/4.2-4/2660193c (running kernel: 4.4.8-1-pve)
without subscriptions.

Cephs storage is configured as:
rbd: cephpool
monhost 192.168.168.10;192.168.168.11;192.168.168.12
pool cephpool
krbd
username admin
content rootdir,images
I can see ceph's storage information on tab "Summary".
I can't upload any content to this storage because buttons "Templates" and "Upload' are inactive (grey).
I can't create container on this storage.
How to fix this problem?

 

[gosha]

Cephs storage is configured as:
...
content rootdir,images
...
I can't upload any content to this storage because buttons "Templates" and "Upload' are inactive (grey).

It is right. CEPH-storage in Proxmox - Block Device (rbd) - not File sysytem. Is used for KVM disk images and Сontainers only.

I can't create container on this storage.
How to fix this problem?

And it is very strange...
In console: ceph health detail
and post here...

 

[D.P.]

root@prox1:/etc/pve# ceph health detail
HEALTH_OK

Do you know: it is possible to create shared filesystem for proxmox nodes from ceph's block device?

 

[tom]

do you use hammer or jewel? jewel is not yet recommended, still some small issues (e.g. with containers, krbd)

 

[gosha]

Do you know: it is possible to create shared filesystem for proxmox nodes from ceph's block device?

For example You may create VM with nfs-server on ceph storage and add it to cluster as storage.



--
Best regards,
Gosha

 

[D.P.]

For example You may create VM with nfs-server on ceph storage and add it to cluster as storage.

I like this idea but I can't create VM on cep storage, see below:

 

[D.P.]

do you use hammer or jewel? jewel is not yet recommended, still some small issues (e.g. with containers, krbd)

I think I have Ceph hammer version:
root@prox1:/etc/pve# ceph --version
ceph version 0.94.7 (d56bdf93ced6b80b07397d57e3fa68fe68304432)

 

[gosha]

I like this idea but I can't create VM on cep storage, see below:

You need add your ceph-storage (rbd) to cluster:



--
Gosha

 

[D.P.]

Ceph storage is added to cluster.

 

[johanw]

I have the exact same issue. I followed the guide here:
http://pve.proxmox.com/wiki/Ceph_Server#Ceph_Pools

I only have two nodes, but the op seems to use three nodes exactly as the guide suggests. My "upload" button is also grayed out and I can't use the ceph storage for backups.

 

[gosha]

I like this idea but I can't create VM on cep storage, see below:

Opps! Select on your picture - is not storage for VM-disk. This select for iso and ct-templates storage (for install vm's OS).
On the next tab (Hard disk), You can select storage for VM-disk.

 

[gosha]

I only have two nodes, but the op seems to use three nodes exactly as the guide suggests. My "upload" button is also grayed out and I can't use the ceph storage for backups.

ceph-storage is not filesystem, ceph-storage - block device for vm- or ct-disk only
Backups can be placed on the file system only. NFS for example.

 

[johanw]

Well, that explains it. Thanks gosha! I was just wondering why the ceph-pool didn't show up as mounted either in the system.
I guess I have to rethink how to share snapshots and backups between the nodes.

 

[D.P.]

Yeah! it works as a hard disk for VM and CT
Thanks a lot gosha.

 

[gosha]

I have to rethink how to share snapshots and backups between the nodes.

The easiest way - to create a VM (nfs server) on ceph and share created nfs-storage in your cluster for backups.
But the more correct way - to place storage for backups not in this cluster.
For example in another cluster - like I did.

 

[johanw]

Thanks a lot gosha!

What I did for now was to install nfs-kernel-server and share my zfs backup pool (on both servers). Then I added the two nfs-shares to the cluster so I can make backups and restores between the hosts and move vm's.

I agree that backups should be saved elsewhere. I even prefer to pull the backups from the backup server for a evil-worst-case scenario where the cluster was compromised. Call me paranoid..

 

[gosha]

I even prefer to pull the backups from the backup server for a evil-worst-case scenario where the cluster was compromised. Call me paranoid..

No. You're not paranoid. This is a very reasonable choice!
I was in a situation where the server-room was flooded with water. All servers died!