Feedback ProxMox3.0-Wheezy+FreeNAs+Sheepdog...2 Bugs

badji · Apr 4, 2013

I tested the test version Proxmox 3.0 on Debian wheezy with 3 physical servers and external server FreeNAS.
On 3 Proxmox servers, everything works, cman and rgmanager work very well.

# clustat

Cluster Status for BADJI @ Thu Apr 4 2013 9:28:05
Member Status: quorate

Member Name ID Status
--------------------
pserver1 1 Online, Local, rgmanager
pserver2 2 Online, rgmanager
pserver3 3 Online, rgmanager
Service Name Owner (Last) State
------- ---- ----- ------ -----
pvevm:100 (pserver3) stopped
service:TestIP (pserver3) stopped

Sheepdog cluster on dedicated disks works well except for a bug:
Each time you start the server, I have the following error message:

# Collie node list
Apr 04 9:21:14 [main] connect_to (250) failed to connect to localhost: 7000: Connection refused
Apr 04 9:21:14 [main] connect_to (250) failed to connect to localhost: 7000: Connection refused
Failed to get node list

I am forced to restart it from each node with:
# Sheep / var/lib/sheepdog/sdb1

After

# collie node list
Id M Host: Port Zone V-Nodes
- 0 106 191 146 176 192.168.100.11:7000
- 1 34 207 923 392 192.168.100.12:7000
- 2 224 700 608 53 192.168.100.13:7000

Another bug to create the NFS server on Proxmox
I can create the FreeNAS storage server, it is well connected and activated, shared but no possibility to change the contained, permission problem when I'm logged in as root
Message:
mkdir / mnt / pve / MyFreeNAS / template: Permission denied at / usr/share/perl5/PVE/Storage/Plugin.pm line 786 (500)

I'm stuck here.
thank you

tom · Apr 4, 2013

If you have a permission problem with your NFS server, fix it. You need to allow rw access for root, which is disabled by default on freenas.

badji · Apr 4, 2013

tom said:
If you have a permission problem with your NFS server, fix it. You need to allow rw access for root, which is disabled by default on freenas.

Thank you Tom, FreeNAS server works.

I have 2 little bugs and I keep my testes

1st : when you want to install debian or centos for example, so more dvd iso, no option to choose more than one in:

Use CD / DVD disc image file (iso) iso image just one.

Seconde : when I use a VM managed by HA sheepdog, it restarted on another physical server, but always it reboote from FreeNAS iso - like a first install- while I have checked the boot order, the hard disk - sheepdog - before the cdrom.

thank's

spirit · Apr 4, 2013

for sheepdog, maybe the service is not enable at boot ?

# update-rc.d sheepdog defaults

also check in : /etc/default/sheepdog

START=yes

tom · Apr 4, 2013

badji said:
...
1st : when you want to install debian or centos for example, so more dvd iso, no option to choose more than one in:

Use CD / DVD disc image file (iso) iso image just one.

there is only one in the wizard but after the wizard you can add a second one via hardware tab, see http://www.youtube.com/watch?feature=player_detailpage&v=ZUxrdTvaQvU#t=228s

badji · Apr 4, 2013

Thank you Tom and thank you Alexandre.

In the file / etc / default / sheepdog i find SATART file = YES
and when i do # update-default rc.c shhepdog

I have this message: The disable | enable API is not stable, and might change in the future.

I reboot my server, but the problem still exists.

bye

tom · Apr 4, 2013

just to note, sheepdog is experimental only so you will see issues.

take a look on Ceph.

badji · Apr 15, 2013

tom said:
just to note, sheepdog is experimental only so you will see issues.

take a look on Ceph.

Hi,

This is the result of my tests.

On each server, I restart the daemon sheepdog manually
# /etc/init.d/sheepdog start
# /etc/init.d/sheepdog status

sheepdog /var/lib/sheepdog/disc0 is running.

# collie node list

Id M Host: Port Zone V-Nodes
- 0 106 191 146 176 192.168.100.11:7000
- 1 34 207 923 392 192.168.100.12:7000
- 2 224 700 608 53 192.168.100.13:7000

and :

# collie vdi list
Id Name Size Used Shared Creation time VDI id Copies Tag
vm-104-disk-1 1 8.0 GB 3.0 GB 0.0 MB 2013-04-12 3 9:17 ac336
vm-102-disk-1 1 12 GB 2.1 GB 0.0 MB 2013-04-12 9:03 175e70 3
vm-105-disk-1 1 8.0 GB 1.6 GB 0.0 MB 2013-04-12 9:32 4121eb 3
vm-101-disk-1 Jan. 22 GB 8.9 GB 0.0 MB 2013-04-12 8:44 468e4f 3
vm-106-disk-1 1 32 GB 2.8 GB 0.0 MB 2013-04-12 7:18 p.m. 4cb58c 3
vm-103-disk-1 1 32 GB 3.3 GB 0.0 MB 2013-04-12 9:12 6286b5 3
vm-108-disk-1 1 8.0 GB 2.9 GB 0.0 MB 2013-04-12 943 022 3 4:24 p.m.
vm-100-disk-1 Jan. 22 GB 3.4 GB 0.0 MB 2013-04-12 8:37 abc07a 3

I start a vm, which works, but as soon as I shutdown a server of a cluster to test the proper operation of MV with other servers.

# Collie vdi list
Id Name Size Used Shared Creation time VDI id Copies Tag
Failed to read object 800ac33600000000 IO HAS halted as there are too FEW living nodes
Failed to read inode header
Failed to read object 80175e7000000000 IO HAS halted as there are too FEW living nodes
Failed to read inode header
Failed to read object 804121eb00000000 IO HAS halted as there are too FEW living nodes
Failed to read inode header
Failed to read object 80468e4f00000000 IO HAS halted as there are too FEW living nodes
Failed to read inode header
Failed to read object 804cb58c00000000 IO HAS halted as there are too FEW living nodes
Failed to read inode header
Failed to read object 806286b500000000 IO HAS halted as there are too FEW living nodes
Failed to read inode header
Failed to read object 8094302200000000 IO HAS halted as there are too FEW living nodes
Failed to read inode header
Failed to read object 80abc07a00000000 IO HAS halted as there are too FEW living nodes
Failed to read inode header
Failed to read object 80d17c9100000000 IO HAS halted as there are too FEW living nodes
Failed to read inode header

and VM crashes completely.

Thank you for your help.

Moula

spirit · Apr 15, 2013

badji said:
IO HAS halted as there are too FEW living nodes

If you have formated with

collie cluster format --copies=3
you always need 3 online servers.

so try with

collie cluster format --copies=2

I'll update the proxmox wiki.

spirit · Apr 15, 2013

I think I have found the sheepdog start but at boot.
corosync need to be started before sheepdog.

Can you try this:

Code:

update-rc.d -f sheepdog remove

edit the file

Code:

/etc/init.d/sheepdog

and change the lines
# Required-Start:    $network $remote_fs $syslog
# Required-Stop:     $network $remote_fs $syslog
to 
# Required-Start:    $network $remote_fs $syslog cman
# Required-Stop:    $network $remote_fs $syslog cman

then

Code:

update-rc.d sheepdog defaults

badji · Apr 15, 2013

Thank you Alexandre, it works

Just a note that all VMS I created with the command
collie cluster format - copies = 3 instead of collie cluster format - copies = 2
no longer work. I had to recreate all.

I'm going to test the Ceph cluster with Proxmox3.0, I tested and I find bugs and Alexander finds solutions

.

Thank you again.

mmenaz · Apr 16, 2013

I've seen the wiki, but I'm a bit confused, could you please clarify this part of the wiki?

Code:

#collie cluster format --copies=2
"--copies" specifies the number of default data redundancy. In this case, the replicated data will be stored on three machines.

why do I need to store in 3 machines when I specify --copies=2 ?

spirit · Apr 16, 2013

mmenaz said:
I've seen the wiki, but I'm a bit confused, could you please clarify this part of the wiki?

Code:

#collie cluster format --copies=2 "--copies" specifies the number of default data redundancy. In this case, the replicated data will be stored on three machines.

why do I need to store in 3 machines when I specify --copies=2 ?

Sorry, I forgot to update the wiki.
of course,copies=2, you have replication on 2 nodes

About cluster halt behaviour, they are a flag to manage this

Code:

--mode safe|quorum|unsafe  

safe mode: halt cluster when nr_nodes < nr_copies (default) 
quorum mode: halt cluster when nr_nodes < nr_copies / 2 + 1 
unsafe mode: never halt the cluster.

I'll try to improve init.d to manage it

jmjosebest · Apr 17, 2013

Sorry for the off-topic
Proxmox 3.0? where is the info of this release?
Thanks!

tom · Apr 17, 2013

its not officially released (but it will be announce here)
you can find some info in the wiki (search for wheezy).

badji · Apr 17, 2013

jmjosebest said:
Sorry for the off-topic
Proxmox 3.0? where is the info of this release?
Thanks!

http://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Wheezy

# pveversion -v
pve-manager: 3.0-6 (pve-manager/3.0/af62c8a0)
running kernel: 2.6.32-20-pve
proxmox-ve-2.6.32: 3.0-96
pve-kernel-2.6.32-20-pve: 2.6.32-96
lvm2: 2.02.95-pve3
clvm: 2.02.95-pve3
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb: not correctly installed
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.0-1
pve-cluster: 3.0-1
qemu-server: 3.0-2
pve-firmware: 1.0-22
libpve-common-perl: 3.0-1
libpve-access-control: 3.0-1
libpve-storage-perl: 3.0-1
vncterm: 1.1-1
vzctl: 4.0-1pve3
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.4-9
ksm-control-daemon: 1.1-1

badji · Apr 20, 2013

Hello,

The rest of my tests:

Everything works, but as soon as I turn off my servers and at the next startup of the cluster no vm sheepdog deployed is not available, they are still available on the Proxmox GUI but it does not work.
I retested at several times by changing the configuration of sheepdog disk (iscsi, virtio) it is always the same worries.
Thank you.

Search

Search

Feedback ProxMox3.0-Wheezy+FreeNAs+Sheepdog...2 Bugs

badji

Renowned Member

tom

Proxmox Staff Member

badji

Renowned Member

spirit

Distinguished Member

tom

Proxmox Staff Member

badji

Renowned Member

tom

Proxmox Staff Member

badji

Renowned Member

spirit

Distinguished Member

spirit

Distinguished Member

badji

Renowned Member

mmenaz

Renowned Member

spirit

Distinguished Member

jmjosebest

Renowned Member

tom

Proxmox Staff Member

badji

Renowned Member

badji

Renowned Member