Feedback ProxMox3.0-Wheezy+FreeNAs+Sheepdog...2 Bugs

Discussion in 'Proxmox VE: Installation and configuration' started by badji, Apr 4, 2013.

  1. badji

    badji Member

    Joined:
    Jan 14, 2011
    Messages:
    171
    Likes Received:
    4
    I tested the test version Proxmox 3.0 on Debian wheezy with 3 physical servers and external server FreeNAS.
    On 3 Proxmox servers, everything works, cman and rgmanager work very well.

    # clustat

    Cluster Status for BADJI @ Thu Apr 4 2013 9:28:05
    Member Status: quorate

    Member Name ID Status
    --------------------
    pserver1 1 Online, Local, rgmanager
    pserver2 2 Online, rgmanager
    pserver3 3 Online, rgmanager
    Service Name Owner (Last) State
    ------- ---- ----- ------ -----
    pvevm:100 (pserver3) stopped
    service:TestIP (pserver3) stopped


    Sheepdog cluster on dedicated disks works well except for a bug:
    Each time you start the server, I have the following error message:

    # Collie node list
    Apr 04 9:21:14 [main] connect_to (250) failed to connect to localhost: 7000: Connection refused
    Apr 04 9:21:14 [main] connect_to (250) failed to connect to localhost: 7000: Connection refused
    Failed to get node list

    I am forced to restart it from each node with:
    # Sheep / var/lib/sheepdog/sdb1

    After

    # collie node list
    Id M Host: Port Zone V-Nodes
    - 0 106 191 146 176 192.168.100.11:7000
    - 1 34 207 923 392 192.168.100.12:7000
    - 2 224 700 608 53 192.168.100.13:7000

    Another bug to create the NFS server on Proxmox
    I can create the FreeNAS storage server, it is well connected and activated, shared but no possibility to change the contained, permission problem when I'm logged in as root
    Message:
    mkdir / mnt / pve / MyFreeNAS / template: Permission denied at / usr/share/perl5/PVE/Storage/Plugin.pm line 786 (500)

    I'm stuck here.
    thank you
     
  2. tom

    tom Proxmox Staff Member
    Staff Member

    Joined:
    Aug 29, 2006
    Messages:
    13,217
    Likes Received:
    363
    If you have a permission problem with your NFS server, fix it. You need to allow rw access for root, which is disabled by default on freenas.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. badji

    badji Member

    Joined:
    Jan 14, 2011
    Messages:
    171
    Likes Received:
    4
    Thank you Tom, FreeNAS server works.

    I have 2 little bugs and I keep my testes

    1st : when you want to install debian or centos for example, so more dvd iso, no option to choose more than one in:

    Use CD / DVD disc image file (iso) iso image just one.

    Seconde : when I use a VM managed by HA sheepdog, it restarted on another physical server, but always it reboote from FreeNAS iso - like a first install- while I have checked the boot order, the hard disk - sheepdog - before the cdrom.

    thank's
     
  4. spirit

    spirit Well-Known Member

    Joined:
    Apr 2, 2010
    Messages:
    3,221
    Likes Received:
    119
    for sheepdog, maybe the service is not enable at boot ?

    # update-rc.d sheepdog defaults


    also check in : /etc/default/sheepdog

    START=yes
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  5. tom

    tom Proxmox Staff Member
    Staff Member

    Joined:
    Aug 29, 2006
    Messages:
    13,217
    Likes Received:
    363
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  6. badji

    badji Member

    Joined:
    Jan 14, 2011
    Messages:
    171
    Likes Received:
    4
    Thank you Tom and thank you Alexandre.

    In the file / etc / default / sheepdog i find SATART file = YES
    and when i do # update-default rc.c shhepdog

    I have this message: The disable | enable API is not stable, and might change in the future.

    I reboot my server, but the problem still exists.

    bye
     
  7. tom

    tom Proxmox Staff Member
    Staff Member

    Joined:
    Aug 29, 2006
    Messages:
    13,217
    Likes Received:
    363
    just to note, sheepdog is experimental only so you will see issues.

    take a look on Ceph.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  8. badji

    badji Member

    Joined:
    Jan 14, 2011
    Messages:
    171
    Likes Received:
    4
    Hi,

    This is the result of my tests.

    On each server, I restart the daemon sheepdog manually
    # /etc/init.d/sheepdog start
    # /etc/init.d/sheepdog status

    sheepdog /var/lib/sheepdog/disc0 is running.

    # collie node list

    Id M Host: Port Zone V-Nodes
    - 0 106 191 146 176 192.168.100.11:7000
    - 1 34 207 923 392 192.168.100.12:7000
    - 2 224 700 608 53 192.168.100.13:7000

    and :

    # collie vdi list
    Id Name Size Used Shared Creation time VDI id Copies Tag
    vm-104-disk-1 1 8.0 GB 3.0 GB 0.0 MB 2013-04-12 3 9:17 ac336
    vm-102-disk-1 1 12 GB 2.1 GB 0.0 MB 2013-04-12 9:03 175e70 3
    vm-105-disk-1 1 8.0 GB 1.6 GB 0.0 MB 2013-04-12 9:32 4121eb 3
    vm-101-disk-1 Jan. 22 GB 8.9 GB 0.0 MB 2013-04-12 8:44 468e4f 3
    vm-106-disk-1 1 32 GB 2.8 GB 0.0 MB 2013-04-12 7:18 p.m. 4cb58c 3
    vm-103-disk-1 1 32 GB 3.3 GB 0.0 MB 2013-04-12 9:12 6286b5 3
    vm-108-disk-1 1 8.0 GB 2.9 GB 0.0 MB 2013-04-12 943 022 3 4:24 p.m.
    vm-100-disk-1 Jan. 22 GB 3.4 GB 0.0 MB 2013-04-12 8:37 abc07a 3

    I start a vm, which works, but as soon as I shutdown a server of a cluster to test the proper operation of MV with other servers.

    # Collie vdi list
    Id Name Size Used Shared Creation time VDI id Copies Tag
    Failed to read object 800ac33600000000 IO HAS halted as there are too FEW living nodes
    Failed to read inode header
    Failed to read object 80175e7000000000 IO HAS halted as there are too FEW living nodes
    Failed to read inode header
    Failed to read object 804121eb00000000 IO HAS halted as there are too FEW living nodes
    Failed to read inode header
    Failed to read object 80468e4f00000000 IO HAS halted as there are too FEW living nodes
    Failed to read inode header
    Failed to read object 804cb58c00000000 IO HAS halted as there are too FEW living nodes
    Failed to read inode header
    Failed to read object 806286b500000000 IO HAS halted as there are too FEW living nodes
    Failed to read inode header
    Failed to read object 8094302200000000 IO HAS halted as there are too FEW living nodes
    Failed to read inode header
    Failed to read object 80abc07a00000000 IO HAS halted as there are too FEW living nodes
    Failed to read inode header
    Failed to read object 80d17c9100000000 IO HAS halted as there are too FEW living nodes
    Failed to read inode header

    and VM crashes completely.

    Thank you for your help.

    Moula
     
  9. spirit

    spirit Well-Known Member

    Joined:
    Apr 2, 2010
    Messages:
    3,221
    Likes Received:
    119
    If you have formated with

    collie cluster format --copies=3
    you always need 3 online servers.

    so try with

    collie cluster format --copies=2


    I'll update the proxmox wiki.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  10. spirit

    spirit Well-Known Member

    Joined:
    Apr 2, 2010
    Messages:
    3,221
    Likes Received:
    119
    I think I have found the sheepdog start but at boot.
    corosync need to be started before sheepdog.

    Can you try this:
    Code:
    update-rc.d -f sheepdog remove
    
    edit the file
    Code:
    /etc/init.d/sheepdog
    
    and change the lines
    # Required-Start:    $network $remote_fs $syslog
    # Required-Stop:     $network $remote_fs $syslog
    to 
    # Required-Start:    $network $remote_fs $syslog cman
    # Required-Stop:    $network $remote_fs $syslog cman
    
    then

    Code:
    update-rc.d sheepdog defaults
    
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    #10 spirit, Apr 15, 2013
    Last edited: Apr 15, 2013
  11. badji

    badji Member

    Joined:
    Jan 14, 2011
    Messages:
    171
    Likes Received:
    4
    Thank you Alexandre, it works

    Just a note that all VMS I created with the command
    collie cluster format - copies = 3 instead of collie cluster format - copies = 2
    no longer work. I had to recreate all.

    I'm going to test the Ceph cluster with Proxmox3.0, I tested and I find bugs and Alexander finds solutions :D.

    Thank you again.
     
  12. mmenaz

    mmenaz Member

    Joined:
    Jun 25, 2009
    Messages:
    735
    Likes Received:
    5
    I've seen the wiki, but I'm a bit confused, could you please clarify this part of the wiki?
    Code:
    #collie cluster format --copies=2
    "--copies" specifies the number of default data redundancy. In this case, the replicated data will be stored on three machines. 
    
    why do I need to store in 3 machines when I specify --copies=2 ?
     
  13. spirit

    spirit Well-Known Member

    Joined:
    Apr 2, 2010
    Messages:
    3,221
    Likes Received:
    119
    Sorry, I forgot to update the wiki.
    of course,copies=2, you have replication on 2 nodes ;)


    About cluster halt behaviour, they are a flag to manage this

    Code:
    
    --mode safe|quorum|unsafe  
    
    safe mode: halt cluster when nr_nodes < nr_copies (default) 
    quorum mode: halt cluster when nr_nodes < nr_copies / 2 + 1 
    unsafe mode: never halt the cluster.

    I'll try to improve init.d to manage it
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  14. jmjosebest

    jmjosebest Member

    Joined:
    Jan 16, 2009
    Messages:
    135
    Likes Received:
    6
    Sorry for the off-topic
    Proxmox 3.0? where is the info of this release?
    Thanks!
     
  15. tom

    tom Proxmox Staff Member
    Staff Member

    Joined:
    Aug 29, 2006
    Messages:
    13,217
    Likes Received:
    363
    its not officially released (but it will be announce here)
    you can find some info in the wiki (search for wheezy).
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  16. badji

    badji Member

    Joined:
    Jan 14, 2011
    Messages:
    171
    Likes Received:
    4
    http://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Wheezy

    # pveversion -v
    pve-manager: 3.0-6 (pve-manager/3.0/af62c8a0)
    running kernel: 2.6.32-20-pve
    proxmox-ve-2.6.32: 3.0-96
    pve-kernel-2.6.32-20-pve: 2.6.32-96
    lvm2: 2.02.95-pve3
    clvm: 2.02.95-pve3
    corosync-pve: 1.4.5-1
    openais-pve: 1.1.4-3
    libqb: not correctly installed
    redhat-cluster-pve: 3.2.0-2
    resource-agents-pve: 3.9.2-4
    fence-agents-pve: 4.0.0-1
    pve-cluster: 3.0-1
    qemu-server: 3.0-2
    pve-firmware: 1.0-22
    libpve-common-perl: 3.0-1
    libpve-access-control: 3.0-1
    libpve-storage-perl: 3.0-1
    vncterm: 1.1-1
    vzctl: 4.0-1pve3
    vzprocps: 2.0.11-2
    vzquota: 3.1-2
    pve-qemu-kvm: 1.4-9
    ksm-control-daemon: 1.1-1
     
  17. badji

    badji Member

    Joined:
    Jan 14, 2011
    Messages:
    171
    Likes Received:
    4
    Hello,

    The rest of my tests:

    Everything works, but as soon as I turn off my servers and at the next startup of the cluster no vm sheepdog deployed is not available, they are still available on the Proxmox GUI but it does not work.
    I retested at several times by changing the configuration of sheepdog disk (iscsi, virtio) it is always the same worries.
    Thank you.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice