[SOLVED] 4.0beta-39 ceph mon creation failure

[Q-wulf]

I am trying to set up a 3 node proxmox with ceph cluster on a Virtual Box as a testing environment for a planned physical Cluster.

I followed this guide:
https:*/pve.proxmox.com/wiki/Ceph_Server

~# pveceph init --network 10.2.0.0/24
~# pveceph createmon
command 'monmaptool --create --clobber --add 0 10.2.0.1:6789 --print /tmp/monmap' failed: open3: exec of monmaptool --create --clobber --add 0 10.2.0.1:6789 --print /tmp/monmap failed at /usr/share/perl5/PVE/Tools.pm line 374.

Is this an beta issue, or am i doing the setup wrong ?


for reference:


~# pveversion -v

proxmox-ve: 4.0-13 (running kernel: 4.2.0-1-pve)
pve-manager: 4.0-39 (running version: 4.0-39/ab3cc94a)
pve-kernel-4.2.0-1-pve: 4.2.0-13
lvm2: 2.02.116-pve1
corosync-pve: 2.3.5-1
libqb0: 0.17.2-1
pve-cluster: 4.0-19
qemu-server: 4.0-25
pve-firmware: 1.1-7
libpve-common-perl: 4.0-24
libpve-access-control: 4.0-8
libpve-storage-perl: 4.0-23
pve-libspice-server1: 0.12.5-1
vncterm: 1.2-1
pve-qemu-kvm: 2.4-8
pve-container: 0.9-23
pve-firewall: 2.0-11
pve-ha-manager: 1.0-7
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u1
lxc-pve: 1.1.3-1
lxcfs: 0.9-pve2
cgmanager: 0.37-pve2
criu: 1.6.0-1
zfsutils: 0.6.5-pve1~jessie
openvswitch-switch: 2.3.2-1

cat /etc/network/interfaces

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage part of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!


allow-vmbr0 vlan1
iface vlan1 inet static
address 10.1.0.1
netmask 255.255.255.0
ovs_type OVSIntPort
ovs_bridge vmbr0
ovs_options tag=1
ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
mtu 1500


allow-vmbr0 vlan2
iface vlan2 inet static
address 10.2.0.1
netmask 255.255.255.0
ovs_type OVSIntPort
ovs_bridge vmbr0
ovs_options tag=2
ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
mtu 9000


auto lo
iface lo inet loopback


auto eth2
iface eth2 inet manual


auto eth3
iface eth3 inet manual


auto eth1
iface eth1 inet manual


auto eth0
iface eth0 inet static
address 192.168.2.101
netmask 255.255.255.0
gateway 192.168.2.1


allow-vmbr0 bond0
iface bond0 inet manual
ovs_bonds eth1 eth2 eth3
ovs_type OVSBond
ovs_bridge vmbr0
ovs_options lacp=active bond_mode=balance-tcp other_config:lacp-time=fast
pre-up ( ifconfig eth1 mtu 9000 && ifconfig eth2 mtu 9000 && ifconfig eth3 mtu 9000)
mtu 9000
#Forcing the physical interfaces to accept Jumbo Frames
# OVSIntPorts dont nessesarily need to use Jumbo Frames


auto vmbr0
iface vmbr0 inet manual
ovs_type OVSBridge
ovs_ports bond0 vlan1 vlan2
mtu 9000

cat /etc/hosts

127.0.0.1 localhost.localdomain localhost
10.1.0.1 proxmox1.datamile.local proxmox1 pvelocalhost


# The following lines are desirable for IPv6 capable hosts


::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

 

[Q-wulf]

I noticed the following errors popping up on my second round of setting the cluster up:

pvecm nodes

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = "UTF-8",
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").

possibly was caused by installing

apt-get install openvswitch-switch
(only manually installed package, that was not in the install / cluster guide for proxmox4.x )


Fix seems to be:

export LANGUAGE=en_US.UTF-8
export LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8
locale-gen en_US.UTF-8
dpkg-reconfigure locales

needs to be run every single node
Edit: does not retain the config after a node restart... so not a fix

permanent fix is to add the following lines to /root/.bashrc on every node:

export LANGUAGE=en_US.UTF-8
export LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8

 

[bbrendon]

I'm getting this error as well on a new install of 4.4 but updated basrc and reconnecting the session didn't fix it.

 

[tom]

I'm getting this error as well on a new install of 4.4 but updated basrc and reconnecting the session didn't fix it.

maybe this helps?

https://www.thomas-krenn.com/en/wiki/Perl_warning_Setting_locale_failed_in_Debian

 

[bbrendon]

I'm not sure what the solution was because I ran into a lot of strange issues getting this setup in the lab but I _think_ the solution to this one was for some reason the ceph-base package wasn't installed.