ceph ve 4.4 hilfe noout flag set

M

markus wenger

New Member
Dec 4, 2016
2
0
1
46
guten tag.

ich stolpere leider immer an der gleichen stelle.

ich habe 3 pve erstellt unter virtualisierung zum testen von ceph.
leider krieg ich immer einen fehler (noout flag(s) set und kann kein drive erstellen auf dem ceph.

habe 3 pve im cluster betrieb. alle identisch aufgebaut. klar ist die virtualisierung sehr langsam sollte aber ja trotzdem gehen.

mach ich da generell was falsch? danke für einen kleinen type

gruss markus

Informationen:


crush map:
# begin crush map
tunable choose_local_tries 0
tunable choose_local_fallback_tries 0
tunable choose_total_tries 50
tunable chooseleaf_descend_once 1
tunable straw_calc_version 1
# devices
device 0 osd.0
device 1 osd.1
device 2 osd.2
# types
type 0 osd
type 1 host
type 2 chassis
type 3 rack
type 4 row
type 5 pdu
type 6 pod
type 7 room
type 8 datacenter
type 9 region
type 10 root
# buckets
host vmpm1 {
id -2 # do not change unnecessarily
# weight 0.060
alg straw
hash 0 # rjenkins1
item osd.0 weight 0.060
}
host vmpm2 {
id -3 # do not change unnecessarily
# weight 0.060
alg straw
hash 0 # rjenkins1
item osd.1 weight 0.060
}
host vmpm3 {
id -4 # do not change unnecessarily
# weight 0.060
alg straw
hash 0 # rjenkins1
item osd.2 weight 0.060
}
root default {
id -1 # do not change unnecessarily
# weight 0.180
alg straw
hash 0 # rjenkins1
item vmpm1 weight 0.060
item vmpm2 weight 0.060
item vmpm3 weight 0.060
}
# rules
rule replicated_ruleset {
ruleset 0
type replicated
min_size 1
max_size 10
step take default
step chooseleaf firstn 0 type host
step emit
}
# end crush map

ceph config:
[global]
auth client required = cephx
auth cluster required = cephx
auth service required = cephx
cluster network = 172.17.19.0/24
filestore xattr use omap = true
fsid = 81916a3b-1008-48ab-bcdd-401bd4414336
keyring = /etc/pve/priv/$cluster.$name.keyring
osd journal size = 5120
osd pool default min size = 1
public network = 172.17.19.0/24
[osd]
keyring = /var/lib/ceph/osd/ceph-$id/keyring
[mon.0]
host = vmpm1
mon addr = 172.17.19.231:6789
[mon.2]
host = vmpm3
mon addr = 172.17.19.233:6789
[mon.1]
host = vmpm2
mon addr = 172.17.19.232:6789

ceph log:
2017-03-13 15:11:21.179864 mon.0 172.17.19.231:6789/0 215 : cluster [WRN] message from mon.2 was stamped 0.223479s in the future, clocks not synchronized
2017-03-13 15:11:27.236264 mon.0 172.17.19.231:6789/0 216 : cluster [WRN] message from mon.1 was stamped 0.138095s in the future, clocks not synchronized
2017-03-13 15:11:54.295243 mon.0 172.17.19.231:6789/0 217 : cluster [WRN] message from mon.1 was stamped 0.138301s in the future, clocks not synchronized
2017-03-13 15:11:59.729477 mon.1 172.17.19.232:6789/0 60 : cluster [WRN] message from mon.0 was stamped 334.967849s in the future, clocks not synchronized
2017-03-13 15:11:59.814990 mon.2 172.17.19.233:6789/0 67 : cluster [WRN] message from mon.0 was stamped 334.882368s in the future, clocks not synchronized
2017-03-13 15:17:34.701446 mon.0 172.17.19.231:6789/0 218 : cluster [WRN] mon.2 172.17.19.233:6789/0 clock skew 334.883s > max 0.05s
2017-03-13 15:17:34.701543 mon.0 172.17.19.231:6789/0 219 : cluster [WRN] mon.1 172.17.19.232:6789/0 clock skew 334.968s > max 0.05s
2017-03-13 15:12:26.886419 mon.2 172.17.19.233:6789/0 68 : cluster [WRN] message from mon.0 was stamped 334.882450s in the future, clocks not synchronized
2017-03-13 15:18:04.702711 mon.0 172.17.19.231:6789/0 220 : cluster [WRN] mon.1 172.17.19.232:6789/0 clock skew 334.968s > max 0.05s
2017-03-13 15:18:04.702743 mon.0 172.17.19.231:6789/0 221 : cluster [WRN] mon.2 172.17.19.233:6789/0 clock skew 334.882s > max 0.05s
2017-03-13 15:12:45.052146 mon.1 172.17.19.232:6789/0 61 : cluster [INF] mon.1 calling new monitor election
2017-03-13 15:18:20.018846 mon.2 172.17.19.233:6789/0 69 : cluster [INF] mon.2 calling new monitor election
2017-03-13 15:18:20.020472 mon.0 172.17.19.231:6789/0 222 : cluster [INF] mon.0 calling new monitor election
2017-03-13 15:18:20.023571 mon.0 172.17.19.231:6789/0 223 : cluster [INF] mon.0@0 won leader election with quorum 0,1,2
2017-03-13 15:18:20.025712 mon.0 172.17.19.231:6789/0 224 : cluster [INF] HEALTH_WARN; noout flag(s) set
2017-03-13 15:18:20.026744 mon.0 172.17.19.231:6789/0 225 : cluster [WRN] mon.1 172.17.19.232:6789/0 clock skew 334.967s > max 0.05s
2017-03-13 15:18:20.028404 mon.0 172.17.19.231:6789/0 226 : cluster [INF] monmap e7: 3 mons at {0=172.17.19.231:6789/0,1=172.17.19.232:6789/0,2=172.17.19.233:6789/0}
2017-03-13 15:18:20.028485 mon.0 172.17.19.231:6789/0 227 : cluster [INF] pgmap v221: 62 pgs: 62 active+clean; 0 bytes data, 104 MB used, 179 GB / 179 GB avail
2017-03-13 15:18:20.028555 mon.0 172.17.19.231:6789/0 228 : cluster [INF] mdsmap e1: 0/0/0 up
2017-03-13 15:18:20.028642 mon.0 172.17.19.231:6789/0 229 : cluster [INF] osdmap e115: 3 osds: 3 up, 3 in
2017-03-13 15:18:27.749286 mon.1 172.17.19.232:6789/0 62 : cluster [INF] mon.1 calling new monitor election
2017-03-13 15:18:27.751654 mon.0 172.17.19.231:6789/0 230 : cluster [INF] mon.0 calling new monitor election
2017-03-13 15:18:27.754303 mon.0 172.17.19.231:6789/0 231 : cluster [INF] mon.0@0 won leader election with quorum 0,1,2
2017-03-13 15:18:27.756052 mon.0 172.17.19.231:6789/0 232 : cluster [INF] HEALTH_WARN; noout flag(s) set
2017-03-13 15:18:27.758784 mon.0 172.17.19.231:6789/0 233 : cluster [INF] monmap e7: 3 mons at {0=172.17.19.231:6789/0,1=172.17.19.232:6789/0,2=172.17.19.233:6789/0}
2017-03-13 15:18:27.758870 mon.0 172.17.19.231:6789/0 234 : cluster [INF] pgmap v221: 62 pgs: 62 active+clean; 0 bytes data, 104 MB used, 179 GB / 179 GB avail
2017-03-13 15:18:27.759050 mon.0 172.17.19.231:6789/0 235 : cluster [INF] mdsmap e1: 0/0/0 up
2017-03-13 15:18:27.759146 mon.0 172.17.19.231:6789/0 236 : cluster [INF] osdmap e115: 3 osds: 3 up, 3 in
2017-03-13 15:39:29.684502 mon.0 172.17.19.231:6789/0 7 : cluster [INF] mon.0 calling new monitor election
2017-03-13 15:39:29.685422 mon.1 172.17.19.232:6789/0 1 : cluster [INF] mon.1 calling new monitor election
2017-03-13 15:39:29.687368 mon.0 172.17.19.231:6789/0 8 : cluster [INF] mon.0 calling new monitor election
2017-03-13 15:39:34.689481 mon.0 172.17.19.231:6789/0 19 : cluster [INF] mon.0@0 won leader election with quorum 0,1
2017-03-13 15:39:34.690693 mon.0 172.17.19.231:6789/0 20 : cluster [INF] HEALTH_WARN; noout flag(s) set; 1 mons down, quorum 0,1 0,1
2017-03-13 15:39:34.696617 mon.0 172.17.19.231:6789/0 21 : cluster [INF] monmap e7: 3 mons at {0=172.17.19.231:6789/0,1=172.17.19.232:6789/0,2=172.17.19.233:6789/0}
2017-03-13 15:39:34.696727 mon.0 172.17.19.231:6789/0 22 : cluster [INF] pgmap v221: 62 pgs: 62 active+clean; 0 bytes data, 104 MB used, 179 GB / 179 GB avail
2017-03-13 15:39:34.696797 mon.0 172.17.19.231:6789/0 23 : cluster [INF] mdsmap e1: 0/0/0 up
2017-03-13 15:39:34.696885 mon.0 172.17.19.231:6789/0 24 : cluster [INF] osdmap e115: 3 osds: 3 up, 3 in
2017-03-13 15:39:32.419953 mon.2 172.17.19.233:6789/0 1 : cluster [INF] mon.2 calling new monitor election
2017-03-13 15:39:37.426378 mon.0 172.17.19.231:6789/0 27 : cluster [INF] mon.0 calling new monitor election
2017-03-13 15:39:37.426749 mon.1 172.17.19.232:6789/0 14 : cluster [INF] mon.1 calling new monitor election
2017-03-13 15:39:37.429640 mon.0 172.17.19.231:6789/0 28 : cluster [INF] mon.0@0 won leader election with quorum 0,1,2
2017-03-13 15:39:37.431456 mon.0 172.17.19.231:6789/0 29 : cluster [INF] HEALTH_WARN; noout flag(s) set
2017-03-13 15:39:37.435191 mon.0 172.17.19.231:6789/0 30 : cluster [INF] monmap e7: 3 mons at {0=172.17.19.231:6789/0,1=172.17.19.232:6789/0,2=172.17.19.233:6789/0}
2017-03-13 15:39:37.435277 mon.0 172.17.19.231:6789/0 31 : cluster [INF] pgmap v221: 62 pgs: 62 active+clean; 0 bytes data, 104 MB used, 179 GB / 179 GB avail
2017-03-13 15:39:37.435355 mon.0 172.17.19.231:6789/0 32 : cluster [INF] mdsmap e1: 0/0/0 up
2017-03-13 15:39:37.435449 mon.0 172.17.19.231:6789/0 33 : cluster [INF] osdmap e115: 3 osds: 3 up, 3 in
2017-03-13 15:55:49.764284 mon.1 172.17.19.232:6789/0 1 : cluster [INF] mon.1 calling new monitor election
2017-03-13 15:55:50.161074 mon.0 172.17.19.231:6789/0 11 : cluster [INF] mon.0 calling new monitor election
2017-03-13 15:55:54.735535 mon.1 172.17.19.232:6789/0 12 : cluster [WRN] message from mon.0 was stamped 0.430899s in the future, clocks not synchronized
2017-03-13 15:55:55.163409 mon.0 172.17.19.231:6789/0 22 : cluster [INF] mon.0@0 won leader election with quorum 0,1
2017-03-13 15:55:55.164692 mon.0 172.17.19.231:6789/0 23 : cluster [INF] HEALTH_WARN; noout flag(s) set; 1 mons down, quorum 0,1 0,1
2017-03-13 15:55:55.166775 mon.0 172.17.19.231:6789/0 24 : cluster [INF] monmap e7: 3 mons at {0=172.17.19.231:6789/0,1=172.17.19.232:6789/0,2=172.17.19.233:6789/0}
2017-03-13 15:55:55.166824 mon.0 172.17.19.231:6789/0 25 : cluster [INF] pgmap v221: 62 pgs: 62 active+clean; 0 bytes data, 104 MB used, 179 GB / 179 GB avail
2017-03-13 15:55:55.166862 mon.0 172.17.19.231:6789/0 26 : cluster [INF] mdsmap e1: 0/0/0 up
2017-03-13 15:55:55.166911 mon.0 172.17.19.231:6789/0 27 : cluster [INF] osdmap e115: 3 osds: 3 up, 3 in
2017-03-13 15:55:55.167346 mon.0 172.17.19.231:6789/0 28 : cluster [WRN] mon.1 172.17.19.232:6789/0 clock skew 0.430321s > max 0.05s
2017-03-13 15:55:56.246871 mon.0 172.17.19.231:6789/0 34 : cluster [INF] osdmap e116: 3 osds: 1 up, 3 in
2017-03-13 15:55:56.250523 mon.0 172.17.19.231:6789/0 35 : cluster [INF] pgmap v222: 62 pgs: 39 stale+active+clean, 23 active+clean; 0 bytes data, 104 MB used, 179 GB / 179 GB avail
2017-03-13 15:55:51.922502 mon.2 172.17.19.233:6789/0 1 : cluster [INF] mon.2 calling new monitor election
2017-03-13 15:55:56.843892 mon.1 172.17.19.232:6789/0 17 : cluster [INF] mon.1 calling new monitor election
2017-03-13 15:55:56.956903 mon.2 172.17.19.233:6789/0 12 : cluster [WRN] message from mon.0 was stamped 0.330619s in the future, clocks not synchronized
2017-03-13 15:55:57.275727 mon.0 172.17.19.231:6789/0 40 : cluster [INF] mon.0 calling new monitor election
2017-03-13 15:55:57.278981 mon.0 172.17.19.231:6789/0 41 : cluster [INF] mon.0@0 won leader election with quorum 0,1,2
2017-03-13 15:55:57.280349 mon.0 172.17.19.231:6789/0 42 : cluster [INF] HEALTH_WARN; 39 pgs stale; noout flag(s) set
2017-03-13 15:55:57.287813 mon.0 172.17.19.231:6789/0 43 : cluster [INF] monmap e7: 3 mons at {0=172.17.19.231:6789/0,1=172.17.19.232:6789/0,2=172.17.19.233:6789/0}
2017-03-13 15:55:57.287928 mon.0 172.17.19.231:6789/0 44 : cluster [INF] pgmap v223: 62 pgs: 39 stale+active+clean, 23 active+clean; 0 bytes data, 104 MB used, 179 GB / 179 GB avail
2017-03-13 15:55:57.288018 mon.0 172.17.19.231:6789/0 45 : cluster [INF] mdsmap e1: 0/0/0 up
2017-03-13 15:55:57.288133 mon.0 172.17.19.231:6789/0 46 : cluster [INF] osdmap e117: 3 osds: 3 up, 3 in
2017-03-13 15:55:57.291897 mon.0 172.17.19.231:6789/0 47 : cluster [WRN] mon.2 172.17.19.233:6789/0 clock skew 0.340582s > max 0.05s
2017-03-13 15:55:57.292105 mon.0 172.17.19.231:6789/0 48 : cluster [WRN] mon.1 172.17.19.232:6789/0 clock skew 0.429905s > max 0.05s
2017-03-13 15:55:58.279932 mon.0 172.17.19.231:6789/0 49 : cluster [INF] osdmap e118: 3 osds: 2 up, 3 in
2017-03-13 15:55:58.290749 mon.0 172.17.19.231:6789/0 50 : cluster [INF] pgmap v224: 62 pgs: 62 stale+active+clean; 0 bytes data, 104 MB used, 179 GB / 179 GB avail
2017-03-13 15:55:59.273997 mon.0 172.17.19.231:6789/0 51 : cluster [INF] osd.1 172.17.19.232:6800/1774 boot
2017-03-13 15:55:59.280360 mon.0 172.17.19.231:6789/0 52 : cluster [INF] osdmap e119: 3 osds: 3 up, 3 in
2017-03-13 15:55:59.283557 mon.0 172.17.19.231:6789/0 53 : cluster [INF] pgmap v225: 62 pgs: 62 stale+active+clean; 0 bytes data, 104 MB used, 179 GB / 179 GB avail
2017-03-13 15:55:59.863697 mon.1 172.17.19.232:6789/0 18 : cluster [WRN] message from mon.0 was stamped 0.430858s in the future, clocks not synchronized
2017-03-13 15:56:00.306027 mon.0 172.17.19.231:6789/0 54 : cluster [INF] osdmap e120: 3 osds: 3 up, 3 in
2017-03-13 15:56:00.309420 mon.0 172.17.19.231:6789/0 55 : cluster [INF] pgmap v226: 62 pgs: 62 stale+active+clean; 0 bytes data, 104 MB used, 179 GB / 179 GB avail
2017-03-13 15:56:02.044024 mon.2 172.17.19.233:6789/0 15 : cluster [WRN] message from mon.0 was stamped 0.340620s in the future, clocks not synchronized
2017-03-13 15:56:02.385261 mon.0 172.17.19.231:6789/0 56 : cluster [INF] pgmap v227: 62 pgs: 62 active+clean; 0 bytes data, 104 MB used, 179 GB / 179 GB avail
2017-03-13 15:56:11.262173 mon.0 172.17.19.231:6789/0 57 : cluster [WRN] message from mon.2 was stamped 0.577219s in the future, clocks not synchronized
2017-03-13 15:56:57.280583 mon.0 172.17.19.231:6789/0 60 : cluster [INF] HEALTH_WARN; noout flag(s) set
2017-03-13 15:58:01.410214 mon.0 172.17.19.231:6789/0 63 : cluster [INF] pgmap v228: 62 pgs: 62 active+clean; 0 bytes data, 104 MB used, 179 GB / 179 GB avail
2017-03-13 15:58:02.414306 mon.0 172.17.19.231:6789/0 64 : cluster [INF] pgmap v229: 62 pgs: 62 active+clean; 0 bytes data, 105 MB used, 179 GB / 179 GB avail
 

Attachments

  • Voila_Capture 2017-03-13_04-49-57_PM.png
    Voila_Capture 2017-03-13_04-49-57_PM.png
    55.2 KB · Views: 8
  • Voila_Capture 2017-03-13_04-57-40_PM.png
    Voila_Capture 2017-03-13_04-57-40_PM.png
    42.2 KB · Views: 8
Offenbar ist das "noout" flag gesetzt.

=> dieses flag kann man über das GUI wieder deaktivieren (=> unset noout)
 
Hallo tom. Na ja bin halt im diesem Bereich noch recht grün hinter den Ohren. Dank dir für die Prompte Hilfe.
Habe nun das Drive einbinden können jedoch noch ohne Schlüssel da muss ich nochmals Testen.

Besten Dank Gruss aus der Schweiz Markus
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back