Error: I/O error while accessing persistent configuration storage

C

cdukes

Renowned Member
Sep 11, 2015
89
6
73
Raleigh, NC
www.logzilla.net
I keep getting this error when trying to create my first drbd disk(s). Can anyone please explain how I can fix it?

drbdmanage add-node ucs 10.0.28.201
Operation completed successfully
Operation completed successfully


Executing join command using ssh.
IMPORTANT: The output you see comes from ucs
IMPORTANT: Your input is executed on ucs
You are going to join an existing drbdmanage cluster.
CAUTION! Note that:
* Any previous drbdmanage cluster information may be removed
* Any remaining resources managed by a previous drbdmanage installation
that still exist on this system will no longer be managed by drbdmanage


Confirm:


yes/no: yes
Logical volume ".drbdctrl" successfully removed
Logical volume ".drbdctrl" created.
You want me to create a v09 style flexible-size internal meta data block.
There appears to be a v09 flexible-size internal meta data block
already in place on /dev/drbdpool/.drbdctrl at byte offset 4190208


Do you really want to overwrite the existing meta-data?
*** confirmation forced via --force option ***
NOT initializing bitmap
initializing activity log
Writing meta data...
New drbd meta data block successfully created.
Error: I/O error while accessing persistent configuration storage
 
Last edited:
I just noticed the following in the 2nd server's log after I run drbdmanage new-node ucs 10.0.28.201

Sep 21 20:56:53 ucs kernel: [890029.220380] drbd .drbdctrl bfs: Terminating sender thread
Sep 21 20:56:53 ucs kernel: [890029.253037] drbd .drbdctrl/0 drbd0: disk( Inconsistent -> Detaching )
Sep 21 20:56:53 ucs kernel: [890029.253070] drbd .drbdctrl/0 drbd0: disk( Detaching -> Diskless )
Sep 21 20:56:53 ucs kernel: [890029.253082] drbd .drbdctrl/0 drbd0: drbd_bm_resize called with capacity == 0
Sep 21 20:56:53 ucs kernel: [890029.324997] drbd .drbdctrl: Terminating worker thread
Sep 21 20:56:54 ucs kernel: [890030.184384] drbd .drbdctrl: Starting worker thread (from drbdsetup [13403])
Sep 21 20:56:55 ucs kernel: [890030.388995] drbd .drbdctrl/0 drbd0: disk( Diskless -> Attaching )
Sep 21 20:56:55 ucs kernel: [890030.389011] drbd .drbdctrl/0 drbd0: Maximum number of peer devices = 31
Sep 21 20:56:55 ucs kernel: [890030.389110] drbd .drbdctrl: Method to ensure write ordering: flush
Sep 21 20:56:55 ucs kernel: [890030.389116] drbd .drbdctrl/0 drbd0: drbd_bm_resize called with capacity == 8112
Sep 21 20:56:55 ucs kernel: [890030.389125] drbd .drbdctrl/0 drbd0: resync bitmap: bits=1014 words=496 pages=1
Sep 21 20:56:55 ucs kernel: [890030.389261] drbd .drbdctrl/0 drbd0: recounting of set bits took additional 0ms
Sep 21 20:56:55 ucs kernel: [890030.389274] drbd .drbdctrl/0 drbd0: disk( Attaching -> Inconsistent )
Sep 21 20:56:55 ucs kernel: [890030.389278] drbd .drbdctrl/0 drbd0: attached to current UUID: 0000000000000004
Sep 21 20:56:55 ucs kernel: [890030.395554] drbd .drbdctrl bfs: Starting sender thread (from drbdsetup [13417])
Sep 21 20:56:55 ucs kernel: [890030.406618] drbd .drbdctrl bfs: conn( StandAlone -> Unconnected )
Sep 21 20:56:55 ucs kernel: [890030.407761] drbd .drbdctrl bfs: Starting receiver thread (from drbd_w_.drbdctr [13405])
Sep 21 20:56:55 ucs kernel: [890030.407900] drbd .drbdctrl bfs: conn( Unconnected -> Connecting )
Sep 21 20:56:55 ucs kernel: [890030.409998] drbd .drbdctrl: State change failed: Need access to UpToDate data
Sep 21 20:56:55 ucs kernel: [890030.410082] drbd .drbdctrl: Failed: role( Secondary -> Primary )
Sep 21 20:56:55 ucs kernel: [890030.410085] drbd .drbdctrl: Auto-promote failed: Need access to UpToDate data
Sep 21 20:56:55 ucs kernel: [890030.906404] drbd .drbdctrl bfs: Handshake successful: Agreed network protocol version 110
Sep 21 20:56:55 ucs kernel: [890030.906408] drbd .drbdctrl bfs: Agreed to support TRIM on protocol level
Sep 21 20:56:55 ucs kernel: [890030.906600] drbd .drbdctrl bfs: Peer authenticated using 32 bytes HMAC
Sep 21 20:56:55 ucs kernel: [890030.906619] drbd .drbdctrl bfs: Starting ack_recv thread (from drbd_r_.drbdctr [13423])
Sep 21 20:56:55 ucs kernel: [890030.916877] drbd .drbdctrl bfs: incompatible after-sb-0pri settings
Sep 21 20:56:55 ucs kernel: [890030.916950] drbd .drbdctrl bfs: conn( Connecting -> Disconnecting )
Sep 21 20:56:55 ucs kernel: [890030.916979] drbd .drbdctrl bfs: error receiving P_PROTOCOL, e: -5 l: 1!
Sep 21 20:56:55 ucs kernel: [890030.917056] drbd .drbdctrl bfs: ack_receiver terminated
Sep 21 20:56:55 ucs kernel: [890030.917058] drbd .drbdctrl bfs: Terminating ack_recv thread
Sep 21 20:56:55 ucs kernel: [890030.964934] drbd .drbdctrl bfs: Connection closed
Sep 21 20:56:55 ucs kernel: [890030.964982] drbd .drbdctrl bfs: conn( Disconnecting -> StandAlone )
Sep 21 20:56:55 ucs kernel: [890030.965017] drbd .drbdctrl bfs: Terminating receiver thread
Sep 21 20:56:56 ucs kernel: [890031.451268] drbd .drbdctrl: State change failed: Need access to UpToDate data
Sep 21 20:56:56 ucs kernel: [890031.451355] drbd .drbdctrl: Failed: role( Secondary -> Primary )
Sep 21 20:56:56 ucs kernel: [890031.451359] drbd .drbdctrl: Auto-promote failed: Need access to UpToDate data
Sep 21 20:56:57 ucs kernel: [890032.432546] drbd .drbdctrl: State change failed: Need access to UpToDate data
Sep 21 20:56:57 ucs kernel: [890032.432633] drbd .drbdctrl: Failed: role( Secondary -> Primary )
Sep 21 20:56:57 ucs kernel: [890032.432637] drbd .drbdctrl: Auto-promote failed: Need access to UpToDate data
Sep 21 20:56:58 ucs kernel: [890033.824265] drbd .drbdctrl: State change failed: Need access to UpToDate data
Sep 21 20:56:58 ucs kernel: [890033.824353] drbd .drbdctrl: Failed: role( Secondary -> Primary )
Sep 21 20:56:58 ucs kernel: [890033.824356] drbd .drbdctrl: Auto-promote failed: Need access to UpToDate data
Sep 21 20:57:01 ucs kernel: [890036.587404] drbd .drbdctrl: State change failed: Need access to UpToDate data
Sep 21 20:57:01 ucs kernel: [890036.587493] drbd .drbdctrl: Failed: role( Secondary -> Primary )
Sep 21 20:57:01 ucs kernel: [890036.587497] drbd .drbdctrl: Auto-promote failed: Need access to UpToDate data
Sep 21 20:57:02 ucs kernel: [890038.148296] drbd .drbdctrl: State change failed: Need access to UpToDate data
Sep 21 20:57:02 ucs kernel: [890038.148384] drbd .drbdctrl: Failed: role( Secondary -> Primary )
Sep 21 20:57:02 ucs kernel: [890038.148388] drbd .drbdctrl: Auto-promote failed: Need access to UpToDate data
Sep 21 20:57:03 ucs kernel: [890038.689072] drbd .drbdctrl: State change failed: Need access to UpToDate data
Sep 21 20:57:03 ucs kernel: [890038.689155] drbd .drbdctrl: Failed: role( Secondary -> Primary )
Sep 21 20:57:03 ucs kernel: [890038.689172] drbd .drbdctrl: Auto-promote failed: Need access to UpToDate data
Sep 21 20:57:05 ucs kernel: [890040.461145] drbd .drbdctrl: State change failed: Need access to UpToDate data
Sep 21 20:57:05 ucs kernel: [890040.461234] drbd .drbdctrl: Failed: role( Secondary -> Primary )
Sep 21 20:57:05 ucs kernel: [890040.461252] drbd .drbdctrl: Auto-promote failed: Need access to UpToDate data
Sep 21 20:57:06 ucs kernel: [890041.662641] drbd .drbdctrl: State change failed: Need access to UpToDate data
Sep 21 20:57:06 ucs kernel: [890041.662729] drbd .drbdctrl: Failed: role( Secondary -> Primary )
Sep 21 20:57:06 ucs kernel: [890041.662733] drbd .drbdctrl: Auto-promote failed: Need access to UpToDate data
Sep 21 20:57:07 ucs kernel: [890042.613894] drbd .drbdctrl: State change failed: Need access to UpToDate data
Sep 21 20:57:07 ucs kernel: [890042.614012] drbd .drbdctrl: Failed: role( Secondary -> Primary )
Sep 21 20:57:07 ucs kernel: [890042.614016] drbd .drbdctrl: Auto-promote failed: Need access to UpToDate data
Sep 21 20:57:08 ucs drbdmanaged[11690]: ERROR cannot open control volume '/dev/drbd0' (10 failed attempts)

So it seems the problem is about not being able to open /dev/drbd0. I checked and it is there, but I *thought* the control volume is supposed to be that .drbdctrl lvm that is created?
 
Last edited:
On the remote host, the syslog shows:
Code: 
Sep 21 21:47:44 ucs kernel: [893079.620321] drbd .drbdctrl: State change failed: Need access to UpToDate data
Code: 
Sep 21 21:47:44 ucs kernel: [893079.620409] drbd .drbdctrl: Failed: role( Secondary -> Primary )
Code: 
Sep 21 21:47:44 ucs kernel: [893079.620413] drbd .drbdctrl: Auto-promote failed: Need access to UpToDate data
Code: 
Sep 21 21:47:45 ucs kernel: [893080.821808] drbd .drbdctrl: State change failed: Need access to UpToDate data
Code: 
Sep 21 21:47:45 ucs kernel: [893080.821896] drbd .drbdctrl: Failed: role( Secondary -> Primary )
Code: 
Sep 21 21:47:45 ucs kernel: [893080.821915] drbd .drbdctrl: Auto-promote failed: Need access to UpToDate data
Code: 
Sep 21 21:47:47 ucs kernel: [893082.944299] drbd .drbdctrl: State change failed: Need access to UpToDate data
Code: 
Sep 21 21:47:47 ucs kernel: [893082.944388] drbd .drbdctrl: Failed: role( Secondary -> Primary )
Code: 
Sep 21 21:47:47 ucs kernel: [893082.944392] drbd .drbdctrl: Auto-promote failed: Need access to UpToDate data
Code: 
Sep 21 21:47:48 ucs drbdmanaged[11690]: ERROR      cannot open control volume '/dev/drbd0' (10 failed attempts)
So it seems the problem is about not being able to open /dev/drbd0. I checked and it is there, but I *thought* the control volume is supposed to be that .drbdctrl lvm that is created? P.S., I had to put [code tags on every line because I guess this forum software doesn't allow line breaks inside the code tags? odd...
 
Hi,

I've got the same " Error: I/O error while accessing persistent configuration storage" - plain PVE 4.0 with testing repository following the proxmon drbd9 howto.

did you found a solution?

Regards,
Tom
 
I've alredy installed 0.50.2. :(
Code: 
root@px1:~# dpkg -l | grep '^ii' | grep drbdii  drbd-utils                     8.9.4-1                        amd64        RAID 1 over TCP/IP for Linux (user utilities)ii  drbdmanage                     0.50-2                         amd64        Distributed configuration management for DRBD
 
I do this after package installation - I've installed only this package, because drbdmanage was no tavailable before. Thats why I have made my first post here. I added the test repository and do "apt-get install drbdmanage" - I follow step-by-step the drbd9 howto.
 
plepps said:
I do this after package installation - I've installed only this package, because drbdmanage was no tavailable before.
Click to expand...

That is strange, because the output you posted is from an old version (0.49):

Code: 
[COLOR=#333333][FONT=courier new]Logical volume ".drbdctrl" successfully removed[/FONT][/COLOR]
[COLOR=#333333][FONT=courier new]Logical volume ".drbdctrl" created.
[/FONT][/COLOR]


new 0.50 creates 2 different drbdctrl volumes instead.
The output from new version looks like:
Code: 
Logical volume ".drbdctrl_0" created.          
Logical volume ".drbdctrl_1" created.
 
plepps said:
thats not my output, I've got only the same error message as the thread starter.
Click to expand...


It was my output, but I have upgraded the servers today and still get the same error:

Code: 
root@bfs:~# drbdmanage delete-node ucs
Code: 
You are going to remove the node 'ucs' from the cluster. This will remove all resources from the node.
Please confirm:
  yes/no: yes
Code: 
Operation completed successfully
root@bfs:~# drbdmanage new-node ucs 10.0.28.201
Operation completed successfully
Operation completed successfully
tput: No value for $TERM and no -T specified
tput: No value for $TERM and no -T specified
tput: No value for $TERM and no -T specified

Executing join command using ssh.
IMPORTANT: The output you see comes from ucs
IMPORTANT: Your input is executed on ucs
tput: No value for $TERM and no -T specified
tput: No value for $TERM and no -T specified
tput: No value for $TERM and no -T specified
You are going to join an existing drbdmanage cluster.
CAUTION! Note that:
  * Any previous drbdmanage cluster information may be removed
  * Any remaining resources managed by a previous drbdmanage installation
    that still exist on this system will no longer be managed by drbdmanage

Confirm:

  yes/no: yes
  Logical volume ".drbdctrl" successfully removed
  Logical volume ".drbdctrl" created.
You want me to create a v09 style flexible-size internal meta data block.
There appears to be a v09 flexible-size internal meta data block
already in place on /dev/drbdpool/.drbdctrl at byte offset 4190208

Do you really want to overwrite the existing meta-data?
*** confirmation forced via --force option ***
NOT initializing bitmap
initializing activity log
Writing meta data...
New drbd meta data block successfully created.
Error: I/O error while accessing persistent configuration storage
root@bfs:~#
 
Hi,

my problem seems to be solved:
however on node 1 was the public address and not ip from storrage interface in /etc/drbd.d/drbdctrl.res.
After made the changes I rebooted all nodes and drbd creation works.

cheers,
tom
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom