Open Directory not starting on XSAN metadata controller

tycen's picture

Help! Everything with our metadata controllers have been running fun. I have two MDC and am on XSAN 4.1/El Cap. I have added a client to the XSAN since the upgrade so I'm pretty sure nothing bad happened during the upgrade.

One of my MDC's has been acting funky - I think it's an issue with the 8 Promise Pegasus2 arrays I have connected it it via Thunderbolt - and I've had to do a few hard reboots over the last couple of weeks.

Well, yesterday I went to add a new client to the XSAN (we had to wipe and reinstall the OS on a previous client). It failed and the error led me to find that my MDC that's been giving me trouble doesn't have Open Directory running. OD won't start and from other searching it seems that the database has become corrupted. Running sudo /usr/libexec/slapd -Tt gives the error:
57880743 bdb(cn=authdata): /var/db/openldap/authdata/id2entry.bdb: unexpected file type or format
57880743 bdb_db_open: database "cn=authdata": db_open(/var/db/openldap/authdata/id2entry.bdb) failed: Invalid argument (22).
57880743 backend_startup_one (type=bdb, suffix="cn=authdata"): bi_db_open failed! (22)

I've tried recovering it using various methods posted on different forums but am having no luck at all. I did not know to be archiving my OD Master - so I do not have an archive.

The XSAN environment seems to be working fine otherwise - all the clients are happy. I just can't add the new client.

I'm only using OD for the XSAN environment - I don't use it for users or groups (the XSAN clients just have a local user).

Environment: 2 XSAN MDCs; 7 XSAN clients; all running 10.11.5.

Here's some questions:
1) is there anyway to add a client with the XSAN in this state?
2) Does Time Machine help me at all (I have had it running)?
3) How do I get this fixed long term?

Thanks for any help!

tycen's picture

I tried rolling back the MDC to a previous date via Time Machine (boot to Recovery, restore from Time Machine). It worked! Open Directory started on the master and I was able to add the client.

The only leftover issue I see is that it looks like on the 2nd MDC (the one that was the OD replica) it doesn't look like it's a replica anymore. It sees the master, but doesn't show itself as a replica.
https://www.dropbox.com/s/leyftrvtkztzpk6/Screenshot%202016-07-14%2017.2...

Any insights on the smoothest way to fix that? I'm going to leave it alone for a couple of days before messing with it.

And yes, the first thing I did was to Archive my Master OD. :)

anniegilson35's picture

Informational blog! This is a great example to search out particular topics in the given time frame to work out. Thumbs up for you
click now

Annie gilson