LJDT: LDAP Data and Schema Comparisons

By: ab

August 7, 2013 2:53 pm





Have you ever wanted to compare data between eDirectory trees, or schema between servers, or objects between entirely different directories (OpenLDAP and eDirectory, for example)? It should be possible, of course, as both systems are accessible via LDAP. Still, how does one go about doing this kind of comparison? Well, because you are running on the right server or workstation, Linux Just Does That.

This is part of a series designed to show what can be done in the real world, with nothing more than a Linux box, out of the box. While many of these topics may cover topics outside what your mother or sister may want to do on a computer, the reality is that we are not them, and we have work to do. Because we have work to do, we choose Linux because it is able to get the work done. It just does that. For more on this series, search on the various NetIQ/Novell/SUSE Communities sites for ‘LJDT’ articles. It is possible that some do not apply to your “day job”, but their focus is more about skills and the default tools that come from a great operating system, so hopefully something can be gleaned from each one.

Included with SLE/openSUSE installations is a package called openldap2-client which has all of your favorite (if you know about them) LDAP tools; ldapsearch, ldapmodify, and several others are all included. These tools are from the OpenLDAP set of packages, but since LDAP is a standard they work nicely against other implementations which comply with the LDAP standards, like eDirectory has for who-knows-how-many years. The tool I use primarily is ldapsearch which makes querying a directory very simple. ldapmodify is the only other tool I use regularly as you can add/modify/delete/move/rename with this single tool assuming you have the ‘changetype’ correctly specified in the input data. ldapcompare is also a tool that is part of this package, but I only mention it right now to let you know that it is NOT what you want to compare schema. ldapcompare’s purpose is to compare an attribute on an object with the hope that the query will return TRUE or FALSE (permissions allowing) and is not to compare multiple objects together.

It seems appropriate at this point to mention some other tools that will be used. First, LDAP output (LDAP Data Interchange Format (LDIF)) is regularly folded which means that lines longer than seventy-eight characters are wrapped to the next line. This is not line wrapping, because it actually happens (vs. just being a neat trick of the editor automatically moving text to the next line) and because it is done by prepending the next line(s) with a single space character (0x20). An example of this type of output follows:

dn: cn=server,ou=context,o=goes,dc=here

publicKey:: MIIFODCCBCCgAwIBAgIlAhwU4W55h4gioJ0q96mmItr6HAKyAFkFp





Notice how the attribute, publicKey, wraps to multiple lines, each one prepended with a single space character. All of these lines, with the space removed, are concatenated to hold the full value of (in this case) the publicKey attribute for the ‘cn=server’ object. This particular value also happens to be base64-encoded which is a way of representing data which is easily transferred and preserved via various channels. This is not the same as encyprted or anything…. see more about base64 encoding on Wikipedia

The reason folding is important is that in order to compare objects from the directory the lines must be sortable in some way. Shouldn’t the lines be sorted or in some particular order by default, you may ask? No, LDAP makes no guarantees, by default, about the order of objects returned or the order of attribute values within those objects. As a result one of the commands we’ll be using is the ‘sort’ command (before comparing) but then where does that leave us if single values are folded across multiple lines? Correct, it’s a bad place since while things may line up, it will be very hard to tell what is wrong if everything is not exactly identical. With the lines unfolded, though, it is much easier to tell what is different within a single object, or within the schema itself. None of this really solves the problem of comparing multiple objects, but this article is about comparing objects or the schema, so we’ll deal with bigger compares at some point in the future, probably with a fancy script or some much-fancier line concatenation.

To unfold lines we’ll employe a little ‘sed’ command. ‘sed’ is another built-in command that, if you do not know now, is part of every administrator’s toolbox. ‘sed’ stands for ‘Stream EDitor’ and basically means it will modify streams of data. In our case, we’ll use it to unfold lines by searching for lines that are followed by a line which starts with a space, and then concatenating the lines together while removing the space. Rinse and repeat for the entire document and you’re left with unfolded data. The command do this is shown below and is meant to accept input via a pipe from the ldapsearch command:

sed -n '1 {h; $ !d}; $ {x; s/\n //g; p}; /^ / {H; d}; /^ /! {x; s/\n //g; p}'

This particular line is a bit messy and hard to understand, but it does the job. ‘sed’ can do some very simple operations for simple search/replace on streams, or files, and this is just one example of a lot of things being done in a single command to give some powerful functionality. We can see this work with the following example from the LDIF snippet above:

cat <<EOF | sed -n '1 {h; $ !d}; $ {x; s/\n //g; p}; /^ / {H; d}; /^ /! {x; s/\n //g; p}'

dn: cn=server,ou=context,o=goes,dc=here

publicKey: MIIFODCCBCCgAwIBAgIlAhwU4W55h4gioJ0q96mmItr6HAKyAFkFp







Notice the output is now unfolded:

dn: cn=server,ou=context,o=goes,dc=here



With the data object now organized in a way that can be reliably sorted, the next step is to grab objects or schema from various systems and sort them for comparison. Assuming I have two LDAP servers with schema located at the ‘cn=schema’ object, the following commands will get me output from the servers, and, respectively:

ldapsearch -LLL -x -h '' -p '389' -b 'cn=schema' -s 'base' | sed -n '1 {h; $ !d}; $ {x; s/\n //g; p}; /^ / {H; d}; /^ /! {x; s/\n //g; p}' | sort > ./
ldapsearch -LLL -x -h '' -p '389' -b 'cn=schema' -s 'base' | sed -n '1 {h; $ !d}; $ {x; s/\n //g; p}; /^ / {H; d}; /^ /! {x; s/\n //g; p}' | sort > ./

Now that the output is in a format that I can compare, it’s just a matter of which tool to use. The basic tool is ‘diff’, part of the ‘diffutils’ package, and if running that comes back with nothing then everything was identical. For a little more human-friendly comparison, vimdiff is great. For folks with ‘meld’, it is a fully graphical tool that can compare files, directory, or both, though it is not installed by default as I recall… still trivial to add from an RPM. At the end of the day any of the following commands will do the job:

diff ./ ./

vimdiff ./ ./

meld ./ ./

With everything completed this basically shows the value of modular and granular tools that stick to the old adage of, “Do one thing and do it well” which is part of the Linux/Unix philosophy. With a single system, which comes with some very modular tools (ldapsearch, sed, sort, and diff) we can carry out tasks that are part of an IT person’s routine… comparing objects, or schema, in various directories. These same techniques can be applied to any types of data comparison, and some common examples include text files (configuration data), log files (what was different in the broken case vs. the working case), or even things that we do not typically think of as being found in files (information on a system’s CPU, RAM, disks, drivers, etc.).

It is likely that there are a bunch of tools out there which can do this same LDIF comparison very quickly, and which may be designed specifically for that task, but part of the philosophy behind having modular tools is that they are not restricted to doing just one thing. There is no need for a tool to compare configuration files for Apache httpd, schema output from OpenLDAP or eDirectory, and database rows from PostgreSQL, all of which are somehow powering your customer-facing website. All of these types of comparisons can be done with a system, out of the box, by just knowing how.

Why does this matter to the average administrator or consultant? Because we aren’t paid for how well we click trough limited tools, or at least if we are, we should be worried. There will likely always be somebody who can click a little faster, or work for a little less, than we will and if our strength is in something as trivial as clicking through screens, i.e. busywork, then the next person who comes along and can do that busywork will replace us. On the other hand, if we have a toolbox that lets us work efficient and accurately, across a wide variety of technologies, then we have become more valuable because that’s will be a harder value to get the high school student willing to work for peanuts to emulate.

With that little segue down, go ahead and try out the tools. Use your new skills to get those seemingly-painful, manual tasks out of the way, and with technology doing the repetitive stuff go and tackle something truly valuable for your organization, or, if done for the day, for yourself.

2 votes, average: 5.00 out of 52 votes, average: 5.00 out of 52 votes, average: 5.00 out of 52 votes, average: 5.00 out of 52 votes, average: 5.00 out of 5 (2 votes, average: 5.00 out of 5)
You need to be a registered member to rate this post.

Tags: , , , , , ,
Categories: Open Enterprise Server on SLES, SUSE Linux Enterprise, SUSE Linux Enterprise Server, Technical Solutions

Disclaimer: As with everything else in the SUSE Blog, this content is definitely not supported by SUSE (so don't even think of calling Support if you try something and it blows up).  It was contributed by a community member and is published "as is." It seems to have worked for at least one person, and might work for you. But please be sure to test, test, test before you do anything drastic with it.

Comments are closed.