SUSE Conversations


HOWTO: OpenLDAP 2.4.x Replication on SLES11 SP1



By: fpernet

April 8, 2011 1:41 pm

Reads:3931

Comments:1

Rating:0

Introduction

YaST has a module for the server and the client part of OpenLDAP (plus a very basic LDAP browser / editor).

On SLES11 we had the following problems regarding the use of the YaST module and OpenLDAP server configuration:

  • Database version only to store the configuration (cn=config).
  • No way to set up any replication method or mode using this module.
  • OpenLDAP modules available build in the SLAPD binary.

Then your choices in this SLES11 version:

  • Use the configuration file /etc/openldap/slapd.conf but in this case, forget YaST for ever.
  • Use YaST to set up a basic OpenLDAP server then use an external tool to set up the rest.
  • If use of YaST only is mandatory, forget the replication.

An RFE (Request For Enhancements) was created at this time, to be able to set up replication from the YaST module, on SLES11 SP1. They did it! So, let’s check how it works.

Note: On SLES11 or SLES11 SP1, by playing with /etc/sysconfig/opendlap, you can use the configuration file (/etc/openldap/slapd.conf) to configure OpenLDAP server but in this case do not use YaST anymore.

Goal of this HOWTO:

  • Understand all replication modes available.
  • Configure and test OpenLDAP replication on a SLES11 SP1.
  • Check if YaST can be used in these different modes

LAB

LAB is very basic:

All servers are: SLES11 SP1 x86 – No patch – OpenLDAP version: 2.4.20-0.4.29

No firewall is activated. No extra package has been selected during install.

OpenLDAP 2.4.x Replication Methods

Overall

OpenLDAP official documentation is high quality and has been updated recently: OpenLDAP 2.4 Replication

Quick history of replication with OpenLDAP

  • Synchronization introduced in version OpenLDAP 2.2.
  • First replication type used a separate demon named slurpd.
  • New replication methods (Delta Sync) introduced in version 2.3.
  • In version 2.4, new modes were implemented (Proxy, Mirror Node, Multi-Master) and slurpd was removed.
  • All OpenLDAP configuration is inside the database itself (but still available in config file form).

Terms: Master and Slave are replaced by Provider and Consumer.

Syncrepl is a replication thread (slapd) executed on the consumer side. It allows to maintain a copy of the whole (or a part) DIT tree. This thread connects on the provider in order to obtain an original copy and keep it up to date.
Syncrepl uses two modes and does not need to keep a history of changes (in a log for instance even if a session-log can be possible): push-based or pull-based.

  • pull-based replication polls the provider at regular intervals to check for changes.
  • push-based replication keeps the consumer listens in real time for all changes sent by the provider.

Syncrepl keeps track of replication state by an exchange of synchronization cookies. Using this way, Syncrepl has the following features:

  • Syncrepl lets a consumer start and stop the synchro without any dialog with the provider.
  • With Syncrepl, the consumer must have access to the DIT fragment (partition) to replicate.
  • Syncrepl supports partial replication based on search filters (base, scope, filter, and attribute list) and the replica content is tied to the access privileges of the identity used for the replication connection (read access on the objects).

Notes on the databases backends: bdb or hdb
The bdb backend to slapd(8) is the recommended primary backend for a normal slapd database. It uses the Oracle Berkeley DB (BDB) package to store data. It makes extensive use of indexing and caching (to tune it, see here) to speed data access.
hdb is a variant of the bdb backend that uses a hierarchical database layout which supports subtree renames. It is otherwise identical to the bdb behavior, and all the same configuration options apply.
An hdb database needs a large idlcachesize for good search performance, typically three times the cachesize (entry cache size) or larger.

Syncrepl method

Syncrepl engine uses a present phase and a delete phase during the synchronization refresh. A session log can be created to store all the entryUUIDs for a certain amount of deleted entries in the database. Multiple replicas can use the same session log file.

  • If session log file is present : Syncrepl engine starts a Delete phase if the consumer’s replica is recent enough.
  • If session log file is not present : Syncrepl engine uses a Present phase if the consumer’s replica is too old.

The Syncrepl engine could be set up using different backends but is more efficient with bdb or hdb backends.

Syncrepl « RefreshOnly » type:

image2.png

Syncrepl « RefreshAndPersist » type:

image3.png

Disadvantages of Syncrepl replication method:
This method is « object » oriented. Then any change of a single attribute drives the synchro for the whole object with this attribute. If you want to use your directory to change small modifications on any different object, this kind of replication will have heavy use of the bandwidth only for small changes.

Advantages of Syncrepl replication method:
When multiple changes occur on a single object (change of multiple attributes), the order of change is not important and the replication of the whole object is very efficient.

Delta Sync Repl method

Delta-syncrepl method uses a changelog (accesslog overlay). Delta-syncrepl maintains a log (you can choose the size) on the provider. The consumer, during the replication, will parse this log and apply all the changes. If the replica on the consumer is too old, the classic syncrepl method is used and then switch to the Delta-syncrepl method.

Disadvantages of Delta-Sync method:
This replication method takes more place on the provider because of this change log (even if we can select the purge period).
This method is today only available for classic syncrepl mode (Provider -> Consumer) and not for other modes (Mirror, N-Way,…).

Advantages of Delta-Sync method:
We can observe more granularity because this replication method applies changes on attributes.

N-Way multi-master mode (syncrepl method only)

The Multi-Master mode uses Syncrepl and can be build on many providers for each consumer. Each consumer is itself a provider for all or some of the other member servers. Very complicated and various structure can be achieved but you must keep in mind that this kind of complicated configuration can drive you in more problems than advantages.

image5.png

Arguments for Multi-Master replication
If one provider fails, other providers remain for modifications.
Avoid a SPOF (single point of failure).
Providers can be deployed in different sites.
Solution for Automatic failover/High Availability.

Arguments against Multi-Master replication
Does not guarantee data consistency: Consistency
This is not, itself, a solution for load balancing
If a network is partitioned and multiple clients start writing to each of the “masters” then reconciliation will be a pain; it may be best to simply deny writes to the clients that are partitioned from the single provider

Important details for this mode:

  • Delta-Synchro is not available for this mode
  • Time synchronization is crucial.
  • Each server must have a unique serverid among all other servers, plus a rid (remote id or provider id) unique for each of the providers inside a single server.
  • The option mirrormode TRUE is necessary.

Mirroring mode (method syncrepl only)

This is a kind of hybrid mode between classic Syncrepl (single master) and the N-Way (multi master) mode. Two servers are configured in mirror mode and each one is the provider and the consumer of the other. However, to guarantee data consistency, all write modifications (delete, modify, add) must be directed to only one server that we can call « Active ».

If the active server fails, write operations must be externally (automatically ou manually) redirected to the remaining server. When the first server will ba back, it will receive all modifications and then be able to become again the « active » server.

Arguments for MirrorMode synchro
Can be considered as a high-availability (HA) solution for writes operations (read operations can be directed to both of them).
The system can work until at least one server remains up.

Arguments against MirrorMode synchro
MirrorMode is not really a Multi-Master mode.
MirrorMode can be considered as an Active-Active Hot-Standby ONLY with an external device to detect which one is active (hardware or another LDAP server in proxy mode).
Backups must be organised on the actual « active » server.
Delta-Synch method is not available in this mode.

Set up classic replication: Provider -> Consumer, Syncrepl

Simple replication Provider -> Consumer (Syncrepl) using RefreshAndPersist connection.

Pre-installation on the provider (provider = DA3)

Initialization

Note: TLS can be set up later. The best is to set up everything without, in order to simplify, then, when everything works fine, do the security part.

This is the password for the « cn=config » database (see below).

Overall configuration

Rerun YaST to see the configuration.

All details can then be reviewed to set up correctly the server.

See Enable Session Log above.

The Replication Type is the connection mode (see above):

  • RefreshOnly = pull-based
  • RefreshAndPersist = push-based

This configuration will be read by the consumer, by default. This is a crucial point and you can do one of the following:

  • either you declare it as standalone and later you can modify it in a provider or consumer. This is the easiest way.
  • either you declare it immediatly in a provider or consumer (like in this case) but be aware that the consumer will try to reach the cn =config from the provider directly, plus of course the regular database (dc=idsatraining,dc=com).

Pre-installation on the consumer (consumer = DA4)

Then, configure the address of the Provider from which it will retrieve the regular database plus the configuration database:

Verify everything

At this point, we should see all the data from the regular database. We can use the LDAP browser inside YaST or more easily the comment line:

Check the regular database:

da3:/etc/openldap # ldapsearch -x -h 127.0.0.1 -s sub -b dc=idsatraining,dc=com -D cn=admin,dc=idsatraining,dc=com –W (cn=*)

Check the config (TLS required):

da3:/etc/openldap # ldapsearch -Z -h 127.0.0.1 -s sub -b cn=config -D cn=admin,dc=idsatraining,dc=com –W

Clean up configuration on the PROVIDER

This behaviour is not really clear. The configuration (cn=config) has been synchronized between the provider and the consumer. Because of that, each of the two servers are at the same time, on the two databases (base config and regular base). I would prefer to have config totally separated on each of the servers and then remain insulated.

Let’s re-configure everything correctly (be careful of the order of configuration).

The result of a correct configuration then appear as:

Access control on the config database

We are not provider on the config db

We are not consumer on this database

Access control on the regular database

We are provider on the regular database

We are not consumer for this database

Re-configure the CONSUMER

Same thing but in this case we are consumer only for the regular database:

Access control

We are not provider for the config

We are not consumer for this database

Access control for this regular database

We are not provider for the regular database

We are consumer for the regular database

« Custom update referral »: this field defines which server will be responsible for all the write requests. In the case of simple Syncrepl, we should define on the consumer, the address of the Provider (because the consumer is read-only).

Verify complete behaviour

For instance, we can modify, using the YaST LDAP Browser, one attribute of the regular database (Description of the object cn=syncrepl,ou=system,dc=idsatraining,dc=com).

Then with the LDAP browser again, on the consumer, we can verify that the object has been synchronized.

Tools to manipulate the Configuration

Configuration: file or database

Without a real configuration file (/etc/openldap/slapd.conf), it is difficult but necessary to be able to see the configuration database:

LDAPSEARCH

The command line LDAPSEARCH allows us to see the content of any database:

For a regular database:

da3:/etc/openldap # ldapsearch -x -h 127.0.0.1 -s sub -b dc=idsatraining,dc=com -D cn=admin,dc=idsatraining,dc=com –W (cn=*)

For the config database(TLS):

da3:/etc/openldap # ldapsearch -Z -h 127.0.0.1 -s sub -b cn=config -D cn=admin,dc=idsatraining,dc=com –W

Example to check the provider configuration:

ldapsearch -Z -h 127.0.0.1 -b cn=config -D cn=admin,dc=idsatraining,dc=com -W "(ObjectClass=olcSyncProvconfig)"                                               
Enter LDAP Password:                                                                                                                                                  
# extended LDIF                                                                                                                                                       
#                                                                                                                                                                     
# LDAPv3                                                                                                                                                              
# base <cn=config> with scope subtree                                                                                                                                 
# filter: (ObjectClass=olcSyncProvconfig)                                                                                                                             
# requesting: ALL                                                                                                                                                     
#                                                                                                                                                                     
                                                                                                                                                                      
# {0}syncprov, {1}bdb, config                                                                                                                                         
dn: olcOverlay={0}syncprov,olcDatabase={1}bdb,cn=config                                                                                                               
objectClass: olcSyncProvConfig                                                                                                                                        
olcOverlay: {0}syncprov                                                                                                                                               
olcSpCheckpoint: 100 10                                                                                                                                               
olcSpSessionlog: 20000                                                                                                                                                
                                                                                                                                                                      
# search result                                                                                                                                                       
search: 3                                                                                                                                                             
result: 0 Success                                                                                                                                                     
                                                                                                                                                                      
# numResponses: 2                                                                                                                                                     
# numEntries: 1

Example to check the consumer configuration:

ldapsearch -Z -h 127.0.0.1 -b cn=config -D cn=admin,dc=idsatraining,dc=com -W "(ObjectClass=olcBdbconfig)"
Enter LDAP Password:                                                                                                                                                  
# extended LDIF                                                                                                                                                       
#                                                                                                                                                                     
# LDAPv3                                                                                                                                                              
# base <cn=config> with scope subtree                                                                                                                                 
# filter: (ObjectClass=olcBdbconfig)                                                                                                                                  
# requesting: ALL                                                                                                                                                     
#                                                                                                                                                                     
                                                                                                                                                                      
# {1}bdb, config                                                                                                                                                      
dn: olcDatabase={1}bdb,cn=config                                                                                                                                      
objectClass: olcDatabaseConfig                                                                                                                                        
objectClass: olcBdbConfig                                                                                                                                             
olcDatabase: {1}bdb                                                                                                                                                   
olcDbDirectory: /var/lib/ldap                                                                                                                                         
olcSuffix: dc=idsatraining,dc=com                                                                                                                                     
olcAccess: {0}to * by dn.base="cn=admin,dc=idsatraining,dc=com" manage break b                                                                                        
 y dn.base="uid=syncrepl,ou=system,dc=idsatraining,dc=com" read break by * rea                                                                                        
 d                                                                                                                                                                    
olcAccess: {1}to attrs=userPassword by self write by * auth                                                                                                           
olcAccess: {2}to attrs=shadowLastChange by self write by * read                                                                                                       
olcAccess: {3}to attrs=userPKCS12 by self read by * none                                                                                                              
olcLimits: {0}dn.exact="uid=syncrepl,ou=system,dc=idsatraining,dc=com" size.so                                                                                        
 ft=unlimited                                                                                                                                                         
olcRootDN: cn=admin,dc=idsatraining,dc=com                                                                                                                            
olcRootPW: {SSHA}PM5qDUDDfQoJJThPBkqmWzOfx3ZEUkZQRA==                                                                                                                 
olcSyncrepl: {0}rid=2 provider="ldap://da3.idsatraining.com/" searchbase="dc=i                                                                                        
 dsatraining,dc=com" type="refreshAndPersist" retry="120 +" starttls=critical                                                                                         
 tls_reqcert=demand bindmethod="simple" binddn="cn=admin,dc=idsatraining,dc=co                                                                                        
 m" credentials="novell"                                                                                                                                              
olcUpdateRef: ldap://da3.idsatraining.com/                                                                                                                            
olcDbCacheSize: 10000                                                                                                                                                 
olcDbCheckpoint: 1024 5                                                                                                                                               
olcDbIDLcacheSize: 30000                                                                                                                                              
olcDbIndex: objectclass eq                                                                                                                                            
olcDbIndex: uidNumber eq                                                                                                                                              
olcDbIndex: gidNumber eq                                                                                                                                              
olcDbIndex: member eq                                                                                                                                                 
olcDbIndex: memberUid eq                                                                                                                                              
olcDbIndex: mail eq                                                                                                                                                   
olcDbIndex: cn eq,sub                                                                                                                                                 
olcDbIndex: displayName eq,sub                                                                                                                                        
olcDbIndex: uid eq,sub                                                                                                                                                
olcDbIndex: sn eq,sub                                                                                                                                                 
olcDbIndex: givenName eq,sub                                                                                                                                          
olcDbIndex: entryUUID eq                                                                                                                                              
olcDbIndex: entryCSN eq                                                                                                                                               
                                                                                                                                                                      
# search result                                                                                                                                                       
search: 3                                                                                                                                                             
result: 0 Success                                                                                                                                                     
                                                                                                                                                                      
# numResponses: 2                                                                                                                                                     
# numEntries: 1

SLAPCAT

Slapcat is a dump tool for OpenLDAP databases (on the screen or in a LDIF file).

config database (number 0):

slapcat -n 0 -l /opt/config.ldif

regular database (number 1 here in our directory):

slapcat -n 1 -l /opt/idsatraining.ldif

Check replication

Remember that the YaST module allows us to select the content of the log file:

Tools to manipulate the Configuration

Configuration : file or database

Without a real configuration file (/etc/openldap/slapd.conf), it is difficult but necessary to be able to see the configuration database:

LDAPSEARCH

The command line LDAPSEARCH allows us to see the content of any database:

For a regular database:

da3:/etc/openldap # ldapsearch -x -h 127.0.0.1 -s sub -b dc=idsatraining,dc=com -D cn=admin,dc=idsatraining,dc=com –W (cn=*)

For the config database(TLS):

da3:/etc/openldap # ldapsearch -Z -h 127.0.0.1 -s sub -b cn=config -D cn=admin,dc=idsatraining,dc=com –W

Example to check the provider configuration:

ldapsearch -Z -h 127.0.0.1 -b cn=config -D cn=admin,dc=idsatraining,dc=com -W "(ObjectClass=olcSyncProvconfig)"                                               
Enter LDAP Password:                                                                                                                                                  
# extended LDIF                                                                                                                                                       
#                                                                                                                                                                     
# LDAPv3                                                                                                                                                              
# base <cn=config> with scope subtree                                                                                                                                 
# filter: (ObjectClass=olcSyncProvconfig)                                                                                                                             
# requesting: ALL                                                                                                                                                     
#                                                                                                                                                                     
                                                                                                                                                                      
# {0}syncprov, {1}bdb, config                                                                                                                                         
dn: olcOverlay={0}syncprov,olcDatabase={1}bdb,cn=config                                                                                                               
objectClass: olcSyncProvConfig                                                                                                                                        
olcOverlay: {0}syncprov                                                                                                                                               
olcSpCheckpoint: 100 10                                                                                                                                               
olcSpSessionlog: 20000                                                                                                                                                
                                                                                                                                                                      
# search result                                                                                                                                                       
search: 3                                                                                                                                                             
result: 0 Success                                                                                                                                                     
                                                                                                                                                                      
# numResponses: 2                                                                                                                                                     
# numEntries: 1

Example to check the consumer configuration:

ldapsearch -Z -h 127.0.0.1 -b cn=config -D cn=admin,dc=idsatraining,dc=com -W "(ObjectClass=olcBdbconfig)"
Enter LDAP Password:                                                                                                                                                  
# extended LDIF                                                                                                                                                       
#                                                                                                                                                                     
# LDAPv3                                                                                                                                                              
# base <cn=config> with scope subtree                                                                                                                                 
# filter: (ObjectClass=olcBdbconfig)                                                                                                                                  
# requesting: ALL                                                                                                                                                     
#                                                                                                                                                                     
                                                                                                                                                                      
# {1}bdb, config                                                                                                                                                      
dn: olcDatabase={1}bdb,cn=config                                                                                                                                      
objectClass: olcDatabaseConfig                                                                                                                                        
objectClass: olcBdbConfig                                                                                                                                             
olcDatabase: {1}bdb                                                                                                                                                   
olcDbDirectory: /var/lib/ldap                                                                                                                                         
olcSuffix: dc=idsatraining,dc=com                                                                                                                                     
olcAccess: {0}to * by dn.base="cn=admin,dc=idsatraining,dc=com" manage break b                                                                                        
 y dn.base="uid=syncrepl,ou=system,dc=idsatraining,dc=com" read break by * rea                                                                                        
 d                                                                                                                                                                    
olcAccess: {1}to attrs=userPassword by self write by * auth                                                                                                           
olcAccess: {2}to attrs=shadowLastChange by self write by * read                                                                                                       
olcAccess: {3}to attrs=userPKCS12 by self read by * none                                                                                                              
olcLimits: {0}dn.exact="uid=syncrepl,ou=system,dc=idsatraining,dc=com" size.so                                                                                        
 ft=unlimited                                                                                                                                                         
olcRootDN: cn=admin,dc=idsatraining,dc=com                                                                                                                            
olcRootPW: {SSHA}PM5qDUDDfQoJJThPBkqmWzOfx3ZEUkZQRA==                                                                                                                 
olcSyncrepl: {0}rid=2 provider="ldap://da3.idsatraining.com/" searchbase="dc=i                                                                                        
 dsatraining,dc=com" type="refreshAndPersist" retry="120 +" starttls=critical                                                                                         
 tls_reqcert=demand bindmethod="simple" binddn="cn=admin,dc=idsatraining,dc=co                                                                                        
 m" credentials="novell"                                                                                                                                              
olcUpdateRef: ldap://da3.idsatraining.com/                                                                                                                            
olcDbCacheSize: 10000                                                                                                                                                 
olcDbCheckpoint: 1024 5                                                                                                                                               
olcDbIDLcacheSize: 30000                                                                                                                                              
olcDbIndex: objectclass eq                                                                                                                                            
olcDbIndex: uidNumber eq                                                                                                                                              
olcDbIndex: gidNumber eq                                                                                                                                              
olcDbIndex: member eq                                                                                                                                                 
olcDbIndex: memberUid eq                                                                                                                                              
olcDbIndex: mail eq                                                                                                                                                   
olcDbIndex: cn eq,sub                                                                                                                                                 
olcDbIndex: displayName eq,sub                                                                                                                                        
olcDbIndex: uid eq,sub                                                                                                                                                
olcDbIndex: sn eq,sub                                                                                                                                                 
olcDbIndex: givenName eq,sub                                                                                                                                          
olcDbIndex: entryUUID eq                                                                                                                                              
olcDbIndex: entryCSN eq                                                                                                                                               
                                                                                                                                                                      
# search result                                                                                                                                                       
search: 3                                                                                                                                                             
result: 0 Success                                                                                                                                                     
                                                                                                                                                                      
# numResponses: 2                                                                                                                                                     
# numEntries: 1

SLAPCAT

Slapcat is a dump tool for OpenLDAP databases (on the screen or in a LDIF file).

config database (number 0):

slapcat -n 0 -l /opt/config.ldif

regular database (number 1 here in our directory):

slapcat -n 1 -l /opt/idsatraining.ldif

Check replication

Remember that the YaST module allows us to select the content of the log file:

Regular log is by default /var/log/messages but it is better to configure syslog-ng, in order to have it separated:

# Filter slapd activity                                                                                                                                               
filter f_slapd     { match('^slapd'); };                                                                                                                             
destination slapd { file("/var/log/slapd.log"); };                                                                                                                    
log { source(src); filter(f_slapd); destination(slapd); };

Example of log for the synchro of the object : uid=syncrepl,ou=system,dc=idsatraining,dc=com (attribute locality modified):

On the provider

Mar 13 10:03:33 da3 slapd[9621]: conn=1003 op=14 ENTRY dn="uid=syncrepl,ou=system,dc=idsatraining,dc=com"                                                             
Mar 13 10:03:33 da3 slapd[9621]: slap_queue_csn: queing 0xb4e99912 20110313090333.982428Z#000000#000#000000                                                           
Mar 13 10:03:33 da3 slapd[9621]: syncprov_sendresp: cookie=rid=002,csn=20110313090333.982428Z#000000#000#000000                                                       
Mar 13 10:03:33 da3 slapd[9621]: slap_graduate_commit_csn: removing 0xb790d5b8 20110313090333.982428Z#000000#000#000000                                               
Mar 13 10:03:33 da3 slapd[9621]: conn=1002 op=2 ENTRY dn="uid=syncrepl,ou=system,dc=idsatraining,dc=com"     

On the consumer

Mar 13 10:01:57 da4 slapd[26884]: do_syncrep2: rid=002 cookie=rid=002,csn=20110313090333.982428Z#000000#000#000000                                                    
Mar 13 10:01:57 da4 slapd[26884]: syncrepl_entry: rid=002 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_MODIFY)                                                                     
Mar 13 10:01:57 da4 slapd[26884]: syncrepl_entry: rid=002 be_search (0)                                                                                               
Mar 13 10:01:57 da4 slapd[26884]: syncrepl_entry: rid=002 uid=syncrepl,ou=system,dc=idsatraining,dc=com                                                               
Mar 13 10:01:57 da4 slapd[26884]: slap_queue_csn: queing 0xb79b88f0 20110313090333.982428Z#000000#000#000000                                                          
Mar 13 10:01:57 da4 slapd[26884]: slap_graduate_commit_csn: removing 0xb79eb398 20110313090333.982428Z#000000#000#000000                                              
Mar 13 10:01:57 da4 slapd[26884]: syncrepl_entry: rid=002 be_modify uid=syncrepl,ou=system,dc=idsatraining,dc=com (0)                                                 
Mar 13 10:01:57 da4 slapd[26884]: slap_queue_csn: queing 0xb79b88f0 20110313090333.982428Z#000000#000#000000                                                          
Mar 13 10:01:57 da4 slapd[26884]: slap_graduate_commit_csn: removing 0xb79b7d68 20110313090333.982428Z#000000#000#000000 

Note: the whole object has been synchronized (Syncrepl, not DeltaSync).

Modify configuration

It is more difficult to modify the configuration. We have two problems:

  • YaST does not propose all the options (DeltaSync, Mode Mirror, N-way, etc…)
  • YaST LDAP browser is really poor

Without YaST, I tested two different ways:

Brute way

Under SLES11 SP1, configuration is stored in LDIF files under config directory (/etc/openldap/slapd.d/). We can then modify stuff directly in these files and then restart the demon (here olcDatabase\=\{1\}bdb.ldif).

da3:/etc # rcldap stop
da3:/etc # vi /etc/openldap/slapd.d/cn=config/olcDatabase\=\{1\}bdb.ldif
<modifications>
da3:/etc # rcldap start

Elegant way

We need a more powerful LDAP Browser/Editor. I can think about two of them:

I prefer jxplorer because easier to install (but it requires Xorg because written in Java). The advantages of such tools are:

  • they learn the schema and propose then the good attributes.
  • they allow you to select any base (for instance cn=config).

Here an example of the config database configuration with jxplorer (jxplorer does not know TLS so we must use SSL+ user + password):

Backup

Under SLES11 SP1, you must distinguish the configuration (special databases) and content (regular bases, here dc=idsatraining,dc=com). If we want a complete backup and restore, we can choose between two ways:

Brute way

Backup:

rcldap stop
-> save all /etc/openldap
-> save all /var/lib/ldap
rcldap start

Restore:

rcldap stop
-> put back /etc/openldap
-> put back /var/lib/ldap
rcldap start

Elegant way

Even if many examples and documentation exist on internet, including those from OpenLDAP, under SLES11 SP1, the basic tools (slapcat, slapadd) can not be used for the configuration side, but are very useful for the regular databases. Then a procedure like that could work (if we have only one regular database – see delta sync part for more databases):

Backup:

rcldap stop
-> save all /etc/openldap
da3:/etc # slapcat –n 1 –l /backup/savebase1.ldif
rcldap start

Restore:

rcldap stop
-> put back /etc/openldap
da3:/etc # slapadd –n 1 –l /backup/savebase1.ldif
rcldap start

Mirrormode Replication Mode

Note: Even if YaST can’t be used to set up such replication, its use does not destroy any mirrodmode already in place.

MirrorMode is very easy to set up.

If you start from a regular Syncrepl mode (Provider -> Consumer), then you only need to add the following on both the consumer and the provider:

mirrormode  on		-> in (1) bdb
serverID    1		-> in global part of cn=config on the provider
serverID    2		-> in global part of cn=config on the consumer

Each node will then be the consumer and the provider of the second one (this can be achieved with YaST).

Because in this case we can’t speak about consumer and provider anymore, we’ll say Node1 (DA3) and Node2 (DA4).

Configuration on Node 1 (ex Provider)

With jxplorer, fulfill the following fields:

olcServerID:

olcMirrorMode

Configuration on Node 2 (ex Consumer)

With jxplorer, fulfill the following fields:

olcServerID:

olcMirrorMode

Verify behaviour

Modify one object on Node 1 and verify replication. Then modify another object on Node 2 and verify again it works. Stop Node 1, figuring a crash, go to modify some objects on Node 2. Restart Node 1 and verify it retrieves everything.

Simple Replication mode with DeltaSync method

Note: You can’t use YaST to set up such replication. More of that, be ready to NEVER use YaST anymore for LDAP server or you’ll have great chances it’ll crash your configuration.

DeltaSync method set up a log (AccessLog) on the provider, which stores all changes (logops writes) successful (logsuccess TRUE). This log is in fact another database you need to create (hdb or bdb). On the consumer, we must redirect syncrepl (RefreshOnly ou refreshAndPersist) on this accesslog database.

Provider configuration

In file mode, for reference, here is the configuration on the provider:

database hdb
…
suffix "dc=idsatraining,dc=com"
...
index entryCSN,entryUUID eq
...
overlay accesslog
logdb "cn=accesslog"
logops writes
logsuccess TRUE 
logpurge 2+00:00 1+00:00
…
overlay syncprov
syncprov-checkpoint 100 10
…
database hdb
...
suffix "cn=accesslog"
index default eq
index entryCSN,objectClass,reqEnd,reqResult,reqStart 
...
overlay syncprov
syncprov-nopresent TRUE
syncprov-reloadhint TRUE

Using jxplorer we should arrive to the following:

On the left we can see the new structure

Our regular database dc=idsatraining,dc=com (nothing special on the provider)

The AccessLog overlay points on the database cn=accesslog and stores all successful write operations.

Syncprov is needed in case the replica is too old (see theory above)

The database cn=accesslog must exist with the correct indexes for performances issues.

This database is also provider for DA4. Be careful, NoPresent and ReloadHint are mandatory with DeltaSync.

The resulting structure on our server is the following:

da3:/etc/openldap/slapd.d # ls -R
.:
cn=config  cn=config.ldif

./cn=config:
cn=schema       olcDatabase={-1}frontend.ldif  olcDatabase={0}config.ldif  olcDatabase={1}hdb.ldif  olcDatabase={2}hdb.ldif
cn=schema.ldif  olcDatabase={0}config          olcDatabase={1}hdb          olcDatabase={2}hdb

./cn=config/cn=schema:
cn={0}core.ldif  cn={1}cosine.ldif  cn={2}inetorgperson.ldif  cn={3}rfc2307bis.ldif  cn={4}yast.ldif

./cn=config/olcDatabase={0}config:

./cn=config/olcDatabase={1}hdb:
olcOverlay={0}accesslog.ldif  olcOverlay={1}syncprov.ldif

./cn=config/olcDatabase={2}hdb:
olcOverlay={0}syncprov.ldif


da3:/etc/openldap/slapd.d # ls -R /var/lib/ldap
/var/lib/ldap:
DB_CONFIG          __db.001  __db.003  __db.005  accesslog  cn.bdb     entryCSN.bdb   id2entry.bdb    objectClass.bdb  uid.bdb
DB_CONFIG.example  __db.002  __db.004  __db.006  alock      dn2id.bdb  entryUUID.bdb  log.0000000001  reqStart.bdb

/var/lib/ldap/accesslog:
DB_CONFIG  __db.002  __db.004  __db.006  dn2id.bdb     id2entry.bdb    objectClass.bdb  reqResult.bdb
__db.001   __db.003  __db.005  alock     entryCSN.bdb  log.0000000001  reqEnd.bdb       reqStart.bdb

Consumer configuration

In file mode, for reference, here is the configuration on the consumer:

# database section
database bdb
suffix "dc=idsatraining,dc=com"
...
syncrepl rid=000 
  provider=ldap://da3.idsatraining.com
  type=refreshAndPersist
  retry="5 5 300 +" 
  searchbase="dc=idsatraining,dc=com"
  attrs="*,+"
  bindmethod=simple
  binddn="cn=accesslog"
  credentials=novell
  logbase="cn=deltalog"
  logfilter="(&(objectClass=auditWriteObject)(reqResult=0))"
  syncdata=accesslog 
...

With jxplorer, we have:

Everything is inside the syncrepl definition:

Verify the behaviour

To check if our DeltaSync method works, let’s do the following.

Modify one attribute

Here we modify one attribute on the provider ( for instance the syncrepl object):

From jxplorer we can connect directly on the database cn=accesslog, on the provider:

On this database, we can see that all entries are changes descriptions. We can find our recent modification. These changes are store as requests for changes and will be read by the consumer.

On the consumer we can check that the object is in sync.

Check the log

We can have details about how the operations are organized:

On the provider:

Mar 17 08:04:49 da3 slapd[29939]: slap_queue_csn: queing 0xb359d912 20110317070449.221204Z#000000#000#000000
Mar 17 08:04:49 da3 slapd[29939]: slap_queue_csn: queing 0xb2c9d370 20110317070449.221204Z#000000#000#000000
Mar 17 08:04:49 da3 slapd[29939]: slap_graduate_commit_csn: removing 0xb79c8fb0 20110317070449.221204Z#000000#000#000000
Mar 17 08:04:49 da3 slapd[29939]: syncprov_sendresp: cookie=rid=000,csn=20110317070449.221204Z#000000#000#000000
Mar 17 08:04:49 da3 slapd[29939]: slap_graduate_commit_csn: removing 0xb79c54e8 20110317070449.221204Z#000000#000#000000

On the consumer:

Mar 17 08:02:12 da4 slapd[11981]: do_syncrep2: rid=000 cookie=rid=000,csn=20110317070449.221204Z#000000#000#000000
Mar 17 08:02:12 da4 slapd[11981]: slap_queue_csn: queing 0xb4325dfa 20110317070449.221204Z#000000#000#000000
Mar 17 08:02:12 da4 slapd[11981]: slap_graduate_commit_csn: removing 0xb4300558 20110317070449.221204Z#000000#000#000000
Mar 17 08:02:12 da4 slapd[11981]: syncrepl_message_to_op: rid=000 be_modify uid=syncrepl,ou=system,dc=idsatraining,dc=com (0)
Mar 17 08:02:12 da4 slapd[11981]: slap_queue_csn: queing 0xb4300c78 20110317070449.221204Z#000000#000#000000
Mar 17 08:02:12 da4 slapd[11981]: slap_graduate_commit_csn: removing 0xb4326770 20110317070449.221204Z#000000#000#000000

Note: In this example, even if everything is fine, we must absolutely correct the time synchronization (snychronize the consumer on the provider for instance, with NTP). The time stamps are vital with this method.

Replication Multi-Master (N-Way)

This mode has not been tested in this document but I hope you have everything now to set it up if you want …

Conclusion

OpenLDAP replication works perfectly under SLES11 SP1, at least with the tested modes:

  • Mode Provider/Consumer
  • Mode MirrorMode
  • Mode N-Way (not tested)

Unfortunately, only the regular Syncrepl has been implemented in the YaST module, but this is at least something.

A « Request for Enhancement » (RFE # 42370) has been created to include in the next releases, the DeltaSync method and the MirrorMode.

Precisely, this document has been published to help you, guys, to set up any kind of replication on a SLES11 SP1.

References

OpenLDAP replication
zytrax ldap
replication strategies

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)

Tags:
Categories: SUSE Linux Enterprise Server, Technical Solutions

Disclaimer: As with everything else at SUSE Conversations, this content is definitely not supported by SUSE (so don't even think of calling Support if you try something and it blows up).  It was contributed by a community member and is published "as is." It seems to have worked for at least one person, and might work for you. But please be sure to test, test, test before you do anything drastic with it.

1 Comment

  1. By:gjnnovell

    It is very problematic to read this HOWTOW the mismatch from the ldap Databases HDB, BDB is very problematic :-(.

Comment

RSS