Intermittent horizon authentication failures with LDAP keystone backend

This document (000019838) is provided subject to the disclaimer at the end of this document.

Environment

SUSE OpenStack Cloud 9
SUSE OpenStack Cloud 8

Situation

User is unable to log in to Horizon dashboard with error message:

   "An error occurred authenticating. Please try again later."

keystone.log shows
16167 (ldappool): 2021-01-04 08:26:45,281 INFO __init__ _create_connector Failure attempting to create and bind connector; will retry after 0.1 seconds
Traceback (most recent call last):
  File "/opt/stack/service/keystone/venv/lib/python2.7/site-packages/ldappool/__init__.py", line 253, in _create_connector
    self._bind(conn, bind, passwd)
  File "/opt/stack/service/keystone/venv/lib/python2.7/site-packages/ldappool/__init__.py", line 226, in _bind
    conn.simple_bind_s(bind, passwd)
  File "/opt/stack/service/keystone/venv/lib/python2.7/site-packages/ldappool/__init__.py", line 100, in simple_bind_s
    clientctrls)
  File "/opt/stack/service/keystone/venv/lib/python2.7/site-packages/ldap/ldapobject.py", line 1090, in simple_bind_s
    res = self._apply_method_s(SimpleLDAPObject.simple_bind_s,*args,**kwargs)
  File "/opt/stack/service/keystone/venv/lib/python2.7/site-packages/ldap/ldapobject.py", line 1071, in _apply_method_s
    return func(self,*args,**kwargs)
  File "/opt/stack/service/keystone/venv/lib/python2.7/site-packages/ldap/ldapobject.py", line 383, in simple_bind_s
    resp_type, resp_data, resp_msgid, resp_ctrls = self.result3(msgid,all=1,timeout=self.timeout)
  File "/opt/stack/service/keystone/venv/lib/python2.7/site-packages/ldap/ldapobject.py", line 680, in result3
    resp_ctrl_classes=resp_ctrl_classes
  File "/opt/stack/service/keystone/venv/lib/python2.7/site-packages/ldap/ldapobject.py", line 687, in result4
    ldap_result = self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop)
  File "/opt/stack/service/keystone/venv/lib/python2.7/site-packages/ldap/ldapobject.py", line 263, in _ldap_call
    result = func(*args,**kwargs)
UNWILLING_TO_PERFORM: {'info': u'Too many failed logins.\n', 'desc': u'Server is unwilling to perform'}

 

Resolution

Set pool_retry_max to 0 in keystone configuration file.

For CLM deployment:
On the deployer node run following commands:

   vim ~/openstack/my_cloud/config/keystone/keystone_configure_ldap_XYZ.yml

In "ldap:" section add:

   pool_retry_max: 0

cd ~/openstack
git checkout site
git add -A
git commit -m "Setting pool_retry_max to 0"

cd ~/openstack/ardana/ansible
ansible-playbook -i hosts/localhost config-processor-run.yml
ansible-playbook -i hosts/localhost ready-deployment.yml

cd ~/scratch/ansible/next/ardana/ansible
ansible-playbook -i hosts/verb_hosts keystone-reconfigure.yml \
  -e@/var/lib/ardana/openstack/my_cloud/config/keystone/keystone_configure_ldap_XYZ.yml


For Crowbar deployment:
On all controllers run:

   vim /etc/keystone/keystone.conf.d/500-keystone.conf
   
add:

[ldap]
pool_retry_max = 0


On admin node run:

     crowbarctl proposal commit keystone default

If cluster is used, on a controller run:

  crm resource restart cl-keystone

Configuration without a cluster, on all controllers run:

    systemctl restart apache2.service
 

Cause

Ldappool is configured to continually retry (based on the number of retries specified) on all LDAP errors. 
This is an issue for Keystone because by default it is configured to retry 3 times with .1 seconds between retries. 
When a user enters a bad password while authenticating to Keystone ldappool will try that password once, and then 
catch the ldap.LdapError class which is thrown. 
This is the base class for all Python-LDAP errors including bad passwords. 
Ldappool will then retry until the retry limit is reached. 
This will lock out keystone users if the retry limit is configured to be higher than the LDAP bad password lockout number.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:000019838
  • Creation Date: 19-Jan-2021
  • Modified Date:21-Jan-2021
    • SUSE Open Stack Cloud

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback@suse.com

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

Join Our Community

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.


SUSE Customer Support Quick Reference Guide SUSE Technical Support Handbook Update Advisories
Support FAQ

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.

Go to Customer Center