Creating additional images results in random OSDs being reported as down.

This document (7021060) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Enterprise Storage 4

Situation

On a healthy cluster when creating new additional images this results in random OSDs (Object Storage Daemons) being reported as down.

Resolution

Set the following in "/etc/ceph/ceph.conf" and reboot the cluster nodes:

ms_type = async

NOTE: See the additional information section for more details, take caution when making this change.
Alternatively contact SUSE Customer Care by opening a Service Request.

Cause

The default SimpleMessenger used with SUSE Enterprise Storage 4 (which is based on the Ceph Jewel release) is much more resource intensive and can cause problems on larger clusters.

Additional Information

The following is an excerpt of a related OSD failure logged to the relevant OSDs log file (/var/log/ceph/ceph-osd.X) :

     0> 2017-05-16 15:42:42.583883 7f7f5f416700 -1 *** Caught signal (Aborted) **
 in thread 7f7f5f416700 thread_name:ms_accepter

 ceph version 10.2.5-239-g3a6a822 (3a6a822c8125858afaeac7a1ee0d121d063660f0)
 1: (()+0x91e682) [0x55696038a682]
 2: (()+0x10b10) [0x7f7f779a8b10]
 3: (gsignal()+0x37) [0x7f7f7543b8d7]
 4: (abort()+0x13a) [0x7f7f7543ccaa]
 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x265) [0x556960493775]
 6: (Thread::create(char const*, unsigned long)+0xba) [0x55696046398a]
 7: (SimpleMessenger::add_accept_pipe(int)+0x66) [0x556960458e26]
 8: (Accepter::entry()+0x36b) [0x55696052f86b]
 9: (()+0x8744) [0x7f7f779a0744]
 10: (clone()+0x6d) [0x7f7f754f0d3d]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

There are however known issues with the AsyncMessenger, for more information see:

http://tracker.ceph.com/issues/18761

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:7021060
  • Creation Date: 04-Jul-2017
  • Modified Date:03-Mar-2020
    • SUSE Enterprise Storage

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback@suse.com

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

Join Our Community

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.


SUSE Customer Support Quick Reference Guide SUSE Technical Support Handbook Update Advisories
Support FAQ

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.

Go to Customer Center