Updated Ceph Recommendations | SUSE Communities

Updated Ceph Recommendations


Periodically, it makes sense to revisit and update recommendations based on what we see in the field and what is happening with a project’s code base.  This is one of those updates.

In the past, we have recommended the use of two physically separate network fabrics to help better manage the I/O paths and allow for the back end traffic to have a dedicated link without other contention.  We are now changing that recommendation to reduce complexity and improve general availability.  The new recommendation is as follows:

  • Stack Top of Rack Network Switches
    • Create 802.3ad (LACP) bonding groups across the switches to protect against switch failure
  • Create a single bonded interface on the SUSE Enterprise Storage node
    • Use mode 4 (802.3ad)
  • Use VLANs to logically segment traffic

It is important to remember to architect the environment with more bandwidth for the cluster than the outbound links to the clients.  This will help ensure the cluster has the bandwidth resources needed for replication, recovery, and other back end actions.  If your switches support it, it might also be worth using QoS to ensure that the backend VLAN has higher performance priority than the client-facing VLAN.

You should still continue to utilize jumbo frames for the back-end, private network, and if possible, the public network.  These larger frame sizes provide a significant boost in data payload and potential throughput to the storage.  It is important to note that you should NOT mix frame sizes on the same VLAN or try to have a routing device convert from standard to jumbo MTUs, as this will quickly overrun the capability of processors on most switching infrastructure.

From a security perspective, the VLANs will control where the data goes, thus keeping those back end flows isolated to the storage networking switches.

This change will be reflected in the documentation for our next product release and is present in our current reference architectures that I discussed in a previous blog.  For your reference, the updated papers are linked below.

SUSE Enterprise Storage Architectural Overview with Recommendations

SUSE Enterprise Storage Deployment for HPE Proliant DL Series




  • Avatar photo toddsimmer says:

    any recommendations on when using Suse Storage with OpenStack?

    • Avatar photo davidbyte says:

      Great question! The answer is, “It depends.” The depends is about workload. If you are using the storage for virtual machines, the networking recommendations in the post are appropriate. The biggest variable is in regards to disk type, journal usage, etc. In general, I would suggest an architecture similar to what is presented in the HPE ProLiant DL series document in the post above.

  • Avatar photo rushdir005 says:

    I am fascinated reading your blog posts. The one which really got my attention was on the lab setup you had blogged.

    I would like to get some advise on setting up a poor man’s lab to test SUSE Enterprise Storage using either VMware Workstation or Oracle virtualbox for learning purposes.

    Appreciate if you could let me know if its possible and it will be a great help if you can guide me in a step by step process.

  • Leave a Reply

    Your email address will not be published. Required fields are marked *

    Avatar photo