Configuring a buffer with total_limit_size in Logging v2

Task

Some users run into an issue where fluentd buffers grow indefinetely and fill up the disk on nodes. This documentation explains how to limit this size in V2 Logging.

Pre-requisites

• Rancher 2.5.x and above
• Logging App installed with Cluster Explorer

Resolution

When you're ready to configure your Output (the location that the logs are sent to) you can get there with the following steps after you have Logging installed:

1. Cluster Explorer
2. Dropdown in the top left
3. Logging
4. ClusterOutput/Output
5. Create

From here, you can edit the Output as YAML to access the buffer configurations. Using ElasticSearch as an example you can fill out the form easily, but then Edit as YAML:

apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterOutput
metadata:
    name: "elasticsearch-output"
    namespace: "cattle-logging-system"
  elasticsearch:
    host: 1.2.3.4
    index_name: some-index
    port: 9200
    scheme: http
    buffer:
      type: file
      total_limit_size: 2GB

As far as the buffer size is concerned, the total_limit_size is the important parameter to change. 2GB is a good starting point and is unlikely to need to be changed, but you can adjust this based on your needs.

Further reading

As stated in the pre-reqs, this is only possible on the V2 Logging App which is available in Rancher 2.5. This uses the Banzai Cloud logging operator, which allows us to use CRDs to configure it.

• V2 Logging Documentation
• Banzai Cloud Operator Documentation