Configuring a buffer with total_limit_size in Logging v2
This document (000020013) is provided subject to the disclaimer at the end of this document.
Some users run into an issue where fluentd buffers grow indefinetely and fill up the disk on nodes. This documentation explains how to limit this size in V2 Logging.
- Rancher 2.5.x and above
- Logging App installed with Cluster Explorer
When you're ready to configure your Output (the location that the logs are sent to) you can get there with the following steps after you have Logging installed:
- Cluster Explorer
- Dropdown in the top left
From here, you can edit the Output as YAML to access the buffer configurations. Using ElasticSearch as an example you can fill out the form easily, but then Edit as YAML:
apiVersion: logging.banzaicloud.io/v1beta1 kind: ClusterOutput metadata: name: "elasticsearch-output" namespace: "cattle-logging-system" elasticsearch: host: 184.108.40.206 index_name: some-index port: 9200 scheme: http buffer: type: file total_limit_size: 2GB
As far as the buffer size is concerned, the
total_limit_size is the important parameter to change. 2GB is a good starting point and is unlikely to need to be changed, but you can adjust this based on your needs.
As stated in the pre-reqs, this is only possible on the V2 Logging App which is available in Rancher 2.5. This uses the Banzai Cloud logging operator, which allows us to use CRDs to configure it.
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000020013
- Creation Date: 06-May-2021
- Modified Date:06-May-2021
- SUSE Rancher
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com