rancher-logging: how to control fluentd logs flushing frequency

By default fluentd in rancher-logging flushes logs every 11 minutes as it uses timekey for the range of logs which is set to 10 minutes and timekey_wait set to 1 minute.

 <buffer tag,time>
      @type file
      chunk_limit_size 8MB
      path /buffers/clusterflow:cattle-logging-system:test-es-flow-1:clusteroutput:cattle-logging-system:test-es-1.*.buffer
      retry_forever true
      timekey 10m
      timekey_wait 1m
    </buffer>

You have two parameters to manage how often fluentd should flush logs. The timekey specifies the time range of logs that should be grouped in chunks. With timekey of 10m, the chunk will contain logs within a 10 minutes time range.

Example: timekey 10m: ["12:00:00", ..., "12:09:59"], ["12:10:00", ..., "12:19:59"]

The timekey_wait  configures the flush delay for events. Below is an example to illustrate this

timekey 10m
-------------------------------------------------------
time range for chunk | timekey_wait | actual flush time
12:00:00 - 12:09:59  | 60s                 | 12:11:00
 

The two parameters are to be specified at the output/clusteroutput level to manage the logs flushing frequency from fluentd. At the creation, you have a section called Output Buffer (Logging > Output > Create > Output Buffer or Logging > ClusterOutput > Create > Output Buffer) where this can be changed, or if you are using kubectl command line, then you can update the buffer as shown in the example below.

<buffer tag,time>
      @type file
      chunk_limit_size 8MB
      path /buffers/clusterflow:cattle-logging-system:test-es-flow-1:clusteroutput:cattle-logging-system:test-es-1.*.buffer
      retry_forever true
      timekey 3m
      timekey_wait 1m
    </buffer>