Skip to content

Commit 381ff40

Browse files
committed
log-driver-buffer-limit: add error message emitted when buffer is full
Signed-off-by: Wesley Pettit <[email protected]>
1 parent 4f695bf commit 381ff40

File tree

1 file changed

+8
-1
lines changed
  • examples/fluent-bit/log-driver-buffer-limit

1 file changed

+8
-1
lines changed

examples/fluent-bit/log-driver-buffer-limit/README.md

+8-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,14 @@
22

33
The `log-driver-buffer-limit` option is now supported on ECS EC2 and ECS Fargate with PV1.4+. This setting tells FireLens to configure the Fluentd Docker Log Driver field [fluentd-buffer-limit](https://docs.docker.com/config/containers/logging/fluentd/#fluentd-buffer-limit). This setting applies to the log driver that FireLens uses [under the hood](https://aws.amazon.com/blogs/containers/under-the-hood-firelens-for-amazon-ecs-tasks/) to capture stdout & stderr container logs and send them to Fluentd or Fluent Bit.
44

5-
As we know, FireLens is a container log router for Amazon ECS and AWS Fargate that gives customers extensibility to use the breadth of services at AWS or partner solutions for log analytics and storage. FireLens works with Fluentd and Fluent Bit and makes it easy to use these two popular open source logging projects.
5+
If the Fluentd buffer limit is insufficient, log loss can occur. The Fluentd log driver in the container runtime will emit the following [error message](https://github.com/fluent/fluent-logger-golang/blob/v1.9.0/fluent/fluent.go#L422) when logs are lost:
6+
```
7+
fluent#appendBuffer: Buffer full
8+
```
9+
10+
On EC2, the Docker container runtime logs can be collected from the system journal with `sudo journalctl -fu docker.service`. On Fargate, container runtime logs are not available to customers. The buffer limit setting controls the [maximum size of the pending log record buffer](https://github.com/fluent/fluent-logger-golang/blob/v1.9.0/fluent/fluent.go#L181) in the log driver; configure a large enough size to ensure there is no log loss for your workload.
11+
12+
FireLens is a container log router for Amazon ECS and AWS Fargate that gives customers extensibility to use the breadth of services at AWS or partner solutions for log analytics and storage. FireLens works with Fluentd and Fluent Bit and makes it easy to use these two popular open source logging projects.
613

714
![FireLens](https://d2908q01vomqb2.cloudfront.net/fe2ef495a1152561572949784c16bf23abb28057/2019/11/16/Screen-Shot-2019-09-26-at-5.21.35-PM-1024x572.png)
815

0 commit comments

Comments
 (0)