Gaps in the logs after the data is transferred off to S3?

[kevintijssen]

Describe the bug
Logs are missing after they are push to a S3 bucket.?
When a new deployment is created the old logs are coming back after sometime.
The new logs are also incoming but when its transferred to S3 they are gone, until a new deployment is created or a rollback.
This is just happening for 1 deployment. There are no errors related to this issue within the Loki components

Already upgrade Loki from 3.1.1 to 3.3.2

Expected behavior
This should not happen. Logs should be present

Environment:

• Infrastructure: Digital Ocean, Kubernetes (1.29.5)
• Deployment tool: Helm
• Loki: SimpleScalable, 3.3.2 (https://artifacthub.io/packages/helm/grafana/loki)
• Alloy: 1.6.1 (https://artifacthub.io/packages/helm/grafana/k8s-monitoring/1.6.1)

Screenshots, Promtail config, or terminal output