Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

performance drop due to incorrect logql type parsing #16653

Open
ftong2020 opened this issue Mar 9, 2025 · 0 comments
Open

performance drop due to incorrect logql type parsing #16653

ftong2020 opened this issue Mar 9, 2025 · 0 comments

Comments

@ftong2020
Copy link

Describe the bug

Image

Loki's query parser incorrect parse query

{app="my-app", service="my-service"} |= `some-filter-content` |= ``

as "limit" type instead of filter, which incorrectly set query parallelism to 1, leading to drastic performance drop.

This actually happens a lot since some one browsing logs in grafana and use grafana menu to add filter will easily trigger this. For logs that reaches 100s of gigs, log volume query will finish very fast(with parallel query), but log content itself seems stuck and never returns.

To Reproduce
Steps to reproduce the behavior:

  1. Started Loki 3.4.2 in distributed mode
  2. Query {app="my-app", service="my-service"} |= some-filter-content |= ``
  3. Find logs in query-frontend, confirm it is parsed as limited query

Expected behavior
Query above should be recognized as filter type, thus utilize all available queriers.

Environment:

  • Infrastructure: kubernetes
  • Deployment tool: helm

Screenshots, Promtail config, or terminal output
There is already a unit test in logql module, I added two lines to demonstrate this issue.

Image

@ftong2020 ftong2020 changed the title incorrect log query type parsing performance drop due to incorrect logql type parsing Mar 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant