Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUGFIX]: 502 error on submission-trends #602

Closed
1 of 3 tasks
craigyu opened this issue Feb 7, 2025 · 2 comments · Fixed by #618 · May be fixed by #619
Closed
1 of 3 tasks

[BUGFIX]: 502 error on submission-trends #602

craigyu opened this issue Feb 7, 2025 · 2 comments · Fixed by #618 · May be fixed by #619
Assignees
Labels
bug Something isn't working

Comments

@craigyu
Copy link
Collaborator

craigyu commented Feb 7, 2025

Bug Summary

502 error preventing the yearly trend to be displayed

Describe the Bug

The openings submission trends widget cannot load due to a network error

Steps to Reproduce

  1. to to silva prod
  2. select dashboard then wait for the openings submission trends to load

Image

Expected Behavior

Load data and shows graph

Actual Behavior

No response

Screenshots

Affected Environments

  • Development
  • Staging
  • Production

Desktop Information

No response

Additional Context

No response

@craigyu craigyu added the bug Something isn't working label Feb 7, 2025
@paulushcgcj
Copy link
Contributor

This issue relates to #592 and will be addressed together with it.

@paulushcgcj paulushcgcj self-assigned this Feb 10, 2025
@paulushcgcj paulushcgcj linked a pull request Feb 12, 2025 that will close this issue
11 tasks
@paulushcgcj
Copy link
Contributor

After some investigation, I was able to pin down this issue and split it into a two-part fix. The root cause of the issue is due to the way the submission trend is calculated. It is done by listing the entries that match the specific conditions (date and some parameters) and then aggregating and counting them. The problem with this approach is that it uses a lot of resources for aggregation, as it will quadruple the memory used for resource loading due to the mapping. Due to that, the maximum amount of memory that the application could use would go through the roof, and the application would restart with an Out of Memory exception that is not that easy to capture in a cloud environment. To fix this, we came up with the following changes:

The first part is being addressed by the PR 618 by increasing the resource limit for the application. This should solve not only this issue but some other issues as well.

The second part is being addressed by the PR 619, where the aggregation code was changed and the heaviest part of the aggregation was moved to the query itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants