Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Add GWO ingest missing data detection #631

Closed
wants to merge 3 commits into from
Closed

Conversation

gsantia
Copy link
Contributor

@gsantia gsantia commented Apr 26, 2021

Closes #626.

Enable detection of missing data in the GWO ingest process and report it through logging and Events. The specific data being examined is detailed in the linked issue.

@gsantia gsantia added the feature New functionality label Apr 26, 2021
@gsantia gsantia self-assigned this Apr 26, 2021
@gsantia gsantia force-pushed the ingest-task-genomic-workflow-output branch from 4f0f8be to a617366 Compare April 28, 2021 14:10
@gsantia gsantia force-pushed the missing-ingest branch 3 times, most recently from 35096a1 to e7b6d4e Compare April 29, 2021 23:17
@gsantia gsantia linked an issue Apr 29, 2021 that may be closed by this pull request
@gsantia gsantia force-pushed the ingest-task-genomic-workflow-output branch 2 times, most recently from 036936f to abfdcb9 Compare April 30, 2021 18:12
@codecov
Copy link

codecov bot commented May 1, 2021

Codecov Report

Merging #631 (3b9eaa7) into master (66cb07b) will increase coverage by 0.02%.
The diff coverage is 97.67%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #631      +/-   ##
==========================================
+ Coverage   93.05%   93.08%   +0.02%     
==========================================
  Files         132      132              
  Lines        5776     5810      +34     
==========================================
+ Hits         5375     5408      +33     
- Misses        401      402       +1     
Impacted Files Coverage Δ
creator/events/models.py 100.00% <ø> (ø)
creator/ingest_runs/genomic_data_loader.py 94.96% <97.61%> (+0.67%) ⬆️
creator/ingest_runs/tasks/ingest_run.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 66cb07b...3b9eaa7. Read the comment docs.

@gsantia gsantia force-pushed the missing-ingest branch 6 times, most recently from 3727700 to 04a3084 Compare May 5, 2021 21:01
@gsantia gsantia changed the base branch from ingest-task-genomic-workflow-output to master May 5, 2021 21:01
@gsantia gsantia force-pushed the missing-ingest branch 5 times, most recently from 1f95238 to 5822819 Compare May 7, 2021 20:22
@gsantia gsantia marked this pull request as ready for review May 7, 2021 20:37
@gsantia gsantia changed the title ✨ Add functionality to detect and report missing data in GWO ingest ✨ Add missing data detection in GWO ingest May 14, 2021
@gsantia gsantia changed the title ✨ Add missing data detection in GWO ingest ✨ Add GWO ingest missing data detection May 14, 2021
@gsantia gsantia force-pushed the missing-ingest branch 3 times, most recently from 40b7355 to 30e348b Compare May 19, 2021 19:07
@gsantia gsantia requested a review from znatty22 May 19, 2021 19:07
@gsantia gsantia force-pushed the missing-ingest branch 2 times, most recently from faa20c0 to 3b9eaa7 Compare May 19, 2021 19:22
Copy link
Member

@znatty22 znatty22 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From browsing through the files, things look good. Have you tested with Data Tracker after making these last round of changes? Also looks like this needs a rebase

@gsantia
Copy link
Contributor Author

gsantia commented Jun 22, 2021

From browsing through the files, things look good. Have you tested with Data Tracker after making these last round of changes? Also looks like this needs a rebase

I might have. But it's been a while and I don't remember, so I will.

@znatty22
Copy link
Member

Closing due to inactivity

@znatty22 znatty22 closed this Nov 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Detect and report missing data in GWO ingest
2 participants