Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate bespoke validation tests to dbt #4093

Open
aesharpe opened this issue Feb 28, 2025 · 0 comments · May be fixed by #4114
Open

Migrate bespoke validation tests to dbt #4093

aesharpe opened this issue Feb 28, 2025 · 0 comments · May be fixed by #4114
Assignees
Labels
data-validation Issues related to checking whether data meets our quality expectations. dbt Issues related to the data build tool aka dbt testing Writing tests, creating test data, automating testing, etc.

Comments

@aesharpe
Copy link
Member

aesharpe commented Feb 28, 2025

Here is a spreadsheet documenting all of our existing tests which need to be migrated to the new framework.

Tests labeled bespoke don't use any common validate.py methods and might require custom handling. I've added a difficulty tag to each of these tests based on my initial vibe when looking at these tests. Those labeled easy indicate that I think these tests can be easily reproduced using off-the-shelf macros provided by dbt_expectations or dbt_utils. medium labels indicate that I don't think off-the-shelf macros will work, but it seemed like they could be recreated with custom sql without any major road-blocks. hard tests indicate that the test felt like it would not be easy/straightforward to develop the test in SQL. I spent pretty minimal time evaluating these tests, so some of these labels may end up being inaccurate.

Old tests to replace:

@aesharpe aesharpe added testing Writing tests, creating test data, automating testing, etc. dbt Issues related to the data build tool aka dbt data-validation Issues related to checking whether data meets our quality expectations. labels Feb 28, 2025
@aesharpe aesharpe changed the title Migrate bespoke validation tests to DBT Migrate bespoke validation tests to dbt Feb 28, 2025
@krivard krivard linked a pull request Mar 5, 2025 that will close this issue
22 tasks
@aesharpe aesharpe moved this from New to Backlog in Catalyst Megaproject Mar 10, 2025
@aesharpe aesharpe moved this from Backlog to In progress in Catalyst Megaproject Mar 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data-validation Issues related to checking whether data meets our quality expectations. dbt Issues related to the data build tool aka dbt testing Writing tests, creating test data, automating testing, etc.
Projects
Status: In progress
Development

Successfully merging a pull request may close this issue.

2 participants