Allow configuring the format of EventTimeFilter #11376
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #11331
Problem
Currently, EventTimeFilter is formatted as YYYY-MM-DD HH:MM:SS+00:00. However this format is not suitable for the following cases:
referencing tables where event_time isn't of type timestamp (especially in BigQuery).
For BigQuery, it's addressed by dbt-labs/dbt-bigquery#1422.
But casting event_time to timestamp causes a full scan.
When event_time is of type date, it's preferable to filter referenced tables using event_time > 'YYYY-MM-DD'
referencing sharded tables (especially in BigQuery).
Daily sharded tables are usually have a suffix like 'YYYYMMDD'.
It's better to filter referenced tables using _table_suffix > 'YYYY-MM-DD'.
Solution
Added format configuration to EventTimeFilter.
This PR depends on dbt-labs/dbt-adapters#893.
Checklist