Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/merge trends and measurements #336

Merged
merged 5 commits into from
Feb 15, 2024

Conversation

caparker
Copy link
Collaborator

@caparker caparker commented Feb 14, 2024

Adds the trends methods: day of week, hour of day and month of year to the sensors/:id/measurements endpoint. To do this we had to use the period model instead of the factor model and I think there could be some improvements we could make to that model. For example, when grouping by day the period has a straight forward first and last datetime of that period. But when grouping by day of week its more complicated.
Closes issue #335

russbiggs and others added 5 commits December 22, 2023 09:49
* Fix/v2 measurements source (#314)

* fix v2 measurements to query measurements table instead of hourly

* realias to match query params

* measurements geom fix

* Added sensors_id to speed up sorting

* update statement_timeout to milliseconds

---------

Co-authored-by: Russ Biggs <[email protected]>

* set statement_timeout default config value

---------

Co-authored-by: Christian Parker <[email protected]>
* Fix/v2 measurements source (#314)

* fix v2 measurements to query measurements table instead of hourly

* realias to match query params

* measurements geom fix

* Added sensors_id to speed up sorting

* update statement_timeout to milliseconds

---------

Co-authored-by: Russ Biggs <[email protected]>

* ratelimit headers (#315)

* ratelimit headers

* dynamic ratelimt-policy values

* Added wait_for method (#317)

---------

Co-authored-by: Christian Parker <[email protected]>
@russbiggs russbiggs merged commit cb25abb into release-20240214 Feb 15, 2024
russbiggs added a commit that referenced this pull request Feb 15, 2024
* Feature/merge trends and measurements (#336)

* Fix/statement timeout (#316)

* Fix/v2 measurements source (#314)

* fix v2 measurements to query measurements table instead of hourly

* realias to match query params

* measurements geom fix

* Added sensors_id to speed up sorting

* update statement_timeout to milliseconds

---------

Co-authored-by: Russ Biggs <[email protected]>

* set statement_timeout default config value

---------

Co-authored-by: Christian Parker <[email protected]>

* Release 20231222 (#318)

* Fix/v2 measurements source (#314)

* fix v2 measurements to query measurements table instead of hourly

* realias to match query params

* measurements geom fix

* Added sensors_id to speed up sorting

* update statement_timeout to milliseconds

---------

Co-authored-by: Russ Biggs <[email protected]>

* ratelimit headers (#315)

* ratelimit headers

* dynamic ratelimt-policy values

* Added wait_for method (#317)

---------

Co-authored-by: Christian Parker <[email protected]>

* Added the trends methods (dow, hod, moy) to the measurements

---------

Co-authored-by: Russ Biggs <[email protected]>

* Feature/add license to location (#337)

* Fix/statement timeout (#316)

* Fix/v2 measurements source (#314)

* fix v2 measurements to query measurements table instead of hourly

* realias to match query params

* measurements geom fix

* Added sensors_id to speed up sorting

* update statement_timeout to milliseconds

---------

Co-authored-by: Russ Biggs <[email protected]>

* set statement_timeout default config value

---------

Co-authored-by: Christian Parker <[email protected]>

* Release 20231222 (#318)

* Fix/v2 measurements source (#314)

* fix v2 measurements to query measurements table instead of hourly

* realias to match query params

* measurements geom fix

* Added sensors_id to speed up sorting

* update statement_timeout to milliseconds

---------

Co-authored-by: Russ Biggs <[email protected]>

* ratelimit headers (#315)

* ratelimit headers

* dynamic ratelimt-policy values

* Added wait_for method (#317)

---------

Co-authored-by: Christian Parker <[email protected]>

* Added the licsense list object to the locations

---------

Co-authored-by: Russ Biggs <[email protected]>

---------

Co-authored-by: Christian Parker <[email protected]>
caparker added a commit that referenced this pull request Jun 4, 2024
* hot-fix resolve auth issues

* add redis in startup event hook

* Release 20230927 (#291)

* Added ability to increase the working memory for a given endpoint (#282) resolves #281

* Added ability to increase the working memory for a given endpoint  resolves #281

* update staging deploy to deploy on release branches (#284)

* Added timer (#280)

* included order_by in v1 and v2 measurements (#263)

Co-authored-by: Gabriel Fosse <[email protected]>

* manufacturers resource (#273)

* manufacturers resource resolves #252
---------

Co-authored-by: Gabriel Fosse <[email protected]>

* Adding `v3/instruments` resource (#271)

* instruments resource resolves #270 

---------

Co-authored-by: Gabriel Fosse <[email protected]>

* Added timer

* Added owner router

---------

Co-authored-by: Gabriel Fosse <[email protected]>
Co-authored-by: Gabriel Fosse <[email protected]>
Co-authored-by: Russ Biggs <[email protected]>

* Add sort and order by query params to v3 endpoints  (#285)

* order by classes

* refactor query builder to handle sort and order by statements

* add sort and order by to v3 endpoints

resolves #163
---------

Co-authored-by: Gabriel Fosse <[email protected]>

* Feature/working memory query 281 (#287)



* Moved the tr.start out of if block

  Fixes bug #286

---------

Co-authored-by: Christian Parker <[email protected]>

* fix missing start variable and return empty string for order (#288)

* Remove count fields (#289)

* removed count fields
---------

Co-authored-by: Gabriel Fosse <[email protected]>

* fix failing unit test (#292)

* rate limiter logging fix (#293)

* remove summary count fields from v3 countries (#294)

* removed locations_count from instruments.py query

---------

Co-authored-by: Christian Parker <[email protected]>
Co-authored-by: Gabriel Fosse <[email protected]>
Co-authored-by: Gabriel Fosse <[email protected]>

* revert to old startup and shutdown events (#296)

* add redis client to app state (#299)

* fixed parameters query param for locations, countries, and providers (#306) (#308)

* fixed parameters query param for locations, countries
and providers

* Made requested changes to implement
ParametersQuery

---------

Co-authored-by: Gabriel Fosse <[email protected]>
Co-authored-by: Gabriel Fosse <[email protected]>

* add tiles parameter query (#310)

* Update test_v3_queries.py file to use the array overlap operator instead of the equality operator. (#312)

Co-authored-by: Gabriel Fosse <[email protected]>

* Fix/statement timeout (#316)

* Fix/v2 measurements source (#314)

* fix v2 measurements to query measurements table instead of hourly

* realias to match query params

* measurements geom fix

* Added sensors_id to speed up sorting

* update statement_timeout to milliseconds

---------

Co-authored-by: Russ Biggs <[email protected]>

* set statement_timeout default config value

---------

Co-authored-by: Christian Parker <[email protected]>

* Release 20231222 (#318)

* Fix/v2 measurements source (#314)

* fix v2 measurements to query measurements table instead of hourly

* realias to match query params

* measurements geom fix

* Added sensors_id to speed up sorting

* update statement_timeout to milliseconds

---------

Co-authored-by: Russ Biggs <[email protected]>

* ratelimit headers (#315)

* ratelimit headers

* dynamic ratelimt-policy values

* Added wait_for method (#317)

---------

Co-authored-by: Christian Parker <[email protected]>

* Release 20240104 (#320)

* Fix/v2 measurements source (#314)

* fix v2 measurements to query measurements table instead of hourly

* realias to match query params

* measurements geom fix

* Added sensors_id to speed up sorting

* update statement_timeout to milliseconds

---------

Co-authored-by: Russ Biggs <[email protected]>

* ratelimit headers (#315)

* ratelimit headers

* dynamic ratelimt-policy values

* Fix/statement timeout (#316)

* Fix/v2 measurements source (#314)

* fix v2 measurements to query measurements table instead of hourly

* realias to match query params

* measurements geom fix

* Added sensors_id to speed up sorting

* update statement_timeout to milliseconds

---------

Co-authored-by: Russ Biggs <[email protected]>

* set statement_timeout default config value

---------

Co-authored-by: Christian Parker <[email protected]>

* Added wait_for method (#317)

* update default date for date_form

---------

Co-authored-by: Christian Parker <[email protected]>

* Release 20240109 (#324)

* add hint to 408 timeout error (#322)

* Fix/bad timeout value (#323)

---------

Co-authored-by: Christian Parker <[email protected]>

* Bad timeout value passed to db class (#321)
  The averages endpoint was passing a config value but unassigned and
  it was therefor being passed as the timeout value. I added a check
  in the db to make sure we are protected from that as well as updated
  the averages endpoint to explicitly name the config argument.

fixes #321

---------

Co-authored-by: Christian Parker <[email protected]>

---------

Co-authored-by: Christian Parker <[email protected]>

* Readme redis update (#327)

* readme updates

* readme typo

* Release 20240207 (#331)

* Feature/ratelimiter update (#330)

* updated request_is_limited algorithm

* Strip out numbers from paths

  This is so we can better aggregate by path in cloudwatch

---------

Co-authored-by: Christian Parker <[email protected]>

* update key setting for ratelimiting

* Release 20240210 (#334)

* Provide numeric value for dates for easier analysis later (#332)

* Feature/ratelimiter update (#330)

* updated request_is_limited algorithm

* Strip out numbers from paths

  This is so we can better aggregate by path in cloudwatch

* Provide numeric value for dates for easier analysis later

---------

Co-authored-by: Russ Biggs <[email protected]>

* Sensors by location (#297)

* Added ability to increase the working memory for a given endpoint (#282) resolves #281

* Added ability to increase the working memory for a given endpoint  resolves #281

* update staging deploy to deploy on release branches (#284)

* Added timer (#280)

* included order_by in v1 and v2 measurements (#263)

Co-authored-by: Gabriel Fosse <[email protected]>

* manufacturers resource (#273)

* manufacturers resource resolves #252
---------

Co-authored-by: Gabriel Fosse <[email protected]>

* Adding `v3/instruments` resource (#271)

* instruments resource resolves #270 

---------

Co-authored-by: Gabriel Fosse <[email protected]>

* Added timer

* Added owner router

---------

Co-authored-by: Gabriel Fosse <[email protected]>
Co-authored-by: Gabriel Fosse <[email protected]>
Co-authored-by: Russ Biggs <[email protected]>

* Add sort and order by query params to v3 endpoints  (#285)

* order by classes

* refactor query builder to handle sort and order by statements

* add sort and order by to v3 endpoints

resolves #163
---------

Co-authored-by: Gabriel Fosse <[email protected]>

* Feature/working memory query 281 (#287)



* Moved the tr.start out of if block

  Fixes bug #286

---------

Co-authored-by: Christian Parker <[email protected]>

* fix missing start variable and return empty string for order (#288)

* Remove count fields (#289)

* removed count fields
---------

Co-authored-by: Gabriel Fosse <[email protected]>

* fix failing unit test (#292)

* rate limiter logging fix (#293)

* location sensors query

* location sensors path query

* Cleaned up the sensors endpoints

  Rewrote the queries to speed them up a litte and added the
  measurements method

* Removed old method that I had renamed

---------

Co-authored-by: Christian Parker <[email protected]>
Co-authored-by: Russ Biggs <[email protected]>
Co-authored-by: Gabriel Fosse <[email protected]>

* add deprecation warning (#333)

* upgrdate to actions/checkout@v4

---------

Co-authored-by: Christian Parker <[email protected]>
Co-authored-by: Gabriel Fosse <[email protected]>
Co-authored-by: Gabriel Fosse <[email protected]>

* Release 20240214 (#338)

* Feature/merge trends and measurements (#336)

* Fix/statement timeout (#316)

* Fix/v2 measurements source (#314)

* fix v2 measurements to query measurements table instead of hourly

* realias to match query params

* measurements geom fix

* Added sensors_id to speed up sorting

* update statement_timeout to milliseconds

---------

Co-authored-by: Russ Biggs <[email protected]>

* set statement_timeout default config value

---------

Co-authored-by: Christian Parker <[email protected]>

* Release 20231222 (#318)

* Fix/v2 measurements source (#314)

* fix v2 measurements to query measurements table instead of hourly

* realias to match query params

* measurements geom fix

* Added sensors_id to speed up sorting

* update statement_timeout to milliseconds

---------

Co-authored-by: Russ Biggs <[email protected]>

* ratelimit headers (#315)

* ratelimit headers

* dynamic ratelimt-policy values

* Added wait_for method (#317)

---------

Co-authored-by: Christian Parker <[email protected]>

* Added the trends methods (dow, hod, moy) to the measurements

---------

Co-authored-by: Russ Biggs <[email protected]>

* Feature/add license to location (#337)

* Fix/statement timeout (#316)

* Fix/v2 measurements source (#314)

* fix v2 measurements to query measurements table instead of hourly

* realias to match query params

* measurements geom fix

* Added sensors_id to speed up sorting

* update statement_timeout to milliseconds

---------

Co-authored-by: Russ Biggs <[email protected]>

* set statement_timeout default config value

---------

Co-authored-by: Christian Parker <[email protected]>

* Release 20231222 (#318)

* Fix/v2 measurements source (#314)

* fix v2 measurements to query measurements table instead of hourly

* realias to match query params

* measurements geom fix

* Added sensors_id to speed up sorting

* update statement_timeout to milliseconds

---------

Co-authored-by: Russ Biggs <[email protected]>

* ratelimit headers (#315)

* ratelimit headers

* dynamic ratelimt-policy values

* Added wait_for method (#317)

---------

Co-authored-by: Christian Parker <[email protected]>

* Added the licsense list object to the locations

---------

Co-authored-by: Russ Biggs <[email protected]>

---------

Co-authored-by: Christian Parker <[email protected]>

* Release 20240220 (#343)

* Feature/auth api (#340)

* auth api updates

* password reset (#341)

* Feature/password email (#342)

* password reset

* password email

* Feature/merge trends and measurements (#339)

* Fix/statement timeout (#316)

* Fix/v2 measurements source (#314)

* fix v2 measurements to query measurements table instead of hourly

* realias to match query params

* measurements geom fix

* Added sensors_id to speed up sorting

* update statement_timeout to milliseconds

---------

Co-authored-by: Russ Biggs <[email protected]>

* set statement_timeout default config value

---------

Co-authored-by: Christian Parker <[email protected]>

* Release 20231222 (#318)

* Fix/v2 measurements source (#314)

* fix v2 measurements to query measurements table instead of hourly

* realias to match query params

* measurements geom fix

* Added sensors_id to speed up sorting

* update statement_timeout to milliseconds

---------

Co-authored-by: Russ Biggs <[email protected]>

* ratelimit headers (#315)

* ratelimit headers

* dynamic ratelimt-policy values

* Added wait_for method (#317)

---------

Co-authored-by: Christian Parker <[email protected]>

* Added the trends methods (dow, hod, moy) to the measurements

* Updated the trends and raw methods

  Added better period calculating for both

* Added validation check for dates

  Still has error related to serializing a datetime object

* Added work around for the value error + datetime issue

  Also fixed syntax errors in the bbox validation

---------

Co-authored-by: Russ Biggs <[email protected]>

* add placeholder for explorer_api_key

---------

Co-authored-by: Christian Parker <[email protected]>

* Removed casting of datetime objects

* Added summary back

* Updated to use new `as_timestamptz` functions

* update prod ratelimiting

* v3 auth updates

* Restrict access to purpleair until we can remove data

* Changed tests to fit api

* add token register endpoint (#345) (#346)

* Allow the latest value to be empty (#351)

Fixes #349

---------

Co-authored-by: Russ Biggs <[email protected]>
Co-authored-by: Gabriel Fosse <[email protected]>
Co-authored-by: Gabriel Fosse <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants