008-tasks.md

008 Tasks

^ Back to parent

Tasks & Deliverables

1. Complete dev setup
2. Dockerize DB server + express server (one route)
3. Implement core features & associated tests
4. Implement additional features & tests
5. Prepare RC
6. Production version

Milestones & Issues

• 02 setup issues
• 03 core feature issues
• 04 additional feature issues
• 05 rc feature issues

Tasks

• main branch

• init git

• init npm

• install TS + tsconfig

• install OpenAPI tools

• alpha branch

• init logger

  • imported library instead using winston
  • back/lib/winston.ts
  • back/lib/winston.test.ts
  • testing standard out logs
  • testing file logs
  • back/middlewares/logger.ts
    • mode from env
    • log levels from env
    • log file/dir from env
  • back/middlewares/logger.test.ts - test logging

• init db

  • mysql connection pool
    • back/database/connector.ts
    • back/database/connector.test.ts
  • config from env
    • back/database/config.ts
    • back/database/config.test.ts
  • env file
    • back/database/devdb.env
  • utility to convert initial json to sql insert statement
    • back/database/products.json.to.sql.js
    • docker-entrypoint-initdb.d/002-database-state-insert.sql
  • automated tests with live test db
    • back/test-db.yaml
  • database creation script
    • database-model.mwb
    • docker-entrypoint-initdb.d/001-database-model.sql - back/databases/sqlite-connection.ts - [ ] import logger - [ ] install sqlite3 - [ ] sqlite file name from env - [ ] reconnect or create db - back/databases/sqlite-connection.test.ts - [ ] test sqlite file is created
  • back/server.ts
    • import logger
    • import db
  • back/server.test.ts
    • test logs
    • test db connection - [ ] test sqlite file is created

• init server

  • openapi.yaml
  • back/server.ts
    • install Express
    • port & domain from env
    • export app for tests
    • conditional call to main method
      • does not call if imported
      • calls if entrypoint
  • error handler
    • back/controllers/errorHandler.ts
    • back/controllers/errorHandler.test.ts
  • back/server.ts
    • listen
  • products route and controller
    • directly implemented a real get /products controller
      • checking relevant tasks further ahead
    • back/controllers/products-get-all.ts
      • dummy controller --> 200
    • back/controllers/products-get-all.test.ts
      • test dummy controller
    • back/routes/products.ts
      • /products sub router
    • back/routes/products.test.ts
      • test router export
  • centralized router
  • back/routes/router.ts
    • dummy route GET /products
    • error handler route
    • default 404 route
    • back/routes/router.test.ts
      • test route exports
  • back/server.test.ts
    • [ ] test listen
    • test routes
    • test error
    • test 404
    • test GET /products

• serve

  • npm script
    • npm start
  • makefile script
    • make start

• build

  • build TS->JS into back-dist directory
  • npm script
    • npm run build
  • makefile script make build-back
  • [DEPRECATED] stand alone dockerfile build of the backend
  • Docker build within docker-compose

• run build

  • npm script
    • [DEPRECATED][TO-REFACTOR] npm run built:start
  • makefile script
    • [DEPRECATED] test-built
    • make start

• list categories

  • [ ] update openapi.yaml (prepopulated)
  • Add model
    • back/Models/Categories.ts
    • Categories class
      • string name
      • number id
    • static method listFromDatabase
      • pass db error
    • static method List (alias)
  • back/Models/Categories.test.ts
    • test Categories.listFromDatabase
    • test Categories.list
  • back/controllers/categories-list-all.ts
    • import logger
    • get list from model
    • serialize to json with view
    • return response
  • back/controllers/categories-list-all.test.ts
    • test with db error
    • test without error
  • Add route
    • Add GET /categories route
      • back/routes/categories.ts
      • back/routes/router.ts
    • test export GET /categories route
      • back/routes/categories.test.ts
      • back/routes/router.test.ts
  • back/server.test.ts
    • test GET /categories

• list products — no options, no paging

  • [ ] update openapi.yaml (prepopulated)
  • back/Models/Products.ts
    • Products class
    • static method listFromDatabase
      • pass db error
    • [ ] static method databaseResponseToInstance
      • constructor throws
    • [ ] static method databaseResponseToInstanceArray
    • static method List (calls to databaseResponseToInstanceArray(listFromDatabase))
  • back/Models/Products.test.ts
    • [ ] test new Products()
    • test Products.listFromDatabase
    • [ ] test Products.databaseResponseToInstance
    • [ ] test Products.databaseResponseToInstanceArray
    • test Products.List
  • back/views/objectToJSON.ts
  • back/views/objectToJSON.test.ts
  • back/views/Products.ts
    • [ ] toJSON
    • [ ] listArrayToJSON
  • back/views/Products.test.ts
    • [ ] test toJSON
    • [ ] test listArrayToJSON
  • back/controllers/products-list-all.ts
    • import logger
    • get list from model
    • serialize to json with view
    • return response
  • back/controllers/products-list-all.test.ts
    • test with db error
    • test withouterror
    • [ ] test empty list
    • [ ] test malformed list
    • [ ] test valid list
  • update tests
    • back/routes/products.test.ts
    • back/routes/router.test.ts
    • back/server.test.ts

• view product by id

  • Add Id model
    • back/models/Id.ts
    • Id.validator static method
    • back/models/Id.test.ts
    • Id.validator tests
  • Add Products.getById static method
    • back/Models/Products.ts
    • static method getFromDatabaseById
      • pass db error
    • static method getById (calls to databaseResponseToInstance(getFromDatabaseById))
    • back/Models/Products.test.ts
    • test Products.getFromDatabaseById
    • test Products.getById
  • URL param mw validation
    • back/middleware/param-id.ts
    • back/middleware/param-id.test.ts
    • add mw to router
      • back/routes/router.ts
  • Controller
    • back/controllers/products-get-one-by-id.ts
    • import logger
    • get one product from model by id
    • serialize to json with view
    • return response
    • back/controllers/products-get-one-by-id.test.ts
      • test no id
      • test unknown/deleted id
      • test valid id
  • add GET /products/:id route
    • back/routes/products.ts
    • Add GET /products/{id} route
    • back/routes/products.test.ts
    • test export GET /products/{id} route
  • [ ] update openapi.yaml
  • back/server.test.ts
    • test GET /products/{id}

• create new product

  • Common validators
    • back/lib/validators.ts
    • back/lib/validators.test.ts
  • Model
    • back/Models/Products.ts
      • category exists
      • return product
      • Unique code
        • static method ensureUniqueActiveCode
        • stored procedure in mysql
        • MYSQL doesn't have filtered indexes
        • Function indexes exists but cannot be stored with the table in mysql workbench
        • Using aggregation column with index as workaround
        • Using insert and update triggers to populate the aggregation column
          • same value as code when deleted = 0
          • null when deleted = 1
      • constructor
        • validates object properties
        • ensure code is unique among active products
        • initialises isSaved=false instance without id
        • initialises isSaved=true instance without id
        • readonly rating
      • static method insertNewToDatabase
        • pass db error
        • calls updateInDatabase when id does not exist
      • static method updateInDatabase
        • pass db error
      • method save (calls to insertNewToDatabase)
    • back/Models/Products.test.ts
      • test new Products()
      • test Products.insertNewToDatabase
      • test save
  • Controller
    • back/controllers/products-create.ts
      • import logger
      • validate post data with model
      • 422 if invalid data
      • 409 if code exists among active products
      • 409 if category does not exist
      • save product to db with model
      • serialize to json with view
      • return response
    • back/controllers/products-create.test.ts
      • test invalid form
      • test existing code
      • test valid code
  • Route
    • update back/routes/products.ts
    • update test back/routes/products.test.ts
  • Test updates
    • back/router.test.ts
      • test export POST /products route
    • back/server.test.ts
      • test endpoint
  • update openapi.yaml

• update product

  • DB update
    • indexes
    • update_product procedure
  • Model
    • back/Models/Products.ts
      • [ ] static method databaseResponseToInstance
        • constructor throws
      • updatedFields property & tracking
      • getter isUpdated
      • method setUpdated updateField
        • integrated with field setters
        • validates one field value
        • f field is code, ensures code is unique among active products
        • update instance field value when new
        • add key to updated fields when new
      • update method
      • static updateInDatabase method
        • call procedure for rollback
    • back/Models/Products.test.ts
      • test Products.databaseResponseToInstance
      • test Products.updateField
      • test save updated
      • test updatedInDb
  • Controller
    • back/controllers/products-update-by-id.ts
      • import logger
      • get product with model from mw
      • 404 if not found
      • update fields with model
      • 422 if invalid data
      • 409 if code exists among active products
      • save product to db with model
      • serialize to json with view
      • return response
    • back/controllers/products-update-by-id.test.ts
      • test invalid form
      • test valid data
  • Routes
    • back/routes/products.ts
      • Add PATCH /products/{id} route
    • back/routes/products.test.ts
  • Test updates
    • back/router.test.ts
      • test export PATCH /products/{id} route
    • back/server.test.ts
      • test endpoint
  • [ ] update openapi.yaml

• delete product

  • Model

    • back/Models/Products.ts
      • static method setDeletedInDatabase
      • static method deleteById calls updateAsDeletedInDatabaseById
        • pass db error
      • method delete
    • back/Models/Products.test.ts
      • test Products.updateAsDeletedInDatabaseById
      • test Products.deleteById

  • Controller

    • back/controllers/products-delete-by-id.ts
      • import logger
      • call delete by id with model
      • 404 if not found
      • 500 if error
      • return 204 response
    • back/controllers/products-delete-by-id.test.ts
      • test missing id
      • test valid id

  • Routes

    • back/routes/products.ts
      • Add DELETE /products/{id} route
    • back/routes/products.test.ts

  • Test updates

    • back/router.test.ts
      • test export DELETE /products/{id} route
    • back/server.test.ts
      • test endpoint

  • [ ] update openapi.yaml

  • House keeping

    • remove unused imports
    • enforce naming conventions
      • file names
      • variables
      • functions
      • classes
      • database
    • jsdoc
      • functions
      • classes
    • consistent logging
      • log level from env
    • documentation
      • split readme
      • reorganize documentation
      • simplify & highlight patterns & choices
    • openapi
      • map api endpoints
      • map models
      • map params

• automated github actions tests

  • unit tests
  • integration tests

• merge main branch

• beta roles branch

• list roles

  • Model
    • back/Models/Roles.ts
      • Roles class
        • string name
        • number id
      • static method listFromDatabase
        • pass db error
      • static method databaseResponseToInstance
        • constructor throws
      • static method databaseResponseToInstanceArray
      • static method List (calls to databaseResponseToInstanceArray(listFromDatabase))
    • back/Models/Roles.test.ts
      • test new Roles()
      • test Roles.listFromDatabase
      • test Roles.databaseResponseToInstance
      • test Roles.databaseResponseToInstanceArray
      • test Roles.List
  • Controller
    • back/controllers/roles-list-all.ts
      • import logger
      • get list from model
      • serialize to json with view
      • return response
    • back/controllers/roles-list-all.test.ts
      • test empty list
      • test malformed list
      • test valid list
  • Routes
    • back/routes/roles.ts
      • Add GET /roles route
    • back/routes/roles.test.ts
  • Test updates
    • back/router.test.ts
      • test export GET /roles route
    • back/server.test.ts
      • test GET /roles
  • update openapi.yaml

• list users

  • Model
    • back/Models/Users.ts
      • Users class
      • static method listFromDatabase
        • pass db error
      • static method databaseResponseToInstance
        • constructor throws
      • static method databaseResponseToInstanceArray
      • static method List (calls to databaseResponseToInstanceArray(listFromDatabase))
    • back/Models/Products.test.ts
      • test new Products()
      • test Products.listFromDatabase
      • test Products.databaseResponseToInstance
      • test Products.databaseResponseToInstanceArray
      • test Products.List
  • Controller
    • back/controllers/products-list-all.ts
      • import logger
      • get list from model
      • serialize to json with view
      • return response
    • back/controllers/products-list-all.test.ts
      • test empty list
      • test malformed list
      • test valid list
  • Routes
  • Test updates
    • back/routes/router.test.ts
      • test exports GET /users route
    • back/server.test.ts
      • update test GET /users
  • update openapi.yaml

• login mockup

  • Model
    • back/models/Users.ts
      • static method getByName
    • back/models/Users.test.ts
      • test getByName
  • Controller
    • back/controllers/login.ts
      • getByName using sanitized post data
      • 501 if not found
      • serialize response using view
      • set user id in session cookie
      • return payload
    • back/controllers/login.test.ts
      • test unknown name
      • test valid name
  • Routes
    • back/routes/users.ts
      • add POST /login route
    • back/routes/users.test.ts
  • Test updates
    • back/router.test.ts
      • test export POST /login route
    • back/server.test.ts
      • test endpoint
  • update openapi.yaml

• secure auth only routes

  • back/middlewares/security-auth.ts
    • test req for session cookie & user id
  • back/middlewares/security-auth.test.ts
    • test mw
  • back/router.ts
    • POST /products
    • PATCH /products/{id}
    • DELETE /products/{id}
  • back/router.test.ts
    • test export updates
  • back/controllers/products-create.test.ts
    • update test with auth cases
  • back/controllers/products-update.test.ts
    • update test with auth cases
  • back/controllers/products-delete.test.ts
    • update test with auth cases
  • back/server.test.ts
    • update test with auth cases

• secure routes by roles

  • back/controllers/login.ts
    • list user roles using model
    • set user roles in session cookie
    • return payload
  • back/controllers/login.test.ts
    • test user roles in cookie
  • back/middlewares/security-role.ts
    • check roles in session cookie vs mw init role
  • back/middlewares/security-role.test.ts
    • test mw
  • back/router.ts
    • POST /products
    • PATCH /products/{id}
    • DELETE /products/{id}
  • back/router.test.ts
    • test export updates
  • back/controllers/products-create.test.ts
    • update test with role cases
  • back/controllers/products-update.test.ts
    • update test with role cases
  • back/controllers/products-delete.test.ts
    • update test with role cases
  • back/server.test.ts
    • update test with role cases

• merge main branch

• beta sql branch

• switch sqlite to mysql

  • dockerized db
    • back/test-db.yaml
    • docker-compose.yaml
    • database container
    • config from env/secret
  • back/makefile
    • [ ] update with db from docker-compose
  • back/package.json
    • [ ] update with db from docker-compose
  • back/databases/sqlite-connection.ts
    • [ ] remove file
  • back/databases/sqlite-connection.test.ts
    • [ ] remove file
  • back/databases/sql-connection.ts
    • [ ] import mysql library
    • [ ] auth from env/secret
  • back/databases/sql-connection.test.ts - [ ] test connection
  • back/models/* - [ ] convert SQL syntax from sqlite to mysql

• automated CI/CD github tests

  • update docker-compose.yaml
  • add necessary Dockerfiles
  • .github/workflows/test-with-db.yaml
    • configure testing env & cmd
  • test on github

• db offline failsafe

  • back/middlewares/failsafe-db-offline.ts
    • test db connection status
    • returns 503 if offline
  • back/middlewares/failsafe-db-offline.test.ts
    • test mw
  • back/server.ts
    • use mw for all
  • back/server.test.ts
    • add db offline test

• merge main branch

• beta data history branch

• save data history

  • back/models/DataHistory.ts
    • DataHistory class
    • static method log inserts to db
  • back/models/DataHistory.test.ts
    • test log
  • back/models/Products.ts
    • update to call to DataHistory.log
  • back/models/Products.test.ts
    • test call to DataHistory.log

• list data history

  • update openapi.yaml
  • back/router.ts
    • add POST /data-history
    • mw: auth, role
  • back/router.test.ts
    • test exported route
  • back/models/DataHistory.ts
    • static method list
  • back/models/DataHistory.test.ts
    • test list
  • back/views/DataHistory.ts
    • static method toJSON
    • static method listToJSONArray
  • back/views/DataHistory.test.ts
    • test list to JSONArray
  • back/controllers/data-history-list.ts
    • list using model
    • serialize using view
    • return payload
  • back/controllers/data-history-list.test.ts
    • test controller
  • back/server.test.ts
    • add list data history test

• merge main branch