Use parameter store instead of or in addition to credentials files

[tvogels01]

Summary

Instead of copying credentials files around and updating for everybody when passwords change, create passwords (to access upstream databases or Redshift clusters) in the AWS Systems Manager Parameter Store.

Details

Initial steps

An initial implementation could just allow a special syntax in the DSN to trigger a lookup in the parameter store.

Example:

WEBAPP_DSN="ssm://etl/dsn/webapp_dsn"

Lookup with overrides

Suggested parameter organization:

prefix: DW-ETL/
hierarchical search:
general: <dw prefix>/<env_type>/<user prefix>/<name>

e.g. looking for connection to "WEBSERVER_REPLICA" in environment development when running validation:
    look for DW-ETL/dev/development/validation/WEBSERVER_REPLICA
             DW-ETL/dev/development/WEBSERVER_REPLICA
             DW-ETL/dev/WEBSERVER_REPLICA

When do overrides occur?

• pick different base stack
• pick different environment (development vs. tom vs. tom/validation)