PMM Upgrades doesn't work with external postgresql db for grafana

[GenLN]

Description

Trying pmm upgrade with helm chart (helm upgrade doesn't works, so I need to run uninstall/install helm chart)
pmm-upgrade fail
TASK [initialization : Change default admin id] ******************************** fatal: [localhost]: FAILED! => {"changed": false, "msg": "unable to connect to database: connection to server on socket \"/var/run/postgresql/.s.PGSQL.5432\" failed: FATAL: database \"grafana\" does not exist\n"}

Where does this task have defined or use this config for postgresql? How to change it?

Expected Results

Upgrade done

Actual Results

Failed/stuck/no revert to previous version

Version

PMM Server 2.41.0 -> 2.41.1

Steps to reproduce

define grafana external postgres db using pmmEnv in values file:

GF_DATABASE_HOST:
GF_DATABASE_NAME:
GF_DATABASE_TYPE:
GF_DATABASE_USER:
GF_DATABASE_PASSWORD:

Relevant logs

TASK [dashboards_upgrade : Copy plugins to the plugin directory] ***************
changed: [localhost]

TASK [dashboards_upgrade : Set permissions for the plugin directory] ***********
changed: [localhost]

TASK [dashboards_upgrade : Check that the SQLite grafana database exists] ******
ok: [localhost]

TASK [dashboards_upgrade : Restart grafana before deleting data EL7] ***********
skipping: [localhost]

TASK [dashboards_upgrade : Restart grafana before deleting data EL9] ***********
skipping: [localhost]

TASK [dashboards_upgrade : Remove old DBaaS dashboard (Before provisioning times)] ***
skipping: [localhost]

TASK [dashboards_upgrade : Remove old PMM Inventory (Before provisioning times)] ***
skipping: [localhost]

TASK [dashboards_upgrade : Remove old PMM Add Instance dashboard (Before provisioning times)] ***
skipping: [localhost]

TASK [dashboards_upgrade : Remove old PMM Database Checks dashboard (Before provisioning times)] ***
skipping: [localhost]

TASK [dashboards_upgrade : Remove old PMM Settings dashboard (Before provisioning times)] ***
skipping: [localhost]

TASK [dashboards_upgrade : Remove the old clickhouse plugin] *******************
changed: [localhost]

TASK [dashboards_upgrade : Restart grafana with new plugins EL7] ***************
skipping: [localhost]

TASK [dashboards_upgrade : Restart grafana with new plugins EL9] ***************
changed: [localhost]

TASK [initialization : Create backup directory] ********************************
ok: [localhost]

TASK [initialization : Check if Postgres 11 exists] ****************************
ok: [localhost]

TASK [Upgrade Postgres database] ***********************************************
skipping: [localhost]

TASK [initialization : Create grafana database in postgres] ********************
skipping: [localhost]

TASK [initialization : Create grafana user in postgres] ************************
skipping: [localhost]

TASK [initialization : Upgrade grafana database (Get the latest schema)] *******
skipping: [localhost]

TASK [initialization : Create working directory for Alertmanager] **************
ok: [localhost]

TASK [initialization : Create working directory for VictoriaMetrics] ***********
ok: [localhost]

TASK [initialization : Create empty configuration file for VictoriaMetrics] ****
changed: [localhost]

TASK [initialization : Check that the SQLite grafana database exists] **********
ok: [localhost]

TASK [initialization : Remove database options (SQLite is default)] ************
skipping: [localhost]

TASK [initialization : Remove database host] ***********************************
skipping: [localhost]

TASK [initialization : Remove database user] ***********************************
skipping: [localhost]

TASK [initialization : Remove database password] *******************************
skipping: [localhost]

TASK [initialization : Upgrade grafana database (Get the latest schema)] *******
skipping: [localhost]

TASK [initialization : Start grafana again] ************************************
skipping: [localhost]

TASK [initialization : Wait for grafana] ***************************************
skipping: [localhost]

TASK [Migrate Grafana database from SQLite to Postgresql] **********************
skipping: [localhost]

TASK [initialization : Change default admin id] ********************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "unable to connect to database: connection to server on socket \"/var/run/postgresql/.s.PGSQL.5432\" failed: FATAL:  database \"grafana\" does not exist\n"}

PLAY RECAP *********************************************************************
localhost                  : ok=27   changed=8    unreachable=0    failed=1    skipped=25   rescued=0    ignored=0   

time="2024-03-21T10:41:22Z" level=fatal msg="RunPlaybook failed: exit status 2"
ProjectName: pmm-update
Version: 2.41.1
PMMVersion: 2.41.1
Timestamp: 2024-01-29 13:32:29 (UTC)
FullCommit: f4004eac5c03e97327ae235dbf40f91e31ba2208

Code of Conduct

• I agree to follow Percona Community Code of Conduct