config: support loading of elasticsearch params separatelly

[ppanero]

In order to provide a complex ES config:

For example:

INVENIO_SEARCH_ELASTIC_HOSTS="[{'host': 'localhost', 'url_prefix': '/es', 'timeout': 30, 'port': 443, 'use_ssl': True, 'verify_certs': False, 'http_auth':('USERNAME_CHANGEME', 'PASSWORD_CHANGEME')}]"

Currently the only way is to load a full dictionary in a single environment variable. Something similar to what CERNDocumentServer has could be ported here:

# Search
# ======
ELASTICSEARCH_HOST = os.environ.get('ELASTICSEARCH_HOST', 'localhost')
ELASTICSEARCH_PORT = int(os.environ.get('ELASTICSEARCH_PORT', '9200'))
ELASTICSEARCH_USER = os.environ.get('ELASTICSEARCH_USER')
ELASTICSEARCH_PASSWORD = os.environ.get('ELASTICSEARCH_PASSWORD')
ELASTICSEARCH_URL_PREFIX = os.environ.get('ELASTICSEARCH_URL_PREFIX', '')
ELASTICSEARCH_USE_SSL = _parse_env_bool('ELASTICSEARCH_USE_SSL')
ELASTICSEARCH_VERIFY_CERTS = _parse_env_bool('ELASTICSEARCH_VERIFY_CERTS')

es_host_params = {
    'host': ELASTICSEARCH_HOST,
    'port': ELASTICSEARCH_PORT,
}
if ELASTICSEARCH_USER and ELASTICSEARCH_PASSWORD:
    es_host_params['http_auth'] = (ELASTICSEARCH_USER, ELASTICSEARCH_PASSWORD)
if ELASTICSEARCH_URL_PREFIX:
    es_host_params['url_prefix'] = ELASTICSEARCH_URL_PREFIX
if ELASTICSEARCH_USE_SSL is not None:
    es_host_params['use_ssl'] = ELASTICSEARCH_USE_SSL
if ELASTICSEARCH_VERIFY_CERTS is not None:
    es_host_params['verify_certs'] = ELASTICSEARCH_VERIFY_CERTS

SEARCH_ELASTIC_HOSTS = [es_host_params]
"""Elasticsearch hosts configuration."""

[lnielsen]

Wouldn't you normally have multiple nodes like in this config example?

[lnielsen]

Ups, hit the wrong comment button by mistake :-)

[ppanero]

It really depends, sometimes you just get an endpoint (to the cluster) and how it is balances in the nodes is transparent to you (Invenio). Then it is helpful to just load separately (Maybe is CERN related, but CDS and CERN Search have some sort of the same code. Then for RDM deployment at CERN it would be beneficial. It is a matter of storing just the secret password in OpenShift and not a full dict as string.

Nonetheless, it might be an overkill, WDYT?