Merging datasource with optional file

[m4rcs]

I'm trying to merge default config files with optional config overrides like in the example. But I can't figure it out.

Can someone give me a hint how to solve it?

Example

Directory structure:

.
├── config
│   ├── common.yml
│   └── ...
├── defaults
│   ├── common.yml
│   ├── database.yml
│   └── ...
├── gomplate.yml
└── templates
    └── config.yml.tpl

./config/common.yml

timezone: "Europe/Berlin"

./defaults/common.yml

timezone: "America/Anchorage"

./defaults/database.yml

host: "localhost"

./gomplate.yml

inputDir: "templates/"
outputMap: |
  {{ .in | strings.ReplaceAll ".tpl" "" }}

datasources:
  common:
    url: "merge:./config/common.yml|./defaults/common.yml"
  database:
    url: "merge:./config/database.yml|./defaults/database.yml"

./templates/config.yml.tpl

common:
  timezone: "{{ (ds "common").timezone }}"
database:
  host: "{{ (ds "database").host }}"

Run gomplate --config ./gomplate.yml

Expected ./config.yml

common:
  timezone: "Europe/Berlin"
database:
  host: "localhost"

Example as Zip: example.zip

[hairyhenderson]

Hi @m4rcs, thanks for posting with all this detail!

I'm trying to merge default config files with optional config overrides like in the example. But I can't figure it out.

Are you referring to your example? Or was there some other example or documentation that implies that it's possible to refer to a missing file? Datasources require valid URLs that refer to an existing resource.

Here's what I got when I ran the command:

$ gomplate --config ./gomplate.yml
08:59:06 ERR  error="template: templates/config.yml.tpl:4:13: executing \"templates/config.yml.tpl\" at <ds \"database\">: error calling ds: Couldn't read datasource 'database': Couldn't read datasource './config/database.yml': Can't stat /Users/hairyhenderson/temp/config/database.yml: stat /Users/hairyhenderson/temp/config/database.yml: no such file or directory"

It works if I create an empty config/database.yml file:

$ touch config/database.yml
$ gomplate --config ./gomplate.yml
$ cat config.yml
common:
  timezone: "Europe/Berlin"
database:
  host: "localhost"

You may be able to work around this by referring to a nonexistent datasource and then optionally defining it in the template through some combination of datasourceReachable and defineDatasource, and then a merge URI like merge:dboverride|./defaults/database.yml. But I'm not sure it'd work and it honestly sounds pretty ugly.

The best approach would be simply to provide an empty override file.

It's possible that I could alter the merge datasource to allow for missing sources, but that would be a significant change in behaviour and would probably require some sort of flag to enable it (maybe a query parameter like ?optional=true). If you really really need that kind of functionality, go ahead and file an issue to request that feature. No guarantees that I'll be able to implement that in an elegant way though.

[m4rcs]

Thanks for the input. I choose another path by using jinja2 and python.

[hairyhenderson]

Ok, thanks for responding @m4rcs! Sorry to hear it didn't work out with gomplate.