Need a way to determine whether a datasource is reachable

[xrayrey]

Hi All,

Currently if the AWS SSM Parameter doesn't exist or can't be fetched, gomplate will just error out and exit with this message:

Error reading aws+smp from AWS using GetParameter with input {
  Name: " /path/to/param",
  WithDecryption: true
}:
ParameterNotFound: Parameter /path/to/param not found.
        status code: 400, request id: 12345

Is there a way of making it so datasourceExists just returns false? That way we can use conditional statements to detect if there was a problem fetching the parameter and get the value from a different data source.

This is what I would like to do:

{{if eq (getenv "ENV_VAR" "") ""}}{{if (datasourceExists "myvar")}}{{(ds "myvar").Value}}{{else}}foobar{{end}}{{else}}{{getenv "ENV_VAR"}}{{end}}

I want to use the value of the environment variable if it exists. If it doesn't exist I would like to use the value of the data source (in this case AWS SSM Parameter). If that doesn't exist either then I would like to use a default value.

Thanks!

[hairyhenderson]

Hi @xrayrey - thanks for logging this issue! This definitely seems like a bug. I'll take a look.

[xrayrey]

Awesome! Thanks!

[hairyhenderson]

This definitely seems like a bug.

6 days later, I'm really not sure why I thought this was a bug 😅... This is actually by design, though I can certainly make things a bit better.

Is there a way of making it so datasourceExists just returns false? That way we can use conditional statements to detect if there was a problem fetching the parameter and get the value from a different data source.

datasourceExists is a bit less powerful than perhaps assumed - see https://gomplate.hairyhenderson.ca/functions/data/#datasourceexists, especially this bit:

Note: this does not verify if the datasource is reachable.

I totally get the value of being able to also make sure the datasource is available, but modifying datasourceExists's behaviour is liable to break existing templates, so that functionality belongs in a new function.

[xrayrey]

hahaha no problem. That looks good. Do you think we would be able to do something like this?

{{if and (datasourceExists "myvar") (datasourceReachable "myvar")}}foobar{{end}}

[hairyhenderson]

@xrayrey sure, I think the datasourceExists function will be redundant there, datasourceReachable would return false if the datasource were undefined.

A caveat would be that datasourceReachable would attempt to read the value first, so there may be cases (especially with more dynamic datasources) where you get a different result the second time around. We may be able to help the performance impact with caching, eventually, but I don't see that as an initial goal.

[xrayrey]

You are right, datasourceReachable would be enough. I'm not concerned about the fact that it has to read twice. I'm only using parameter store where values don't change that often. I'm not sure if it matters that much with other data sources.

Thanks!