Connect eoapi-k8s STAC FastAPI to External PostgreSQL Database

[MathewNWSH]

Description

Hello everyone, I have recently returned to the topic of deploying pgstack and stac-fast API on Kubernetes (my deployment using Docker Compose can no longer handle the storage needs :).

And here is where I have a problem: many thanks for creating the cloud-native PostgreSQL operator, thanks to it eoapi-k8s is really an amazing tool for working with the cloud in every regard, but we have set up our own bare-metal server dedicated to the PostgreSQL database. The server has been configured for HA (master and synchronizing read-only slaves), so I would like to use this database. This means that, I would like to keep it on a different cluster. Can this be done? i.e., is it enough in the YAML to add:

db:
  settings:
    secrets:
      PGUSER: "username"
      POSTGRES_USER: "username"
      PGPASSWORD: "password"
      POSTGRES_PASSWORD: "password"
      PGHOST: "bare metal load balancer floating IP"

Thanks for the response and the developers' work on eoapi.

Labels

• enhancement
• discussion
• help wanted

[ranchodeluxe]

@MathewNWSH: If you are using the old and deprecated db: config and not PGO and assuming your k8s cluster can talk to external networks then, yes, setting the host env vars should work this way. Let me know how that works

[ranchodeluxe]

@MathewNWSH: If you are using the old and deprecated db: config and not PGO and assuming your k8s cluster can talk to external networks then, yes, setting the host env vars should work this way. Let me know how that works

That said PGO should actually support you doing this too (including specifying your own usernames/passwords and hosts) but you'd have to dig through their charts to figure out how to do that. That's probably what I'd recommend doing

[ividito]

Postgres credentials and connection details can be provided as environment variables to each service. An example of how these are populated through db.settings can be found here:

eoapi-k8s/helm-chart/eoapi/templates/db/secrets.yaml

Lines 8 to 17 in e485580

	stringData:
	{{- range $envKey, $envValue := .Values.db.settings.secrets }}
	{{ upper $envKey }}: "{{ $envValue }}"
	{{- /* stac-utils seems to require different environment variable for postgres so handle here via if/else to
	avoid having to pass more arg secrets */ -}}
	{{- if eq $envKey "PGPASSWORD" }}
	POSTGRES_PASS: "{{ $envValue }}"
	{{- end }}
	{{- if eq $envKey "PGDATABASE" }}
	POSTGRES_DBNAME: "{{ $envValue }}"

PGO also provides lots of options to use external DBs, but it generally requires setting up a primary cluster through PGO first AFAIK.

You could also continue to use db.settings.secrets, but this will still create an unneeded DB instance on the cluster.

[pantierra]

Great! Let's keep this issue to add documentation for it.