Skip to content

Postgres

You can provision and configure Postgres through nais.yaml.

Warning

If you change the postgreSQL version your data will be lost, as a new database will be created. In other words to upgrade the database version you will need to do a migration as described here: Upgrade GCP postgreSQL

When you deploy your application with database config, NAIS will ensure the database exists in a Google Cloud SQL instance with the specified Postgres version, and configure the application with means to connect to it.

Info

This feature is only available in GCP clusters. If you need on-prem databases, head over to navikt/database-iac.

Below is an example of the minimal configuration needed. See all configuration options in the nais.yaml reference.

...
kind: Application
metadata:
  name: myapp
spec:
  gcp:
    sqlInstances:
      - type: POSTGRES_12
        databases:
          - name: mydb

Configuration

To connect your application to the database, use information from the environment variables below.

The prefix NAIS_DATABASE_MYAPP_MYDB is automatically generated from the instance name myapp (defaults to application name) and mydb (from database spec). You can customize these environment variable names by setting .spec.gcp.sqlInstances[].databases[].envVarPrefix. For instance, setting this to DB will give you DB_HOST, DB_USERNAME, etc.

key environment variable default
hostname NAIS_DATABASE_MYAPP_MYDB_HOST 127.0.0.1
port NAIS_DATABASE_MYAPP_MYDB_PORT 5432
database name NAIS_DATABASE_MYAPP_MYDB_DATABASE .spec.gcp.sqlInstances[].databases[].name
database user NAIS_DATABASE_MYAPP_MYDB_USERNAME .spec.gcp.sqlInstances[].name
database password NAIS_DATABASE_MYAPP_MYDB_PASSWORD (randomly generated)
database url with credentials NAIS_DATABASE_MYAPP_MYDB_URL postgres://username:password@127.0.0.1:5432/mydb

Cloud SQL Proxy

The application will connect to the database using Cloud SQL Proxy, ensuring that the database communication happens in secure tunnel, authenticated with automatically rotated credentials.

NAIS will add and configure the proxy client container as a sidecar in the pod, making it available on localhost for the application. The application will then connect to the proxy using standard database protocol just as if it was the actual database.

sqlproxy

For more detailed information, check out the Cloud SQL Proxy documentation

Sizing your database

By default, the database server has 1 vCPU, 614 MB RAM and 10GB of SSD storage with no automatic storage increase. If you need to change the defaults you can do this in nais.yaml.

Administration

The database is provisioned into the teams own project in GCP. Here the team has full access to view logs, create and restore backups and other administrative database tasks.

Automated backup

The database is backed up nightly at 3 AM (GMT+1) by default, but can be overridden in nais.yaml by setting spec.gcp.sqlInstances[].autoBackupTime.

Default 7 backups will be kept. More info here.

The backups can be found in the Google Cloud SQL instance dashboard.

Personal database access

Databases should always be accessed using a personal account, and the access should ideally be temporary.

Prerequisites

Install local binaries
Grant privileges to sql IAM users

Info

This is only required once per database instance

Once the database instance is created, we need to grant the IAM users access to the "public" schema. This can either be done by using the default application database user during database creation/migration with scripts (e.g. flyway), or as a one-time setup by using the default postgres user:

In order to use the postgres user, you have to set a password first:

gcloud sql users set-password postgres --instance=<INSTANCE_NAME> --prompt-for-password --project <PROJECT_ID>

Then set up the cloudsql proxy and log in to the database (you will be prompted for the password you just set):

CONNECTION_NAME=$(gcloud sql instances describe <INSTANCE_NAME> --format="get(connectionName)" --project <PROJECT_ID>);
cloud_sql_proxy -instances=${CONNECTION_NAME}=tcp:5432
psql -U postgres -h localhost <DATABASE_NAME> -W

This can be enabled for all cloudsqliamusers with the following command: (all IAM users are assigned the role cloudsqliamuser)

alter default privileges in schema public grant all on tables to cloudsqliamuser;

Or for a specific user: (the IAM user must exist in the database):

alter default privileges in schema public grant all on tables to 'user@nav.no';

Granting temporary personal access

Create database IAM user

Info

This is required once per user

gcloud beta sql users create <FIRSTNAME>.<LASTNAME>@nav.no --instance=<INSTANCE_NAME> --type=cloud_iam_user --project <PROJECT_ID>
Create a temporary IAM binding for 1 hour:
gcloud projects add-iam-policy-binding <PROJECT_ID> --member=user:<FIRSTNAME>.<LASTNAME>@nav.no --role=roles/cloudsql.instanceUser --condition="expression=request.time < timestamp('$(date -v '+1H' -u +'%Y-%m-%dT%H:%M:%SZ')'),title=temp_access"

Log in with personal user:

export PGPASSWORD=$(gcloud auth print-access-token)
psql -U <FIRSTNAME>.<LASTNAME>@nav.no -h localhost <DATABASE_NAME> 

Deleting the database

The database is not automatically removed when deleting your NAIS application. Remove unused databases to avoid incurring unnecessary costs. This is done by setting cascadingDelete in your nais.yaml-specification.

Danger

When you delete an Cloud SQL instance, you cannot reuse the name of the deleted instance until one week from the deletion date.

Maintenance window

Google will automatically perform upgrades, fix bugs and apply security patches to prevent exploits. Your application should be able to handle occational downtime as this maintenance is performed. Read more on maintenance windows here. NAIS will automatically configure the maintenance window to 4 AM (GMT+1), but can be overridden in nais.yaml.

If you wish to be notified about upcoming maintenance, you can opt-in for this on the Communications page(https://console.cloud.google.com/user-preferences/communication) in the GCP console.

Debugging

Check the events on the Config Connector resources

$ kubectl describe sqlinstance <myapp>
$ kubectl describe sqldatabase <mydb>
$ kubectl describe sqluser <myapp>

Check the logs of the Cloud SQL Proxy

$ kubectl logs <pod> -c cloudsql-proxy

Example with all configuration options

See full example.