Google Cloud SQL PostgreSQL¶
This feature is only available in GCP clusters. If you need on-prem databases, head over to navikt/database-iac.
PostgreSQL is a relational database service that is provided by Google Cloud Platform. It is a good choice for storing data that is relational in nature.
The database is provisioned into the teams own project in GCP. Here the team has full access to view logs, create and restore backups and other administrative database tasks.
When you deploy your application with database config, NAIS will ensure the database exists in a Google Cloud SQL instance with the specified Postgres version, and configure the application with means to connect to it.
The Database instance takes a few minutes to be created, so your app will not be able to connect to right away. This only applies to the first time deploy.
Below is an example of the minimal configuration needed. See all configuration options in the nais.yaml reference.
Choosing the right tier for production
By default, the database server is
db-f1-micro which has 1 vCPU, 614 MB RAM and 10GB of SSD storage with no automatic storage increase. Shared CPU machine types (
db-g1-small) are NOT covered by the Cloud SQL SLA. Consider changing to the
db-custom-CPU-RAM tier for your production databases. Please also note that exhausting disk and/or CPU with automatic increase disabled is not covered by the SLA.
To connect your application to the database, use information from the environment variables below.
NAIS_DATABASE_MYAPP_MYDB is automatically generated from the instance name
myapp (defaults to application name) and
mydb (from database spec). You can customize these environment variable names by setting
.spec.gcp.sqlInstances.databases.envVarPrefix. For instance, setting this to
DB will give you
DB_USERNAME, etc. Note that changing or adding
envVarPrefix requires you to manually delete the
google-sql-<MYAPP> secret and
SQLUser with the same name as the application, see below.
|database url with credentials||
The application is the only application that can access the database instance. Other applications can not connect. It is not, for instance, possible to have two applications (e.g. producer and consumer) connecting directly to the database.
Note that if you change your application name, database name or envVarPrefix, and then change it later, you have to manually reset database credentials.
Use database flags with caution, they will alter the behaviour of your postgres instance.
Postgres in gcp supports setting database flags to alter the database instance performance and functionality, the flags available are listed here: Google Cloud SQL supported flags. This listing specifies what value types are expected, which ranges are allowed and if a restart is required.
The value is always required to be a string in
Example of setting database flags:
... kind: Application metadata: name: myapp spec: gcp: sqlInstances: - type: POSTGRES_14 databases: - name: mydb flags: - name: autovacuum_max_workers value: "10" #integer in google spec, requires restart - name: autovacuum value: "on" #boolean in google spec - name: autovacuum_analyze_scale_factor value: "2" #float in google spec
Query insights are now enabled by default in GCP. This feature provides query overview and analysis. The data is available in the Google cloud console.
For further reading see Google Cloud SQL Query Insights
Data is available for seven days, increasing this will incur extra cost.
Google will automatically perform upgrades, fix bugs and apply security patches to prevent exploits. Your application should be able to handle occasional downtime as this maintenance is performed. Read more on maintenance windows. NAIS does not configure the maintenance window, but this can be set up in the application spec:
If you wish to be notified about upcoming maintenance, you can opt-in for this on the Communications page in the GCP console.
The database is backed up nightly at 3 AM (GMT+1) by default, but can be overridden in
nais.yaml by setting
By default, seven backups will be kept. More info about Cloud SQL backups.
The backups can be found in the Google Cloud SQL instance dashboard.
Point-in-time recovery can be enabled by configuring this in the sql instance for your application spec. This feature allows you to recover your database to a specific point in time.
This feature is not enabled by default. When enabled the Postgres instance will be restarted.
Use this feature with automatic storage increase enabled.
In case of catastrophic failure in GCP we are running a daily complete backup of the postgresql databases in GCP to an on-prem location. This backup currently runs at 5 am. This is in addition to the regular backups in GCP.
Metrics for each Postgres can be found in Grafana.
Cloud SQL credentials¶
Cloud SQL uses ConfigConnector/CNRM to create and manage all relevant resources (sqldatabase, sqlinstance, sqluser, credentials) for postgreSQL. When creating an application via your nais.yaml the database in your google project, along with other necessary resources, are created. The creation of the database takes about ten minutes, and the credential settings will be updated after the database is ready for use.
If you delete and recreate your app new credentials will be created and a synchronization is needed. This process can take up to ten minutes. Using the workaround described below you can avoid this synchronization period.
Workaround for password synchronization issues¶
Retrieve the password from the secret google-sql-
Give yourself the role of
roles/cloudsql.admin (which includes the needed permission
Then you can set the new password with the following command.
Log in to the Google Cloud Console and set the password manually for the application user in the sql instance:
Reset database credentials¶
If you have multiple sql users their names will be on the format:
<MYAPP>-<MYDB>-<SQLUSERNAME> instead of
To reset the database credentials for your application (if application name, database name or envVarPrefix has been changed):
Cloud SQL Proxy¶
The application will connect to the database using Cloud SQL Proxy, ensuring that the database communication happens in secure tunnel, authenticated with automatically rotated credentials.
NAIS will add and configure the proxy client container as a sidecar in the pod, making it available on
localhost for the application. The application will then connect to the proxy using standard database protocol just as if it was the actual database.
For more detailed information, check out the Cloud SQL Proxy documentation
Additional user(s) database(s)¶
You can add users to your database by setting database configuration option:
Additional users needs to manually be given access to the database and table.
Either directly or with Flyway or other database migration tools.
Names added must match regex:
^[_a-zA-Z][_a-zA-Z0-9]+$. Secrets is generated and mounted for each user.
.spec.gcp.sqlInstances.databases.envVarPrefix set to
DB and additional username to
_user2 you will get environment variables in format
Details about environment variables is specified her:
If you've deployed your application with an additional users, and then change name or remove the user from configuration, you need to manually delete the
Personal database access¶
Databases should always be accessed using a personal account, and the access should ideally be temporary.
Step 1. Install local binaries
This guide assumes that you have the following installed on your local machine:
Step 2. Allow your user to edit Cloud SQL resources for your project
Ensure that you have authenticated
gcloud by running
To be able to perform the gcloud commands mentioned below you need a role with user edit permissions, e.g.
To grant yourself this role for a given project, run the following command:
<project-id> can be found by running:
Step 3. One-time setup of privileges to SQL IAM users
This is only required once per database instance and should be done before DDL scripts are run in the database in order to ensure the objects have the right permissions.
Once the database instance is created, we need to grant the IAM users access to the
This can either be done by using the default application database user during database creation/migration with scripts (e.g. Flyway), or as a one-time setup by using the default
Step 3a. Set password for
In order to use the
postgres user, you have to set a password first:
Step 3b. Log in to the database with the
Set up the
Log in to the database (you will be prompted for the password you set in the previous step):
If you are using Cloud SQL Auth proxy v1.21.0 or newer you can get the token in the cloud_sql_proxy command so you can run the psql-command without the -W parameter:
Step 3c. Enable privileges for user(s) in database
This can be enabled for all
cloudsqliamusers (all IAM users are assigned the role
cloudsqliamuser) with the following command:
Or for a specific user (the given IAM user must exist in the database):
If your application created the tables before you were able to run these commands, then the owner of the tables is set to the application's user.
Thus, your application must run the following command either through your chosen database migration tool (e.g. Flyway) or manually with the application user's credentials:
Granting temporary personal access¶
Step 1. Create database IAM user
This is required once per user and requires that you have create user permission in IAM in your project, e.g. Cloud SQL Admin.
To grant yourself this role for a given project, run the following command:
Then, to create the database IAM user:
Step 2. Create a temporary IAM binding for 1 hour
Generally, you should try to keep your personal database access time-limited.
The following command grants your user permission to log into the database for 1 hour.
If your system has GNU utilities installed:
Otherwise (e.g. MacOS users):
Step 3. Log in with personal user
Ensure that the
cloudsql-proxy is up and running, if not then:
Then, connect to the database:
Upgrading major version¶
In-place database upgrades through
nais.yaml is currently not supported. If you attempt to change the version this way, you will not get an error message, but your
sqlinstances-resource in the cluster will not update and change its status to
In order to upgrade the database version, you will need to follow the Cloud SQL docs on upgrading PostgreSQL for an instance.
After the upgrade has been done through the Google Cloud Console you need to manually delete the
sqlinstance-resource from the cluster.
First then can you change your
nais.yaml-file and redeploy.
Upgrading requires the instance to become unavailable for a period of time. Be sure to schedule your upgrade when database activity is low.
Deleting the database¶
The database is not automatically removed when deleting your NAIS application. Remove unused databases to avoid incurring unnecessary costs. This is done by setting cascadingDelete in your
When you delete an Cloud SQL instance, you cannot reuse the name of the deleted instance until one week from the deletion date.
Check the events on the Config Connector resources
Check the logs of the Cloud SQL Proxy
Example with all configuration options¶
See full example.
FATAL: password authentication failed for user "<user>"¶
The synchronization of the password to the database may have failed. See workaround for password synchronization issues.
Connect to a cloned database-instance¶
If you have for some reason cloned a database in the console, you need to do some manually changes on the new database to be allowed to connect to it.
First you need to log in to with the old username and password, then run
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO "cloned-user"; to give the new cloned user access to all the old tables. If you have objects outside of tables those also needs to be changed.
Also remember to delete the
google-sql-appname-secret from the cluster, so new secrets are generated for the cloned database.
After this you can update your
nais.yaml-file to use the instance name of the cloned database instead of the old ones.
Remeber to delete the old database when you are finished.