Database Integrations

Overview

KubeArchive requires a database to store resources. This document lists the databases supported, how to set up the database schema and how to add new database integrations.

PostgreSQL

Currently, PostgreSQL is the only fully supported database engine. The KubeArchive integration test suite runs with the PostgreSQL version detailed under the postgresql database integration directory.

The PostgreSQL implementation is available here.

The PostgreSQL schema is available here.

MariaDB

This integration is still in progress.

The MariaDB implementation is available here.

Configuration and Customization

Schema

KubeArchive offers a setup.sql file and migration scripts with the instructions to set up the schema as part of the installation process.

The provided schema includes indexes.

KubeArchive users should review and customize the indexes if needed based on the expected queries and the amount and distribution of the expected archived data.

The schema includes the creation of the database kubearchive owned by the kubearchive user. The schema expects the existence of a kubearchive database user fow owning this database.

An account with admin privileges should run the setup.sql script.

psql -U admin \ (1)
-h database.example.com \ (2)
-f setup.sql

1	The database admin account used to apply the schema. It can be different from the KubeArchive DB user.
2	The database URL

Database Credentials

The kubearchive-database-credentials Secret stores the information to connect KubeArchive with the Database.

The Secret ships with empty values and must be populated before KubeArchive can connect to the database:

kind: Secret
type: Opaque
metadata:
  name: kubearchive-database-credentials
  namespace: kubearchive
stringData:
  DATABASE_KIND: ""
  DATABASE_PORT: ""
  DATABASE_URL: ""
  DATABASE_DB: ""
  DATABASE_USER: ""
  DATABASE_PASSWORD: ""

Populate the secret with the specific values of your database and restart the pods accordingly to pick the new values. The command for setting the most common values, URL and password, is:

kubectl patch secret -n kubearchive kubearchive-database-credentials \
--patch='{"stringData": {
"DATABASE_URL": "database.example.com", (1)
"DATABASE_PASSWORD": "password" (2)
}}'

1	The database URL
2	The database password

If you change the DATABASE_USER or the DATABASE_DB then you should accordingly update the schema provided within the KubeArchive release.

Remember to restart kubearchive sink and api as both access the database:

kubectl rollout -n kubearchive restart deployment kubearchive-sink kubearchive-api-server

Password Rotation

To rotate the password of the kubearchive database user, follow these steps:

Change the password on the database side. A database administrator should update the kubearchive user password. For PostgreSQL:
```
ALTER USER kubearchive WITH PASSWORD 'new-password'; (1)
```
1 Replace with the new password.

Update the kubearchive-database-credentials Secret in the kubearchive namespace:

kubectl patch secret -n kubearchive kubearchive-database-credentials \
--patch='{"stringData": {
"DATABASE_PASSWORD": "new-password" (1)
}}'

1	Replace with the new password.

Rollout restart the KubeArchive deployments that access the database:

kubectl rollout -n kubearchive restart deployment kubearchive-sink kubearchive-api-server

Verify that the deployments are running correctly:

kubectl rollout -n kubearchive status deployment kubearchive-sink kubearchive-api-server

Both the database password and the Kubernetes Secret must be updated to match. If they are out of sync, the KubeArchive components will fail to connect to the database.

Adding a New Database engine

To add a new database integration, follow the instructions:

Create a new file under pkg/database named after the new database engine. If it’s a SQL engine, it should be under pkg/database/sql.
Include a new entry on the RegisteredDatabases map with the instance of the new Database implementation.

The `Database` implementation

This section includes the development guidelines expected by the KubeArchive maintainers.

If the engine is a SQL one, Database implementation should be a struct with sqlDatabaseImpl embedded. sqlDatabaseImpl is a partial implementation, so make sure the new engine implementation fully implements all methods required by the Database interface.

Make sure to implement the interfaces that are part of it:

All the interfaces work with objects from the sqlbuilder Go library. Check out the docs to see how to work with them.

Some interfaces, like DBDeleter, have a default implementation.

Other interfaces, like DBFilter, have a partial implementation.

Those implementations may have the functionality that you need. Check them before implementing your own.

If the database interaction logic changes from the implementation in the Database struct, override the implementation of the affected functions. For example, a database that does not support upsert queries needs to implement ResourceInserter as a series of SELECT and an INSERT statements.

Take a look at the current database integrations and feel free to contribute to our code adding new database integrations!