Stateful Services

Kipper manages stateful services (databases, caches) separately from apps. Services use StatefulSets with persistent storage. They survive restarts and keep their data.

Adding a service

kip service add postgres --name mydb
kip service add mysql --name mydb
kip service add mongodb --name mydb
kip service add redis --name cache
kip service add rabbitmq --name queue
kip service add opensearch --name search
kip service add minio --name storage
kip service add mailhog --name mailhog

Supported types: postgres, mysql, mongodb, redis, rabbitmq, opensearch, minio, mailhog

One name is refused: a service called <app>-git, where <app> is an app in the same project and environment that still keeps its git token under the older naming. Both would store their credentials in the same Secret, so Kipper asks you to pick another name for the service rather than let one read the other's. An app whose token is stored under the current naming does not block the name.

Per-environment services

Each environment can have its own database with separate credentials and storage:

kip service add postgres --name db --project blog --environment test --storage 1Gi
kip service add postgres --name db --project blog --environment acc --storage 2Gi
kip service add postgres --name db --project blog --environment prod --storage 10Gi

Each environment has separate database storage and credentials. Size test and acceptance databases for their workloads.

What this creates

Options

Flag	Default	Description
--name	Required	Service name
--project	default	Project name
--environment	—	Target environment (e.g. test, acc, prod)
--storage	5Gi (postgres/mysql/mongodb/opensearch), 1Gi (redis/rabbitmq), 10Gi (minio)	Storage size

Binding to apps and functions

Bind a service to inject its connection details as environment variables. Both apps and functions accept bindings; the same prefix scheme applies.

# To an app
kip service bind db domain-service --project blog --environment test

# To a function
kip function bind domain-sync db --project blog --environment test

A database of its own, when you ask for one

A binding attaches the app to the service's own database, which is what a service backing one app wants. Several apps on one instance is the case that needs more: name a database and Kipper creates it inside the service and points that binding at it.

kip service bind db domain-service --database domain_service_test --project blog --environment test
kip service bind db identity-service --database identity_service_prod --project blog --environment prod

A manifest binding takes the same value as database:. Each app then reads its own DB_NAME and shares everything else: the instance, the storage, and the credentials.

Bindings share the service’s credentials, so an app can also access sibling databases on that instance. Use separate services when apps need separate access boundaries.

PostgreSQL, MySQL, MongoDB and RabbitMQ can be divided this way. RabbitMQ calls it a vhost and takes the name in the same place, where / means the service's own. Redis, OpenSearch and MinIO have nothing to divide, so a binding there always points at the whole service.

Injected environment variables

The binding injects individual connection components with a type-based prefix. The prefix depends on the service type:

Database services (PostgreSQL, MySQL, MongoDB), prefix DB_:

Variable	Example value
DB_HOST	db.blog-test.svc.cluster.local
DB_PORT	5432
DB_USERNAME	kipper
DB_PASSWORD	a1b2c3d4e5f6...
DB_NAME	domain_service_test

MinIO, prefix S3_:

Variable	Example value
S3_ENDPOINT	http://storage.blog-test.svc.cluster.local:9000
S3_ACCESS_KEY	kipper
S3_SECRET_KEY	a1b2c3d4e5f6...

Redis, prefix REDIS_:

Variable	Example value
REDIS_HOST	cache.blog-test.svc.cluster.local
REDIS_PORT	6379

OpenSearch, prefix OPENSEARCH_:

Variable	Example value
OPENSEARCH_HOST	search.blog-test.svc.cluster.local
OPENSEARCH_PORT	9200

Redis and OpenSearch run without authentication, so their bindings carry an address and nothing else. Redis starts with no requirepass and OpenSearch with its security plugin off, which means a connection string holding a password fails rather than being ignored: Redis answers AUTH with an error when no password is set. Write redis://${REDIS_HOST}:${REDIS_PORT} and leave the userinfo out.

Both are reachable by anything running in the same namespace. Treat them as shared infrastructure for that project's environment rather than as a private store, and keep anything sensitive in a database that does authenticate.

RabbitMQ, prefix AMQP_:

Variable	Example value
AMQP_HOST	rabbit.blog-test.svc.cluster.local
AMQP_PORT	5672
AMQP_USERNAME	kipper
AMQP_PASSWORD	a1b2c3d4e5f6...
AMQP_VHOST	orders (the binding's vhost, or / if you took the default)

Like databases for postgres/mysql, a RabbitMQ binding can either share the default vhost / with every other app or create its own. Pass --database <name> to kip service bind to provision a per-binding vhost. Kipper runs rabbitmqctl add_vhost on the running pod and grants the kipper user full access to it. Leave the flag off to share /.

In the console, the bind picker lists every existing vhost on the service (the default / is tagged) and also offers a Create new field. Picking an existing vhost reuses it; creating a new one is what most apps want.

Where binding credentials come from

A binding that takes the service default draws on the service's own <service>-credentials Secret. A binding with its own database or vhost gets a Secret of its own, named for the service and the workload it belongs to: an app called api bound to db gets db-app-api-credentials, and a function called api in the same project gets db-function-api-credentials, so the two never read each other's database.

The controller rebuilds that Secret from the service's shared credentials on every reconcile, overriding only the database or vhost name, so rotating the service password reaches every binding without anyone re-binding. Anything you write into it by hand is overwritten on the next pass; change the service's credentials instead.

Neither Secret is read by your pods directly. The controller folds them into the workload's published environment along with everything else, which is what lets a connection string you composed from ${DB_PASSWORD} stay in step with the password itself.

Rotating a service password restarts the workloads bound to it. Kipper publishes the updated environment and rolls the workloads so they load the new credentials. Check rollout status after rotation.

MailHog (test SMTP server), prefix MAIL_:

Variable	Example value
MAIL_HOST	mailhog.blog-test.svc.cluster.local
MAIL_PORT	1025

MailHog catches outgoing mail and serves a browseable inbox. The binding injects MAIL_HOST and MAIL_PORT and nothing else, since the image has no authentication and no TLS. Your app needs spring.mail.smtp.auth=false (or equivalent for your framework) and a plain SMTP transport to talk to it. Open the inbox at https://mailhog-<namespace>--<cluster>.kipper.run (or mailhog-<namespace>.<your-domain> on a custom domain) (the console's Open UI button on the service detail page links there), gated by your console sign-in.

Your app constructs a connection URL from these components in whatever format your framework needs:

• Node.js / Python / Ruby / Go: postgres://${DB_USERNAME}:${DB_PASSWORD}@${DB_HOST}:${DB_PORT}/${DB_NAME}
• Java / Spring Boot (JDBC): jdbc:postgresql://${DB_HOST}:${DB_PORT}/${DB_NAME}
• S3 endpoint (MinIO): ${S3_ENDPOINT}

Unbinding

Remove a binding from the console (env tab → click X on the service) or the CLI:

kip service unbind db domain-service --project blog --environment test

Deleting a service automatically unbinds it from all apps. The per-app databases are not dropped. They remain in the PostgreSQL instance for manual cleanup if needed.

The app restarts automatically when a binding is added or removed.

Connection details

After creating a service, the connection details are displayed:

  Host:     mydb.default.svc.cluster.local
  Port:     5432
  Username: kipper
  Password: a1b2c3d4e5f6...
  Database: app

Retrieve them later:

kip service info mydb

The hostname (mydb.default.svc.cluster.local) is an internal Kubernetes DNS name. Apps in the same namespace can connect directly; network policies restrict access from other namespaces.

MinIO (S3-compatible object storage)

MinIO provides S3-compatible object storage for file uploads, media, documents, and other binary data.

kip service add minio --name storage --project blog --environment test

  Connection details:
    Endpoint:   http://storage.blog-test.svc.cluster.local:9000
    Access Key: kipper
    Secret Key: a1b2c3d4e5f6...

Bind MinIO to your app to inject credentials automatically:

kip service bind storage api --project blog --environment test

This injects S3_ENDPOINT, S3_ACCESS_KEY, and S3_SECRET_KEY into the app. Use them with any S3-compatible SDK (AWS SDK, MinIO SDK, boto3). See the Storage page for mc CLI examples and SDK code samples.

File explorer

MinIO services include a built-in file explorer in the web console. Navigate to Storage in the sidebar to browse buckets, upload and download files, delete objects, and generate share links (presigned URLs). See the Storage page for full details.

Browser-based database console

Postgres and MySQL services have a built-in client in the web console: SQL editor with schema-aware autocomplete, table browser with inline row editing, visual table and index designer, AI assistant that knows your schema, and per-user query history. No desktop tool needed.

Click the code icon on a Postgres or MySQL service row in the Services list (or open the side panel and click the same icon next to AI Diagnose) to open it. See the Database Console page for the full tour.

For other database types (MongoDB, Redis, OpenSearch, RabbitMQ), use a desktop client through kip tunnel as described below.

Service web interfaces

See Service UIs & Sharing to open web interfaces such as MailHog and RabbitMQ, manage sessions, or share a temporary access link.

Connecting from your machine

Services run inside the cluster and are not exposed to the internet. To connect with a desktop database client (DBeaver, TablePlus, pgAdmin, RedisInsight, or any other tool), use kip tunnel to open a secure connection from your machine to the service:

kip tunnel mydb

  ✔  Tunnel open: localhost:5432 → mydb (postgres)
  Press Ctrl+C to close

Open your database client and connect to localhost:5432 with the credentials from kip service info mydb.

If the default port is already in use on your machine, pick a different one:

kip tunnel mydb --local-port 15432

For services in a specific environment:

kip tunnel db --project blog --environment staging

See Team Access for the full tunnel documentation, including Redis examples and troubleshooting.

Listing services

kip service list

  NAME       TYPE         STATUS         READY      STORAGE
  mydb       postgres     running        1/1        5Gi
  cache      redis        running        1/1        1Gi

Services also appear in the web console under the Services sidebar item, where you can view connection details with a masked URL and copy-to-clipboard.

A service whose container Kubernetes has given up restarting reads crash-looping, with the detail below the table:

  NAME       TYPE         STATUS         READY      STORAGE
  db         postgres     crash-looping  0/1        20Gi

  !   db (CrashLoopBackOff)
      container "postgres" has restarted 996 times and is not staying up
      (last exit code 1). Check its logs. If the cause is storage rather than
      its image or configuration, 'kip service restart db' recreates the pod,
      which is the one thing a container restart cannot do

A crash loop is usually the image or the configuration, and a recreated pod comes back with both, so the listing says to look before it says to restart. Where recreation genuinely is the remedy is a mount that went read-only underneath the pod, and that gets its own alert.

An app bound to it keeps its own status, because the app really is running. Its broken dependency is named under kip app list:

  !   api depends on db, which is crash-looping
      kip service list  shows why, and what to try

Restarting a service

kip service restart <name>

This recreates the service's pod, which is what a container restart cannot do. The volume is reattached rather than recreated, so the data is untouched.

Reach for it when a service is crash-looping on something that belongs to the pod rather than to the container. The clearest case is a volume that remounted read-only: the mount belongs to the pod, so however many times the container dies it comes back to the same read-only filesystem. See Recovering a read-only volume.

kip service restart finds the service by name across the cluster. Name the project when two of them run a service under the same name:

kip service restart db --project shop --environment prod

Checking that a service owns its credentials

Each service has a credentials Secret, and Kipper only injects those values into an app when the Secret belongs to that service. Ownership is the whole check, so a Secret that lost it leaves the service running normally while every app bound to it is refused the credentials it asked for.

That is rare, and worth checking in two situations: after restoring a backup, and before upgrading a cluster.

kip service credentials --project blog --environment test

  SERVICE                  SECRET                             STATE
  db                       db-credentials                     owned
  cache                    cache-credentials                  unowned
  -                        db-app-reports-credentials            unowned binding secret

  Run again with --repair to fix these.

--repair gives an unowned Secret back to its service and removes per-binding Secrets nothing owns, which the apps that need them render again for themselves:

kip service credentials --project blog --environment test --repair

  ✔  cache-credentials now belongs to service cache
  ✔  db-app-reports-credentials removed; nothing reads it and its workload renders a replacement it owns

A per-binding Secret that a running app still reads is left where it is and reported, because removing one an app points at would let its next restart come up with no credentials at all. Run the check again after the app has restarted and it will be cleared then.

Repair never touches the credentials themselves, so a database keeps the password it already has and the apps bound to it carry on with the values they are already using. A Secret that belongs to something else is reported rather than taken.

Deleting a service

Deleting a service removes the service and the workload behind it. The volume it kept its data on stays where it is unless you ask for that to go too:

# The service goes, the volume stays
kip service delete mydb

# The service goes, and the volume with it
kip service delete mydb --delete-data

DANGER

--delete-data is irreversible. The persistent volume and all data are permanently deleted. There is no undo.

What --delete-data removes is the volume Kipper created for the service. One that was renamed, or whose labels were changed by hand, is no longer recognisable as that service's and stays where it is, and the command says when it found nothing to remove. Recognisable means the name a StatefulSet gives a claim, data-<service>-0, carrying that service's app label. When the service itself has already gone that is the only evidence left, so name a service you no longer have carefully.

A volume left behind is what a service of the same name lands on later, and Kipper blocks that service with DataWithoutCredentials: there is data on the volume and no password recorded that opens it. kip service list names any service in that state under the table. Run the delete again with --delete-data once the data is genuinely finished with.

A delete that stops on a name collision, because the workload under that name turned out to belong to something else, keeps the service until the collision is cleared. The service says which object it means. If the leftover object is the one you want to keep, remove the kipper.run/delete-data annotation from the Service with kubectl and the service finishes leaving with its volume where it is.

Some services have no record for Kipper to delete: ones created before Kipper kept those records, and ones whose record has already gone, leaving the volume behind. Deleting either needs --delete-data, and the volume goes with it. Without the flag the command says so and deletes nothing.

Deleting from the web console destroys the data, which is what its confirmation asks you to type the service name for. The service stays in the list as deleting until the volume has actually gone, because the workload has to stop first, and both kip service list and the console say so. A delete that cannot finish, because the workload turned out to belong to something else or a volume will not go, stays there and says which step stopped it. Deleting the Service with kubectl keeps the volume, the same as kip service delete without the flag; annotate it with kipper.run/delete-data=true first to have the volume go too.

Importing and exporting data

kip service import loads a database dump into a running service, and kip service export pulls one out. The dump streams through the Kubernetes API straight into the engine's own restore tool inside the service pod, so nothing is written to the server's disk and no extra port needs to be open. Supported engines: MongoDB, PostgreSQL, and MySQL.

kip service import mongodb --file backup.archive.gz --project acme --environment test

  Importing backup.archive.gz (48.2 MiB) into acme-test/mongodb...

  ... mongorestore progress ...

  ✔  Import complete

Accepted formats per engine:

Engine	Import accepts	Export produces
mongodb	mongodump --archive (plain or gzipped)	gzipped archive (.archive.gz)
postgres	pg_dump -F c custom dumps or plain SQL (plain or gzipped)	custom-format dump (.dump)
mysql	SQL scripts (plain or gzipped)	gzipped SQL (.sql.gz)

The format is detected from the file's content. Postgres and MySQL default to the service's own database; pass --database to target a different one. A MongoDB archive restores the databases it contains; add --database and --source-database together to rename on restore:

kip service import mongodb --file prod-backup.archive.gz --database acme --source-database prod

--drop replaces existing data instead of failing on it. For postgres it needs a custom-format dump (pg_dump -F c); a plain SQL script manages its own DROP statements.

Exports mirror the same shape:

kip service export mydb --file nightly.dump --database app

  Exporting blog-prod/mydb to nightly.dump...

  ✔  Exported nightly.dump (12.4 MiB)
  Restore with: kip service import mydb --file nightly.dump

While an import or export runs, the resource tuner is paused for that service. A saturation-triggered resource change would restart the database mid-restore. The pause is a short lease the CLI keeps renewing for as long as the transfer runs, so if the CLI dies the tuner resumes by itself within 15 minutes.

Copying data between environments

When you copy an environment via the wizard, the new env's databases come up empty. To bring data over, open a service in the new env and switch to the Migrate data tab in the side panel. The tab lists every same-named, same-type service in other namespaces (e.g. the same backend postgres in your test env). Click Copy data here next to the source you want, type the service name to confirm, and the migration starts as a Kubernetes Job in the target namespace.

What it does, in postgres terms:

PGPASSWORD=$SOURCE pg_dump --clean --if-exists ... | PGPASSWORD=$TARGET psql --set ON_ERROR_STOP=1 ...

The --clean --if-exists makes re-runs safe. Every object is dropped before being recreated. The ON_ERROR_STOP setting fails the job loudly on any restore error rather than leaving you with a half-restored database.

A few things worth knowing:

• The target database is wiped. Tables, sequences, views, everything. The wizard's confirm modal asks you to type the service name on purpose.
• Temporary source credentials. The job mounts a temporary copy of the source service's credentials secret in the target namespace. The mirror is owned by the job and gets garbage-collected when the job is cleaned up (one hour after completion).
• No retries. If the dump or restore fails, the job stops there. Re-run it after fixing the underlying issue. Re-runs overwrite cleanly.
• Supported engine: PostgreSQL.

The status panel polls every couple of seconds while a migration is running and tails the last 50 lines of pod logs so you can see pg_dump progress as it happens.

Resource limits

Configure CPU and memory limits for your services from the Resources tab in the service detail panel. Click a service in the web console, switch to the Resources tab, and adjust the CPU and memory requests and limits.

Resource limits control how much CPU and memory the service pod is allowed to consume. Databases under heavy query load or caches handling high throughput may need higher limits than the defaults.

WARNING

Changing resource limits on a service triggers a pod restart. For databases (PostgreSQL, MySQL, MongoDB), this means a brief period of downtime while the pod restarts with the new limits. Plan resource changes during a maintenance window or low-traffic period.

How services differ from apps

	Apps	Services
Kubernetes resource	Deployment	StatefulSet
Storage	Optional volume mounts	PersistentVolumeClaim
Restart	Rolling update	Pod recreated, volume reattached
Delete	Immediate	Volume kept unless --delete-data
Scaling	kip app scale	Single replica
External access	Via Ingress (public URL)	Internal only (cluster DNS)