Postgres Configuration
Postgres handles identity management and symbol database storage. It stores user authentication data and debugging symbol metadata.
Configuration Modes​
- Use Existing
- AWS Aurora
- GCP Cloud SQL
- Create Mode
Connect to existing Postgres​
This mode connects to your existing Postgres database, AWS Aurora PostgreSQL or GCP Cloud SQL.
- Standard Postgres
- AWS Aurora with IAM
- GCP Cloud SQL
Standard Postgres Configuration​
Standard Postgres Configuration
postgres:
mode: "use_existing"
use_existing:
host: "" # host:port
user: ""
password: ""
database: "zymtrace" # Database name
secure: false # Enable TLS/secure connection
autoCreateDBs: false # When true, zymtrace migration will automatically create the required databases.
# NOTE: For autoCreateDBs to work, the database user must have CREATEDB permission.
# Grant with: ALTER USER "your-user" CREATEDB;
Setting up the Postgres use_existing mode
Postgres setups can vary based on security needs like data classification and role usage. The example below is a simplified guideline.
Note that unlike typical applications, which require only INSERT and SELECT permissions, the zymtrace database migration job requires DDL access.
If you're setting up zymtrace for the first time, the most straightforward approach is to create one role with both DDL and DML permissions.
Database Setup Options​
You have two options for setting up the required databases:
- Manual Setup: Create databases and users manually using SQL commands
- Automatic Setup: Enable
autoCreateDBsto let zymtrace create databases during migration
Option 1: Manual Database Setup​
For manual database setup, use the SQL commands below. This approach gives you full control over user permissions and database ownership.
The commands below assume you're connected to the database using a role with superuser privileges.
CREATE ROLE zystem LOGIN PASSWORD 'metsyz';
CREATE DATABASE zymtrace_identity OWNER zystem;
CREATE DATABASE zymtrace_symdb OWNER zystem;
CREATE DATABASE zymtrace_web OWNER zystem;
\c zymtrace_identity
ALTER SCHEMA public OWNER TO zystem;
\c zymtrace_symdb
ALTER SCHEMA public OWNER TO zystem;
\c zymtrace_web
ALTER SCHEMA public OWNER TO zystem;
Option 2: Automatic Database Setup​
Alternatively, you can enable autoCreateDBs: true in your configuration to let zymtrace automatically create the required databases during migration. This requires the database user to have CREATEDB permission.
When to use automatic setup:
- Development environments where quick setup is preferred
- When you have administrative access to grant broad CREATE permissions
- Testing scenarios where database recreation is acceptable
When to use manual setup:
- Production environments requiring strict permission control
- When following security best practices with minimal required permissions
- Enterprise environments with database administration policies
Security Considerations​
Database Permissions​
zymtrace requires both DDL (Data Definition Language) and DML (Data Manipulation Language) permissions:
- DDL permissions: Required for database schema migrations
- DML permissions: Required for normal application operations
User Role Management​
For production environments, consider creating separate roles for:
- Migration role: Has DDL permissions for schema updates
- Application role: Has minimal DML permissions for runtime operations
Configuration Examples​
- Development Environment
- Production Environment
Development Environment​
For development setups with simplified security:
postgres:
mode: "use_existing"
use_existing:
host: "postgres.dev.company.com:5432"
user: "zymtrace_dev"
password: "dev-password"
database: "zymtrace"
secure: false
autoCreateDBs: true
Production Environment​
For production setups with enhanced security:
postgres:
mode: "use_existing"
use_existing:
host: "postgres.prod.company.com:5432"
user: "zymtrace_prod"
password: "secure-prod-password"
database: "zymtrace"
secure: true
autoCreateDBs: false # Manually create databases for production
AWS Aurora with IAM Authentication​
These instructions should also apply to traditional AWS RDS PostgreSQL instances as it supports IAM authentication.
AWS Aurora IAM Authentication Configuration
postgres:
mode: "use_existing"
use_existing:
host: "" # host:port
user: "" # Database username for IAM authentication
useIAM: "aws" # Enable AWS IAM authentication
awsRegion: null # Override the default region when using IAM (optional)
database: "zymtrace" # Database name prefix
secure: true # Enable TLS/secure connection (required for IAM auth)
autoCreateDBs: false # When true, zymtrace migration will automatically create the required databases.
# NOTE: For autoCreateDBs to work, the database user must have CREATEDB permission.
# Grant with: ALTER USER "your-user" CREATEDB;
Prerequisites for AWS Aurora IAM Authentication:
- Enable IAM database authentication on your Aurora cluster
- Create a database user for IAM authentication
- Configure AWS IAM policies for database access
- Set up AWS credentials or IAM roles for your Kubernetes cluster
Setting up AWS Aurora IAM Authentication​
Step 1: Enable IAM Database Authentication​
Enable IAM database authentication on your Aurora PostgreSQL cluster:
# For Aurora cluster
aws rds modify-db-cluster \
--db-cluster-identifier your-aurora-cluster \
--enable-iam-database-authentication
Or via the AWS Console:
- Navigate to RDS → Databases → Your Aurora Cluster
- Modify → Additional configuration → Database authentication
- Enable "IAM database authentication"
Step 2: Create Database User and Databases​
Connect to your PostgreSQL database using the master user and create the IAM database user:
-- Create IAM database user
CREATE USER zymtrace_user LOGIN;
GRANT rds_iam TO zymtrace_user;
-- Create the three required databases
CREATE DATABASE zymtrace_identity;
CREATE DATABASE zymtrace_symdb;
CREATE DATABASE zymtrace_web;
-- Grant all privileges on databases to the IAM user
GRANT ALL PRIVILEGES ON DATABASE zymtrace_identity TO zymtrace_user;
GRANT ALL PRIVILEGES ON DATABASE zymtrace_symdb TO zymtrace_user;
GRANT ALL PRIVILEGES ON DATABASE zymtrace_web TO zymtrace_user;
Grant schema privileges (connect to each database and run):
-- For zymtrace_identity database
\c zymtrace_identity
GRANT ALL ON SCHEMA public TO zymtrace_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO zymtrace_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO zymtrace_user;
-- For zymtrace_symdb database
\c zymtrace_symdb
GRANT ALL ON SCHEMA public TO zymtrace_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO zymtrace_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO zymtrace_user;
-- For zymtrace_web database
\c zymtrace_web
GRANT ALL ON SCHEMA public TO zymtrace_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO zymtrace_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO zymtrace_user;
The database user needs ALL privileges on the schema for migrations to work properly. The ALTER DEFAULT PRIVILEGES commands ensure that future tables and sequences created by zymtrace will have the correct permissions.
Step 3: Configure IAM Policy​
Create an IAM policy that allows rds-db:connect action:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"rds-db:connect"
],
"Resource": [
"arn:aws:rds-db:REGION:ACCOUNT_ID:dbuser:CLUSTER_RESOURCE_ID/zymtrace_user"
]
}
]
}
Finding your cluster resource ID:
aws rds describe-db-clusters \
--db-cluster-identifier your-aurora-cluster \
--query 'DBClusters[0].DbClusterResourceId' \
--output text
This outputs something like: cluster-ABCDEFGHIJKL01234
Complete example ARN:
arn:aws:rds-db:us-west-2:123456789012:dbuser:cluster-ABCDEFGHIJKL01234/zymtrace_user
Create the policy:
aws iam create-policy \
--policy-name ZymtraceAuroraConnectPolicy \
--policy-document file://aurora-connect-policy.json
Step 4: Create IAM Role with OIDC Trust Policy​
Option A: Using EKS Service Accounts (Recommended)
Create a trust policy for the IAM role. First, get your EKS cluster's OIDC provider:
aws eks describe-cluster \
--name your-cluster-name \
--query "cluster.identity.oidc.issuer" \
--output text
This outputs something like: https://oidc.eks.us-west-2.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE
Create a file named trust-policy.json:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::ACCOUNT_ID:oidc-provider/oidc.eks.REGION.amazonaws.com/id/OIDC_ID"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.REGION.amazonaws.com/id/OIDC_ID:sub": "system:serviceaccount:default:zymtrace-aurora-sa",
"oidc.eks.REGION.amazonaws.com/id/OIDC_ID:aud": "sts.amazonaws.com"
}
}
}
]
}
Replace:
ACCOUNT_IDwith your AWS account IDREGIONwith your AWS region (e.g.,us-west-2)OIDC_IDwith the ID from the OIDC issuer URL (the part after/id/)defaultwith your namespace if different
Create the IAM role and attach the policy:
# Create IAM role with trust policy
aws iam create-role \
--role-name zymtrace-aurora-role \
--assume-role-policy-document file://trust-policy.json
# Attach the Aurora connect policy
aws iam attach-role-policy \
--role-name zymtrace-aurora-role \
--policy-arn arn:aws:iam::ACCOUNT_ID:policy/ZymtraceAuroraConnectPolicy
# Create Kubernetes service account with IAM role annotation
kubectl create serviceaccount zymtrace-aurora-sa
kubectl annotate serviceaccount zymtrace-aurora-sa \
eks.amazonaws.com/role-arn=arn:aws:iam::ACCOUNT_ID:role/zymtrace-aurora-role
Option B: Using EC2 Instance Profile
Attach the IAM policy to your EKS node group's instance profile. Note: This grants database access to all pods on the nodes.
Step 5: Configure zymtrace Helm Values​
Create your Helm values file with the Aurora IAM authentication configuration:
postgres:
mode: "use_existing"
use_existing:
host: "your-cluster.cluster-xxxxx.us-west-2.rds.amazonaws.com" # Aurora cluster endpoint
user: "zymtrace_user" # The IAM database user created in Step 2
password: "" # Must be empty string when using IAM authentication
useIAM: "aws" # Enable AWS IAM authentication
awsRegion: "us-west-2" # Your AWS region
database: "zymtrace" # Database name prefix
secure: true # Required - IAM authentication requires TLS
autoCreateDBs: false # Set to false if databases were created manually
services:
common:
serviceAccountName: "zymtrace-aurora-sa" # The service account with IAM role annotation
- password: Must be
""(empty string, not omitted) when using IAM authentication - secure: Must be
true- IAM authentication requires TLS connection - serviceAccountName: Must match the service account created in Step 4
- autoCreateDBs: Set to
falseif you created databases manually in Step 2
Deploy zymtrace:
helm install zymtrace-backend zymtrace/backend \
-f your-values.yaml \
--namespace default
Verification​
After deploying zymtrace, verify the setup is working correctly:
1. Check service account annotation:
kubectl describe serviceaccount zymtrace-aurora-sa
Should show:
Annotations: eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT_ID:role/zymtrace-aurora-role
2. Check IAM authentication is enabled on Aurora:
aws rds describe-db-clusters \
--db-cluster-identifier your-cluster \
--query 'DBClusters[0].IAMDatabaseAuthenticationEnabled'
Should return: true
3. Check database migration logs:
kubectl logs -l app=db-migration
Should show successful database connection and migrations without password-related errors.
4. Test database connection from a pod:
kubectl run -it --rm postgres-test \
--image=postgres:16 \
--restart=Never \
--serviceaccount=zymtrace-aurora-sa \
-- psql "host=your-aurora-endpoint.rds.amazonaws.com port=5432 dbname=zymtrace_identity user=zymtrace_user sslmode=require"
Security Considerations​
- TLS Connection: IAM authentication requires a secure TLS connection (
secure: true) - Token Expiration: IAM tokens have a 15-minute lifetime and are automatically refreshed
- Least Privilege: Grant only the minimum required database permissions to IAM users
- Network Security: Use VPC security groups to restrict database access
Troubleshooting​
Common Issues:
-
Connection Timeout
- Ensure Aurora security group allows port 5432 from your EKS cluster
- Check VPC security group rules
- Verify Aurora is in same VPC or network is configured correctly
-
IAM Authentication Failed
- Verify the database user exists and has
rds_iamrole:SELECT usename, useconfig FROM pg_user WHERE usename = 'zymtrace_user'; - Check IAM policy has correct cluster resource ID
- Verify trust policy has correct OIDC provider URL
- Verify the database user exists and has
-
"no password supplied" error
- Ensure
password: ""in Helm values (empty string, not omitted) - Verify service account has IAM role annotation
- Check pod is using the correct service account:
kubectl get pod POD_NAME -o jsonpath='{.spec.serviceAccountName}'
- Ensure
-
"permission denied for schema public"
- Run the schema privilege commands from Step 2
- Ensure
ALTER DEFAULT PRIVILEGESwas executed for all three databases - Grant permissions manually:
\c zymtrace_identity
GRANT ALL ON SCHEMA public TO zymtrace_user;
-
SSL/TLS Errors
- Ensure
secure: trueis set in Helm configuration - IAM authentication requires TLS connection
- Check Aurora has SSL/TLS enabled (default for Aurora)
- Ensure
-
Token generation errors
- Verify OIDC provider is configured for your EKS cluster:
aws iam list-open-id-connect-providers - Check trust policy matches service account namespace
- Ensure service account name in trust policy matches actual service account
- Verify OIDC provider is configured for your EKS cluster:
-
Database does not exist errors
- Verify all three databases were created:
zymtrace_identity,zymtrace_symdb,zymtrace_web - Check database names in Aurora:
psql -c "\l" -h your-endpoint -U postgres
- Verify all three databases were created:
-
Migration job fails
- Check if user has sufficient privileges on all schemas
- Verify
ALTER DEFAULT PRIVILEGESwas executed - Check migration job logs:
kubectl logs -l app=db-migration --tail=100
Quick Reference​
Complete setup checklist:
- IAM database authentication enabled on Aurora cluster
- Database user
zymtrace_usercreated withrds_iamrole - Three databases created:
zymtrace_identity,zymtrace_symdb,zymtrace_web - Schema privileges granted with
ALTER DEFAULT PRIVILEGES - IAM policy created with
rds-db:connectpermission - IAM role created with OIDC trust policy
- Kubernetes service account created and annotated with IAM role ARN
- Helm values configured with
password: "",secure: true, and correctserviceAccountName - Security groups allow port 5432 from EKS cluster
Key configuration values:
postgres:
mode: "use_existing"
use_existing:
host: "your-cluster.cluster-xxxxx.REGION.rds.amazonaws.com"
user: "zymtrace_user"
password: "" # Empty for IAM auth
useIAM: "aws"
awsRegion: "REGION"
database: "zymtrace"
secure: true # Required
autoCreateDBs: false
services:
common:
serviceAccountName: "zymtrace-aurora-sa"
GCP Cloud SQL Configuration​
GCP Cloud SQL Configuration
postgres:
mode: "gcp_cloudsql"
gcp_cloudsql:
instance: "" # PROJECT:REGION:INSTANCE format, e.g. zymtrace-cloudsql-psql-1
user: "" # IAM account, e.g zt-db-762@local-bebop-448118-g4.iam (without gserviceaccount.com suffix)
database: "zymtrace" # Database prefix for zymtrace_identity, zymtrace_profiling, and zymtrace_web databases
autoCreateDBs: false # When true, zymtrace migration will automatically create the required databases.
# NOTE: For autoCreateDBs to work, the IAM database user must have CREATEDB permission.
# Grant with: ALTER USER "zt-db-762@local-bebop-448118-g4.iam" CREATEDB;
workloadIdentity:
enabled: true # Enable Workload Identity for authentication
proxy:
image:
repository: gcr.io/cloud-sql-connectors/cloud-sql-proxy
tag: "2.15.0"
# Use nodeSelector if you created a dedicated node pool with cloud-platform scope
nodeSelector:
cloud.google.com/gke-nodepool: cloudsql-pool
resources:
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "500m"
memory: "556Mi"
port: 5432
serviceAccount: "zymtrace-cloudsql-sa" # Kubernetes service account bound to GCP service account via Workload Identity
Prerequisites for GCP Cloud SQL:
- Set up Workload Identity between your Kubernetes service account and GCP service account
- Grant the GCP service account Cloud SQL Client role
- Ensure the IAM database user has appropriate permissions in your Cloud SQL instance
For detailed setup instructions including creating the Cloud SQL instance, configuring Workload Identity, and setting up IAM authentication, refer to the GCP Cloud SQL setup guide.
AWS Aurora PostgreSQL​
This mode provides native support for AWS Aurora PostgreSQL with IAM authentication.
AWS Aurora Configuration
postgres:
mode: "aws_aurora"
aws_aurora:
host: "" # Aurora cluster endpoint
user: "" # Database user for IAM authentication
database: "zymtrace" # Database prefix
region: "" # AWS region
autoCreateDBs: false
serviceAccount: "" # Kubernetes service account with IAM role
Prerequisites:
- AWS EKS cluster with OIDC provider configured
- Aurora PostgreSQL cluster with IAM authentication enabled
- IAM role with RDS connect permissions
- Kubernetes service account annotated with IAM role ARN
For detailed setup instructions including creating the Aurora cluster, configuring IAM authentication, and setting up OIDC integration, refer to the AWS Aurora setup guide
GCP Cloud SQL PostgreSQL​
This mode provides native support for GCP Cloud SQL PostgreSQL with Workload Identity authentication.
GCP Cloud SQL Configuration
postgres:
mode: "gcp_cloudsql"
gcp_cloudsql:
instance: "" # PROJECT:REGION:INSTANCE format
user: "" # IAM account (without gserviceaccount.com suffix)
database: "zymtrace" # Database prefix
autoCreateDBs: false
workloadIdentity:
enabled: true
proxy:
image:
repository: gcr.io/cloud-sql-connectors/cloud-sql-proxy
tag: "2.15.0"
resources:
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "500m"
memory: "556Mi"
port: 5432
serviceAccount: "zymtrace-cloudsql-sa"
For detailed setup instructions, refer to the GCP Cloud SQL setup guide.
Deploy new Postgres instance​
This mode deploys and manages Postgres within your cluster.
Postgres Create Mode Configuration
postgres:
mode: "create"
create:
config:
user: "postgres"
password: "postgres123"
service:
port: 5432
resources:
requests:
cpu: "200m"
memory: "512Mi"
limits:
cpu: "1000m"
memory: "1024Mi"
storage:
type: "persistent"
size: 20Gi
className: ""