Techcle Wiki

Tasks

4 min read

Tasks: N8n Platform - Shared Database Migration

Overview

This task list focuses on migrating from the current per-instance database architecture to the new shared database architecture with the n8n-data namespace. This will resolve the current connectivity issue and enable multi-tenancy.

Tasks

1. Create n8n-data Namespace and Deploy Shared PostgreSQL

Create the dedicated data namespace and deploy a shared PostgreSQL server.

  • 1.1 Create n8n-data namespace manifest
  • 1.2 Create PostgreSQL admin credentials secret
  • 1.3 Create PostgreSQL deployment manifest
    • 1.3.1 Configure PostgreSQL with admin user
    • 1.3.2 Configure resource limits (512Mi-2Gi memory, 500m-2000m CPU)
    • 1.3.3 Configure persistent volume claim (20Gi)
  • 1.4 Create PostgreSQL service manifest (ClusterIP)
  • 1.5 Add manifests to k8s-lab kustomization
  • 1.6 Apply and verify PostgreSQL is running

2. Implement Database Manager in Operator

Add database provisioning logic to the operator to create per-instance databases in the shared PostgreSQL server.

  • 2.1 Create DatabaseManager struct in operator
    • 2.1.1 Add connection to PostgreSQL admin user
    • 2.1.2 Implement ProvisionDatabase method
    • 2.1.3 Implement CleanupDatabase method
    • 2.1.4 Implement database existence check
  • 2.2 Update N8nReconciler to use DatabaseManager
    • 2.2.1 Call ProvisionDatabase during instance creation
    • 2.2.2 Generate unique database name (n8n_)
    • 2.2.3 Generate unique username (n8n__user)
    • 2.2.4 Generate secure random password
    • 2.2.5 Store credentials in instance namespace secret
  • 2.3 Update N8nReconciler deletion logic
    • 2.3.1 Call CleanupDatabase during instance deletion
    • 2.3.2 Handle cleanup failures gracefully
  • 2.4 Add unit tests for DatabaseManager
  • 2.5 Add integration tests for database provisioning

3. Update Operator RBAC for Cross-Namespace Access

Ensure the operator has permissions to access the n8n-data namespace.

  • 3.1 Update ClusterRole to include n8n-data namespace access
  • 3.2 Verify operator can connect to PostgreSQL service
  • 3.3 Test RBAC with actual deployment

4. Update N8n Instance Configuration

Update the N8n CR to reference the shared database with proper namespace.

  • 4.1 Update n8n-instance.yaml
    • 4.1.1 Change database host to reference n8n-data namespace
    • 4.1.2 Update database name to n8n_default
    • 4.1.3 Update username to n8n_default_user
    • 4.1.4 Reference new credentials secret
  • 4.2 Remove old per-instance PostgreSQL deployment
  • 4.3 Remove old per-instance PostgreSQL service
  • 4.4 Remove old per-instance PostgreSQL PVC (after data migration if needed)

5. Verify FQDN Construction Logic

Ensure the existing FQDN construction code works correctly with the new architecture.

  • 5.1 Review buildConnectionString function in database.go
  • 5.2 Add unit tests for FQDN construction
    • 5.2.1 Test short name → FQDN conversion
    • 5.2.2 Test FQDN passthrough (already contains dot)
    • 5.2.3 Test with different namespaces
  • 5.3 Add logging to show constructed connection string (without password)

6. Build and Deploy Updated Operator

Build a new operator image with the database manager changes and deploy it.

  • 6.1 Build multi-arch operator image
  • 6.2 Push to container registry with new tag
  • 6.3 Update kustomization.yaml with new image tag
  • 6.4 Apply operator deployment
  • 6.5 Verify operator starts successfully
  • 6.6 Check operator logs for errors

7. Test End-to-End Flow

Verify the complete flow from N8n CR creation to working instance.

  • 7.1 Delete existing n8n instance (if any)
  • 7.2 Create new N8n CR
  • 7.3 Verify operator provisions database in n8n-data
    • 7.3.1 Check database exists: SELECT datname FROM pg_database WHERE datname='n8n_default'
    • 7.3.2 Check user exists: SELECT usename FROM pg_user WHERE usename='n8n_default_user'
    • 7.3.3 Check permissions: SELECT * FROM pg_database WHERE datname='n8n_default'
  • 7.4 Verify n8n deployment starts successfully
  • 7.5 Verify n8n connects to database (check logs)
  • 7.6 Verify admin user is provisioned
  • 7.7 Access n8n UI and verify functionality
  • 7.8 Create a test workflow
  • 7.9 Delete N8n CR
  • 7.10 Verify database is cleaned up

8. Update Documentation

Document the new architecture and migration process.

  • 8.1 Update k8s-lab README with new architecture
  • 8.2 Document database connection troubleshooting
  • 8.3 Add migration guide for existing instances
  • 8.4 Document multi-tenancy setup (creating additional instances)

Success Criteria

  • PostgreSQL running in n8n-data namespace
  • Operator can provision databases in shared PostgreSQL
  • N8n instance connects successfully using FQDN
  • Admin user provisioning works
  • Database cleanup works on instance deletion
  • No connectivity errors in operator logs
  • N8n UI accessible and functional

Notes

  • The FQDN construction code already exists in database.go - we just need to ensure it’s being used
  • Current error suggests the operator is trying to connect before the FQDN logic runs
  • Database manager will need PostgreSQL admin credentials stored in operator namespace
  • Consider data migration if existing instances have data to preserve

Graph View