Techcle Wiki

SYSTEM CONTEXT

12 min read

HIP Platform - System Context

Project: HIP (Enterprise Integration Platform)
Version: 1.0
Last Updated: 2026-02-10
Status: In Progress (Mature Platform with 20+ API Producers)

1. Executive Summary

HIP is an enterprise API integration platform built on AWS EKS that enables both API producers and consumers to operate within a self-service model. The platform provides GitOps-based API creation for producers and a custom developer portal for consumer discovery and management. Operating at enterprise scale with support for Critical National Infrastructure, HIP manages 10+ API categories across 20+ producer teams with multi-team platform ownership.

Key Characteristics:

  • Architecture: Microservices on AWS EKS with Kong ingress, ALB + WAF ingress control
  • API Creation: GitOps-first (declarative) with UI supplementary
  • Deployment: Single-region, semantic versioning via Git tags
  • Security: Kyverno-managed network policies, egress gateways with backend auth
  • Scale: 85% simple config-only services, growth trajectory toward advanced integrations
  • Teams: 4 platform teams (O11Y + DevEx, Core Infra, Enablement, API Management)

2. Business Context

2.1 Platform Purpose

HIP serves as the internal enterprise API ecosystem for creating, managing, and consuming APIs across the organization. It acts as a central hub for integration, enabling:

  • API producers to expose services and data integrations
  • API consumers to discover, subscribe to, and invoke APIs
  • Platform operators to manage security, compliance, and performance at scale

2.2 Key Stakeholders

  • API Producers (20+ teams): Business units creating APIs for internal consumption
  • API Consumers (multiple teams): Teams discovering and integrating APIs
  • Platform Team:
    • O11Y + DevEx: Observability, developer experience
    • Core Infra: EKS clusters, Kubernetes operators, infrastructure automation
    • Enablement: Documentation, templates, onboarding
    • API Management: API lifecycle, producer support, consumption models

2.3 Regulatory & Compliance Context

  • No formal compliance framework (internal-only platform)
  • Critical National Infrastructure (CNI) running on platform
    • Impacts: Security posture, audit logging, network segmentation
    • Requires: Enhanced monitoring, incident response, compliance reporting

3. Technical Architecture

3.1 Deployment Environment

┌─────────────────────────────────────────────────────────────┐
│                      AWS Account                             │
├─────────────────────────────────────────────────────────────┤
│                                                               │
│  ┌──────────┐      ┌────────────────┐                       │
│  │   WAF    │──────│   AWS ALB      │                       │
│  └──────────┘      └────────────────┘                       │
│                           │                                  │
│   ┌──────────────────────▼──────────────────────────┐       │
│   │    AWS EKS Cluster (Single Region)              │       │
│   ├──────────────────────────────────────────────────┤       │
│   │                                                  │       │
│   │  ┌─────────────────────────────────────────┐   │       │
│   │  │  Kong Node Group (Dedicated)            │   │       │
│   │  │  ├─ Kong Ingress Controller             │   │       │
│   │  │  ├─ Kong Gateway                        │   │       │
│   │  │  └─ Keycloak (API Producer Auth)        │   │       │
│   │  └─────────────────────────────────────────┘   │       │
│   │                     │                           │       │
│   │  ┌──────────────────▼──────────────────────┐   │       │
│   │  │ API Microservices Node Group (Dedicated)│   │       │
│   │  │ ├─ Producer APIs (Simple + Advanced)    │   │       │
│   │  │ ├─ Platform APIs (OAS, Mgmt, MR Worker)│   │       │
│   │  │ └─ Managed Egress Gateways             │   │       │
│   │  └──────────────────────────────────────────┘   │       │
│   │                                                  │       │
│   │  ┌─────────────────────────────────────────┐   │       │
│   │  │  Other Node Groups (Core Infra Managed) │   │       │
│   │  │  ├─ Kyverno & Operators                 │   │       │
│   │  │  ├─ Observability Stack (Example)       │   │       │
│   │  │  └─ Supporting Services                 │   │       │
│   │  └─────────────────────────────────────────┘   │       │
│   │                                                  │       │
│   └──────────────────────────────────────────────────┘       │
│                                                               │
│  ┌────────────────────────────────────────────┐             │
│  │   Backend Systems (External)                │             │
│  │  • REST APIs                                │             │
│  │  • Legacy/SOAP systems                     │             │
│  │  • Multiple integration points             │             │
│  └────────────────────────────────────────────┘             │
│                                                               │
└─────────────────────────────────────────────────────────────┘

Deployment Strategy:

  • Region: Single AWS region
  • Cluster Type: AWS EKS
  • Traffic Flow: WAF/ALB → Kong (Kong Node Group) → API Services (API Microservices Node Group)
  • Node Group Isolation: Separate node groups for Kong and API services provide independent scaling and blast radius containment
  • Stateless Design: Platform does not persist data (pass-through model)

3.2 Core Components

Kong and Keycloak run on a dedicated Kong node group within the cluster, providing isolation from the API microservices node group. This separation enables independent scaling, resilience, and operational flexibility.

Ingress & Gateway Layer

  • AWS WAF: DDoS protection, rate limiting, malicious pattern blocking
  • AWS ALB (Application Load Balancer): Request routing, SSL/TLS termination
  • Kong: Kubernetes-native API gateway running on dedicated Kong node group
    • Routes traffic based on API definitions to API Microservices node group
    • Integrates with Keycloak for API producer authentication
    • Applies request/response transformation policies
    • Enforces rate limiting and authentication at gateway level
  • Keycloak: In-cluster identity provider
    • Manages API producer authentication and identity
    • Integrated with Kong for transparent auth enforcement
    • Centralizes API access control

Microservices Layer

Producer APIs (run on API Microservices node group)

Simple Services (85% - Config-Only)

  • No code deployed; configuration defines behavior
  • Use Kong plugins for transformation
  • Manage via GitOps (git-based configuration)
  • Examples: Request/response mapping, protocol translation, simple routing

Advanced Services (15% - Custom Implementation)

  • Camel-based Java microservices
  • Quarkus-based applications
  • Producer teams own development and deployment
  • Custom business logic and integrations
  • Deployed via GitOps pipelines

Platform APIs (run on API Microservices node group, owned by API Management team)

  • OAS Discovery Service: API specification catalog and OpenAPI document management
  • Platform Management API: Self-service governance, producer onboarding, API lifecycle management
  • MR Worker & Simple API Deployment: ClickOps enablement service for non-GitOps workflows
  • See Section 3.2.2 for detailed documentation

Networking & Security

  • Kyverno: Policy-as-code for Kubernetes network policies
    • Namespace-level isolation boundaries
    • Explicit policies allowing Kong node group to route to API Microservices node group
    • Zero-trust network segmentation
    • Automatic policy generation and enforcement
  • Network Policies: Restrict traffic between namespaces and services
  • Node Group Isolation: Separate Kong and API Microservices node groups provide additional blast radius containment and network boundary enforcement
  • mTLS: Possible service-to-service encryption (clarification needed)

Backend Integration

  • Managed Egress Gateways:
    • Single point of exit for backend system calls
    • Handle authentication to backend systems
    • Support multiple auth mechanisms (OAuth, API keys, certificates, etc.)
    • Centralized credential management
  • Supported Backend Types:
    • REST APIs
    • Legacy/SOAP systems
    • Other enterprise protocols

3.2.1 Node Group Organization

The HIP cluster uses dedicated node groups to provide isolation, independent scaling, and operational flexibility:

Kong Node Group (Dedicated)

  • Kong Ingress Controller: Manages external traffic routing to API services
  • Kong Gateway: Enforces API policies, routing rules, and rate limiting
  • Keycloak: Identity provider for API producer authentication
  • Isolated for independent scaling and blast radius containment
  • Separation from API services ensures ingress stability is independent of backend service health

API Microservices Node Group (Dedicated)

  • Producer APIs: Simple (85% - config-only) and Advanced (15% - Camel/Quarkus) services created by producer teams
  • Platform APIs: Core platform services owned by API Management team (OAS discovery, platform management, MR worker)
  • Managed Egress Gateways: Backend system integration with centralized authentication
  • Isolated from Kong for independent scaling and resource allocation

Other Node Groups (Core Infra Managed)

  • Core platform infrastructure: Kyverno, Kubernetes operators, controllers, and other system components
  • Observability infrastructure: Logging, metrics collection, tracing, and monitoring services
  • Supporting services: Additional platform-level infrastructure managed by Core Infra team

3.2.2 Platform APIs

Beyond APIs created by producer teams, the API Management team provides several critical platform services that run as advanced microservices on the API Microservices node group:

OAS Discovery Service

  • OpenAPI/Swagger specification catalog and management
  • API documentation discovery and search
  • Specification validation and versioning

Platform Management API

  • Self-service API governance and controls
  • Producer team onboarding and offboarding workflows
  • API lifecycle management (creation, versioning, deprecation)
  • Consumption models and quota management
  • API visibility and analytics controls

MR Worker & Simple API Deployment

  • ClickOps enablement service for simplified API deployment
  • Web UI for non-GitOps workflows and configuration management
  • Merge request automation and workflow orchestration
  • Enables teams to create simple APIs without direct Git/CI-CD interaction

3.3 API Creation Workflow (GitOps-First)

Producer Team Workflow:
├─ Create Git Repository (or branch)
├─ Define API Config (YAML/JSON)
│  ├─ API metadata (name, version, category)
│  ├─ Endpoints (REST paths)
│  ├─ Transformation rules (if config-only)
│  └─ Backend integration details
├─ Commit to Feature Branch
├─ Pull Request (Code Review)
├─ Merge to Main (Triggers CI/CD)
├─ Semantic Versioning (Git tags)
└─ Automatic Deployment to EKS
   ├─ Kong config update
   ├─ Simple service deploy (if applicable)
   ├─ Advanced service build (if custom code)
   └─ Health checks & rollout

Supplementary UI:
├─ Browse existing APIs
├─ View producer templates
├─ One-off config management (non-production)
└─ Developer experience enhancements

3.4 API Discovery & Consumption

Developer Portal (Custom-Built)

  • Custom in-house web portal for API discovery
  • API catalog with searchable metadata
  • Documentation and usage examples
  • API subscription/registration flow
  • Interactive API exploration
  • Usage analytics and monitoring

Consumer Authentication

  • Primary Method: API Keys
  • Model: Request-based (API key in header or query parameter)
  • Management: Self-service key generation and rotation
  • Revocation: Real-time or configurable expiration

3.5 Observability

Owned by Separate O11Y Team

  • Logging: Open-source stack (ELK, Loki, or similar)
  • Metrics: Prometheus-based monitoring
  • Tracing: Distributed tracing infrastructure (Jaeger, likely)
  • Dashboards: Grafana or similar visualization
  • Alerting: Alert rules and incident notification

Platform Integration Points:

  • Application logs aggregation
  • Request/response metrics collection
  • Distributed trace correlation
  • API usage analytics
  • Performance metrics and SLOs

4. Service Types & API Categories

4.1 Service Type Distribution

TypePercentageExamplesDeployment
Config-Only85%Data mapping, protocol translation, auth proxyKong + config
Advanced15%Custom logic, workflow engines, calculatorsCamel/Quarkus apps

Growth Trajectory: Platform sees value in increasing advanced service adoption as use cases mature.

4.2 API Categories (10+)

The platform organizes APIs into multiple categories (specific categories not listed by user, but examples might include: Finance APIs, HR APIs, Inventory APIs, etc.). Each category may have:

  • Namespace separation in Kubernetes
  • Dedicated network policies
  • Category-specific authentication models
  • Category-specific monitoring and alerting

5. Security & Multi-Tenancy

5.1 Current State

  • Network Isolation: Namespace-level policies via Kyverno
  • API Key Authentication: Consumer authentication via API keys
  • Backend Auth: Centralized in egress gateways
  • No data persistence: Stateless platform (no shared state concerns)

5.2 Multi-Tenancy Challenges (Key Initiative)

Current Limitations:

  • Namespace-level isolation may not be sufficient for advanced multi-tenancy
  • API key model lacks fine-grained authorization
  • Shared infrastructure between tenants (teams/producers)
  • Network policies based on namespaces only

Roadmap Items:

  • Enhanced security posture and multi-tenancy improvements (Priority Initiative)
  • Fine-grained access control (RBAC/ABAC)
  • Tenant data isolation and quota enforcement
  • Cross-tenant communication policies

5.3 Critical National Infrastructure Considerations

  • Enhanced audit logging
  • Network segmentation
  • Incident response procedures
  • Compliance reporting capabilities
  • Security patching cadence

6. Data Flow Model

6.1 Request Flow (Stateless)

Client Request
    ↓
WAF (AWS)
    ↓
ALB (AWS)
    ↓
Kong Ingress
    ↓
API Service (Simple or Advanced)
    ↓
[Optional Transformation]
    ↓
Managed Egress Gateway
    ↓
Backend System
    ↓
Response (reverse flow)

6.2 Data Persistence

  • Platform-level: None - HIP does not store API payloads
  • Metadata: API definitions, configurations (in Git + Kong)
  • Audit logs: Request/response logs (via O11Y team infrastructure)
  • Credentials: Backend authentication secrets (in secret store, not specified)

7. Operational Concerns & Initiatives

7.1 Current Challenges (Priority Ranking)

  1. Security & Multi-Tenancy (HIGH)

    • Strengthen tenant isolation
    • Enhance authorization models
    • Improve secret management

  2. Observability (HIGH)

    • API-level SLI/SLO tracking
    • Per-consumer usage analytics
    • End-to-end latency optimization
    • Distributed tracing integration

  3. Cost Optimization (HIGH)

    • AWS bill reduction
    • Resource utilization optimization
    • Idle service cleanup
    • Infrastructure sharing efficiency

  4. Developer Experience (MEDIUM)

    • Easier API producer onboarding
    • Template standardization
    • Self-service capabilities expansion
    • Reduced time-to-first-API

  5. Scaling & Performance (MEDIUM)

    • Handle increased API throughput
    • Manage growing producer ecosystem
    • Maintain sub-100ms P99 latency
    • Support new API categories at scale

7.2 Non-Functional Requirements (Key Initiative: Define & Prove NFRs)

To Be Defined:

  • Throughput: X requests/second per API?
  • Latency: Target P50/P95/P99 latencies?
  • Availability: 99.9% / 99.95% / 99.99%?
  • Error Rate: Acceptable error percentage?
  • Security Response Time: MTTR for security incidents?
  • Scalability: Max producers, APIs, consumers?
  • Data Consistency: Eventual vs. strong consistency needs?

8. Team Structure & Ownership

8.1 Platform Teams

TeamResponsibilityKey Focus
O11Y + DevExObservability infrastructure, developer experienceMetrics, logs, traces, portal UX
Core InfraEKS clusters, Kubernetes operators, infrastructureCluster health, node management, upgrades
EnablementDocumentation, templates, best practicesProducer onboarding, knowledge base
API ManagementAPI lifecycle, producer supportGovernance, standards, consumption models

8.2 API Producer Teams

  • ~20+ teams across organization
  • Self-service API creation via GitOps
  • Own deployment and versioning
  • Supported by Enablement team

9. Technology Stack Summary

LayerTechnologyOwnership
CloudAWS (EKS, ALB, WAF)Core Infra
Container OrchestrationKubernetes (EKS)Core Infra
API GatewayKongAPI Management
IngressKong Ingress ControllerAPI Management
Policy EngineKyverno (NetPol)Core Infra
Simple ServicesKong Plugins + ConfigAPI Management
Advanced ServicesCamel (Java), QuarkusProducer Teams
LoggingELK/Loki (TBD)O11Y Team
MetricsPrometheusO11Y Team
TracingJaeger (likely)O11Y Team
Secret Management(TBD - AWS Secrets Manager?)Core Infra
Git/CI-CD(TBD - GitHub Actions? GitLab CI?)Core Infra
Developer PortalCustom-built (tech stack TBD)API Management

10. Key Decisions & Constraints

10.1 Architectural Decisions

  • Single Region: No multi-region failover (cost vs. resilience tradeoff)
  • Kong as Gateway: Standardized, plugin-rich API gateway
  • GitOps-First: Declarative infrastructure as code for reproducibility
  • Namespace-Level NetPol: Simple but potentially limiting for advanced multi-tenancy
  • Stateless Design: Simplifies scaling and disaster recovery
  • Egress Gateway Pattern: Centralized backend auth and credential management

10.2 Operational Constraints

  • Mature Platform: Backward compatibility concerns with 20+ producers
  • Critical Workloads: CNI designation adds compliance and security rigor
  • Multi-Team Ownership: Coordination and communication overhead
  • Growth Pressure: Increasing producer adoption while maintaining stability

11. Integration Points & External Dependencies

11.1 External Systems (Backend)

  • REST APIs (unknown specific systems)
  • Legacy/SOAP systems (unknown specifics)
  • Other enterprise protocols
  • Managed via egress gateways with auth handling

11.2 Internal Platform Dependencies

  • O11Y Infrastructure: Separate team-owned observability stack
  • Core Infra: EKS cluster management, operators
  • Git/CI-CD: Underlying version control and automation (details TBD)
  • Secret Management: Credentials and API key storage (details TBD)

12. Future Roadmap

12.1 Near-Term (Next 6 Months)

  1. Security & Multi-Tenancy: Enhanced isolation and authorization
  2. Cost Optimization: Resource efficiency and bill reduction
  3. Advanced Capabilities: Support more custom transformation types
  4. NFR Definition: Establish and validate non-functional requirements

12.2 Medium-Term (6-12 Months)

  1. Advanced Service Growth: Increase custom Camel/Quarkus adoption
  2. Multi-Region Expansion (if needed): HA/DR capabilities
  3. Enhanced Observability: Per-consumer analytics, SLI/SLO tracking
  4. Developer Portal Evolution: Improved UX and discovery

12.3 Long-Term (12+ Months)

  1. Autonomous API Lifecycle: Reduced manual intervention
  2. AI-Assisted API Design: Code generation, automatic documentation
  3. Ecosystem Expansion: Partner integrations, public API capabilities
  4. Advanced Policy Engine: Attribute-based access control (ABAC)

13. Known Unknowns & Clarifications Needed

13.1 Technology Details

  • Secret management solution (AWS Secrets Manager, Vault, etc.)
  • CI/CD platform and workflow
  • Developer portal technology stack
  • Distributed tracing implementation (Jaeger, DataDog, etc.)
  • Logging backend specifics (ELK, Loki, CloudWatch)

13.2 Operational Details

  • Non-functional requirements (throughput, latency, availability targets)
  • SLA/SLO specifications
  • Incident response procedures
  • Change management process
  • Production support model (on-call rotation, escalation paths)

13.3 Security & Compliance

  • mTLS usage and enforcement
  • Rate limiting policies and algorithms
  • DDoS mitigation specifics
  • Encryption in transit and at rest (beyond TLS)
  • RBAC/ABAC implementation approach

13.4 Business Context

  • Specific API categories (finance, HR, inventory, etc.)
  • Key business metrics (API adoption rate, time-to-value, etc.)
  • Producer and consumer growth targets
  • Revenue/cost allocation models

14. Document Maintenance

Version History:

  • 1.0 (2026-02-10): Initial system context based on stakeholder interview

Next Review: Q2 2026 (after NFR definition completion)

Maintainers: API Management Team, Core Infra Team

Update Triggers:

  • Architectural changes
  • Major initiative completion
  • New technology adoption
  • NFR validation and updates

Graph View