0009: VPD Submission Returns Domain API POC - Implementation Plan

Reference

Specification: codev/specs/0009-vaping-duty-domain-api.md
Repository: repos/vaping-duty-poc (to be created)

Overview

This plan implements a POC Domain API for VPD Submission Returns that orchestrates three backend services (excise, customer, tax-platform) with validation gates and parallel fetching where possible.

Deferred to Future Feature: Idempotency handling via X-Idempotency-Key header is explicitly deferred to a future feature. The POC will use stateless Prism mocks that don’t implement idempotency logic. Full idempotency testing will be done when integrating with real backends.

Design Decisions: See spec.md section “Design Decisions” for rationale behind key technology and architecture choices (Prism vs WireMock, Envoy vs NGINX, Quarkus/Camel vs Spring Boot, k6 vs JMeter, etc.). This plan references those decisions where implementation choices are required.

Phase 0: API Specifications

Objective: Define all OAS specs before any implementation.

Tasks

0.1. Create repository structure

repos/vaping-duty-poc/
├── domain-api/specs/
├── mocks/specs/
└── README.md

0.2. Write excise (Excise Duty System) spec (mocks/specs/excise-api.yaml)

GET /excise/vpd/registrations/{vpdApprovalNumber} - Returns customerId, approval status, registration date
GET /excise/vpd/periods/{periodKey} - Returns period definition, state (OPEN/FILED/CLOSED)
POST /excise/vpd/validate-and-calculate - Validates submission, computes totals, returns customerId
Request: vpdApprovalNumber, periodKey, submission payload
Response: valid flag, customerId, calculations, warnings
Error responses: 404 (unknown approval/period), 409 (already filed), 422 (rule failures), 400 (malformed)

0.3. Write customer (Customer Master Data) spec (mocks/specs/customer-api.yaml)

GET /customers/{customerId} - Returns trader name, type, address
Error responses: 404 (not found)

0.4. Write tax-platform (Submissions & Acknowledgements) spec (mocks/specs/tax-platform-api.yaml)

POST /submissions/vpd - Store submission (idempotent by X-Idempotency-Key)
GET /submissions/vpd/{acknowledgementReference} - Retrieve by ack ref
GET /submissions/vpd?vpdApprovalNumber=...&periodKey=... - Retrieve by approval+period
Headers: X-Idempotency-Key
Error responses: 404 (not found), 409 (idempotent conflict), 412 (precondition failed)

0.5. Write VPD Domain API spec (domain-api/specs/vpd-submission-returns-api.yaml)

POST /duty/vpd/submission-returns/v1 - Submit return, get acknowledgement
GET /duty/vpd/submission-returns/v1?acknowledgementReference=... - Retrieve by ack
GET /duty/vpd/submission-returns/v1?vpdApprovalNumber=...&periodKey=... - Retrieve by approval+period
Headers: X-Correlation-Id, X-Idempotency-Key (POST)
Include fields query parameter for sparse fieldsets

0.6. Create Platform OAS Generator (domain/tools/generate_platform_oas.py)

Input: Producer OAS
Processing:
1. Deep copy producer OAS (preserving all $ref to fragments)
2. Inject sparse fieldsets parameter to GET operations
3. Add platform metadata (x-platform-generated, x-platform-features)
Output: Platform OAS (for API portal)

0.7. Generate Platform OAS from producer OAS

Run: python domain/tools/generate_platform_oas.py domain/producer/vpd-submission-returns-api.yaml
Output: domain/platform/vpd-submission-returns-api.yaml
Verify: Platform OAS preserves $ref to fragments and includes sparse fieldsets

0.8. Review specs for consistency (field names, error formats, header handling)

Success Criteria

All OAS specs pass validation (redocly lint or similar)
Backend specs include example responses for all scenarios
Producer domain API spec uses $ref to fragments
Platform domain API spec preserves $ref and includes sparse fieldsets
Platform generator works correctly
Header handling documented consistently

Deliverables

File	Description
`mocks/excise-api.yaml`	Excise Duty System spec
`mocks/customer-api.yaml`	Customer Master Data spec
`mocks/tax-platform-api.yaml`	Tax Platform Submissions spec
`domain/fragments/*.yaml`	Centrally-versioned platform fragments
`domain/producer/vpd-submission-returns-api.yaml`	Producer domain API spec (source of truth)
`domain/platform/vpd-submission-returns-api.yaml`	Platform domain API spec (generated)
`domain/tools/generate_platform_oas.py`	Platform OAS generator

Phase 1: Foundation

Objective: Set up repository, mocks (all JSON initially), and local dev environment.

Note: All mocks return JSON in this phase. Phase 1a converts excise to XML.

Tasks

1.1. Initialise repository with standard structure

1.2. Create root docker-compose.yaml to run entire stack:

excise mock (Prism - stateless)
customer mock (Prism - stateless)
tax-platform mock (Prism - stateless)
Domain API service (placeholder initially)

Note: Prism chosen over WireMock (see spec Design Decision #1) - OpenAPI-native, example-driven, no custom code required.

1.3. Configure mocks (all Prism):

excise: Prism on port 4010
customer: Prism on port 4011
tax-platform: Prism on port 4012

1.4. Define example data in OAS specs:

excise: Examples for VPD123456 (ACTIVE, maps to CUST789) with periods 24A1 (OPEN), 24A2 (FILED)
customer: Examples for CUST789 → “Example Vapes Ltd” (ORG)
tax-platform: Examples with acknowledgement references like ACK-2026-01-27-000123

1.5. Verify mocks respond with example data via Prism

1.6. Setup interactive API explorer for backend mocks:

Tool choice (see spec Design Decision #7):
- Recommended: Swagger UI (mature, reliable “Try It”)
- Alternative: Stoplight Elements (modern UI/UX)
Add chosen tool to docker-compose
Configure to load all backend OAS specs (excise, customer, tax-platform)
Configure to load domain API spec when available (Phase 2+)
Map to port 8090 (or similar)
Enable CORS on Prism mocks for browser-based requests
Verify “Try It” functionality works against Prism mocks
Later: Use to compare domain API behavior vs direct mock calls

1.7. Setup observability stack (LGTM):

Add grafana/otel-lgtm to docker-compose (Loki + Grafana + Tempo + Mimir)
Provides: distributed tracing (Tempo), metrics (Mimir), logs (Loki), visualization (Grafana)
Accessible via Grafana UI (http://localhost:3000)

1.7a. ~~Setup logging proxies for Prism mocks~~ → Moved to Phase 1a

1.8. Install and configure k6:

Document k6 installation (brew/docker)
Create basic test scripts directory structure
Verify k6 can reach services in docker-compose

Success Criteria

Note: Logging proxies and XML content type are addressed in Phase 1a.

Deliverables

File	Description
`docker-compose.yaml`	Root compose file (all components)
`mocks/docker-compose.yaml`	Mocks-only compose (for dev)
`mocks/*.yaml`	OAS specs for all backend mocks (Prism, JSON)
`api-explorer/`	Swagger UI or Stoplight Elements configuration
`observability/`	LGTM stack configuration (Grafana datasources, dashboards)
`tests/load/`	k6 test scripts directory
`docs/TESTING.md`	k6 installation and usage guide
`docs/OBSERVABILITY.md`	Grafana/Loki/Tempo usage guide

Note: Proxy configurations moved to Phase 1a deliverables.

Phase 1a: XML Content Type & Observability Enhancement

Objective: Update excise mock to return XML responses (per spec requirement) and add logging proxies for full observability.

Background: The spec was updated to require the excise backend to return XML (simulating a legacy system), with the domain API transforming XML→JSON. This phase addresses that requirement and completes the observability setup with logging proxies.

Tasks

1a.1. Convert excise mock to return XML instead of JSON:

Update excise-api.yaml to specify application/xml content type for all responses
Define XML schema equivalents for all response types:
- Registration: <registration><vpdApprovalNumber>, <customerId>, <status>, etc.
- Period: <period><periodKey>, <startDate>, <endDate>, <state>, etc.
- ValidationResult: <validationResult><valid>, <customerId>, <calculations>, etc.
Add XML examples in OAS spec (Prism will serve these)
Note: Request bodies remain JSON (domain API sends JSON, excise responds with XML)

1a.2. Define XML schema for excise responses:

<!-- Registration response -->
<registration>
  <vpdApprovalNumber>VPD123456</vpdApprovalNumber>
  <customerId>CUST789</customerId>
  <status>ACTIVE</status>
  <registeredDate>2024-01-15</registeredDate>
</registration>
 
<!-- Period response -->
<period>
  <periodKey>24A1</periodKey>
  <startDate>2024-01-01</startDate>
  <endDate>2024-03-31</endDate>
  <state>OPEN</state>
  <dutyRates>
    <standardRate>2.20</standardRate>
    <reducedRate>1.10</reducedRate>
  </dutyRates>
</period>
 
<!-- Validation result response -->
<validationResult>
  <valid>true</valid>
  <customerId>CUST789</customerId>
  <calculations>
    <totalDutyDue currency="GBP">12345.67</totalDutyDue>
    <vat rate="0.20">2469.13</vat>
    <calculationHash>sha256:abc123...</calculationHash>
  </calculations>
  <warnings>
    <warning code="WARN-VD-001">Spoilt product close to threshold</warning>
  </warnings>
</validationResult>

1a.3. Update Prism configuration for XML:

Verify Prism can serve XML responses from OAS examples
Test with Accept: application/xml header
Ensure error responses also return XML

1a.4. Setup Envoy proxies with header-controlled fault injection (from original 1.7a):

Add Envoy proxy containers in front of each mock (Prism/WireMock)
Configure request/response logging to Loki
Configure trace context propagation to Tempo
Configure fault injection filter in header-controlled mode:
- Envoy respects x-envoy-fault-delay-request header for delays
- Envoy respects x-envoy-fault-abort-request header for errors
- Domain API generates these headers based on its config (Phase 2)
- No static fault config in Envoy - fully controlled by caller

Update docker-compose with proxy services:

excise-proxy:
  image: envoyproxy/envoy:v1.28-latest
  volumes:
    - ./proxies/envoy-excise.yaml:/etc/envoy/envoy.yaml
  ports:
    - "4010:4010"  # External port
  depends_on:
    - excise-mock

Prism/WireMock mocks move to internal-only ports

1a.5. Envoy fault filter configuration (header-controlled mode):

# proxies/envoy-excise.yaml (fault filter section)
- name: envoy.filters.http.fault
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.fault.v3.HTTPFault
    delay:
      header_delay: {}  # Delay controlled by x-envoy-fault-delay-request header
      percentage:
        numerator: 100
        denominator: HUNDRED
    abort:
      header_abort: {}  # Abort controlled by x-envoy-fault-abort-request header
      percentage:
        numerator: 100
        denominator: HUNDRED

Note: Fault values (delays, error rates) are configured in domain API’s application.yaml and passed via headers. See Phase 2 task 2.5.

1a.6. Per-backend fault profiles (reference for domain API config):

Backend	Delay Range	Error Rate	Notes
excise	100-500ms	5%	Legacy XML system, slower
customer	50-300ms	5%	Fast lookup service
tax-platform	100-400ms	5%	Storage operations

1a.7. Update tests for XML responses:

Update smoke tests to verify XML content type from excise
Add XML parsing validation
Verify other mocks (customer, tax-platform) still return JSON
Test that Envoy respects fault headers when passed

1a.8. Update documentation:

Document XML/JSON content type strategy
Update API explorer notes for XML responses
Document logging proxy architecture

Success Criteria

Deliverables

File	Description
`mocks/excise-api.yaml`	Updated with XML content type and examples
`proxies/envoy-excise.yaml`	Envoy config for excise proxy
`proxies/envoy-customer.yaml`	Envoy config for customer proxy
`proxies/envoy-tax-platform.yaml`	Envoy config for tax-platform proxy
`docker-compose.yaml`	Updated with proxy services
`tests/load/smoke-test-mocks.js`	Updated with XML validation
`docs/CONTENT-TYPES.md`	XML/JSON content type documentation

Docker Compose Architecture

flowchart TB
    subgraph compose["docker-compose.yaml"]
        subgraph obs["Observability (LGTM Stack)"]
            grafana["Grafana<br/>UI<br/>:3000"]
            tempo["Tempo<br/>(traces)"]
            loki["Loki<br/>(logs)"]
            mimir["Mimir<br/>(metrics)"]
        end

        subgraph tools["Development Tools"]
            explorer["API Explorer<br/>(Swagger UI)<br/>:8090"]
            k6["k6<br/>(load tests)"]
        end

        subgraph proxies["Logging Proxies (Envoy/NGINX)"]
            p_excise["excise-proxy<br/>:4010"]
            p_customer["customer-proxy<br/>:4011"]
            p_tax["tax-proxy<br/>:4012"]
        end

        subgraph mocks["Backend Mocks (Prism)"]
            excise["excise-mock<br/>(Prism)<br/>internal"]
            customer["customer-mock<br/>(Prism)<br/>internal"]
            tax["tax-mock<br/>(Prism)<br/>internal"]
        end

        subgraph app["Domain API (Phase 2+)"]
            da["domain-api<br/>(Quarkus/Camel)<br/>:8080"]
        end

         Domain API to backends (via proxies)
        da -->|"validate/calculate"| p_excise
        da -->|"trader details"| p_customer
        da -->|"store/retrieve"| p_tax

         Observability flows
        p_excise -.->|"logs"| loki
        p_customer -.->|"logs"| loki
        p_tax -.->|"logs"| loki
        p_excise -.->|"traces"| tempo
        p_customer -.->|"traces"| tempo
        p_tax -.->|"traces"| tempo
        da -.->|"logs/traces/metrics"| tempo
        da -.->|"logs"| loki
        da -.->|"metrics"| mimir
    end

    host["localhost/browser"]
    host -->|":8080"| da
    host -->|":4010-4012"| proxies
    host -->|":8090"| explorer
    host -->|":3000"| grafana

Key Components:

LGTM Stack: All-in-one observability (Loki, Grafana, Tempo, Mimir) via grafana/otel-lgtm image
Logging Proxies: Envoy or NGINX in front of each Prism mock for request logging and tracing
API Explorer: Swagger UI for interactive API testing (mocks + domain API)
k6: Load testing tool (runs outside compose, targets services)
Prism Mocks: Stateless backends (not directly exposed, accessed via proxies)
Domain API: Quarkus/Camel orchestration service (Phase 2+)

Testing the Compose Stack

1. Start full stack:

docker-compose up

2. Verify backend mocks (via logging proxies):

# Excise: Get registration
curl -s http://localhost:4010/excise/vpd/registrations/VPD123456 | jq .
 
# Excise: Validate and calculate
curl -s -X POST http://localhost:4010/excise/vpd/validate-and-calculate \
  -H "Content-Type: application/json" \
  -d @examples/submission-request.json | jq .
 
# Customer: Get trader details
curl -s http://localhost:4011/customers/CUST789 | jq .
 
# Tax-platform: Store submission
curl -s -X POST http://localhost:4012/submissions/vpd \
  -H "Content-Type: application/json" \
  -H "X-Idempotency-Key: test-123" \
  -d @examples/submission.json | jq .

3. Verify observability:

# Open Grafana
open http://localhost:3000
 
# Explore logs in Loki
# - Go to Explore → Select Loki datasource
# - Query: {job="excise-proxy"} |= "POST"
 
# Explore traces in Tempo
# - Go to Explore → Select Tempo datasource
# - Query: Search for traces with service name
 
# Check metrics in Mimir
# - Go to Explore → Select Prometheus datasource

4. Verify API Explorer:

# Open Swagger UI
open http://localhost:8090
 
# Use "Try It" to execute requests against backend mocks
# Later: Compare domain API responses vs direct mock responses

5. Run load tests:

k6 run tests/load/submit-return.js

Phase 2: Camel Domain Service

Objective: Create the Quarkus/Camel application with basic routes and header propagation.

Framework Choice (see spec Design Decision #5): Quarkus + Apache Camel chosen for declarative orchestration DSL and cloud-native design.

Tasks

2.1. Generate Quarkus project with Camel extensions:

mvn io.quarkus:quarkus-maven-plugin:create \
  -DprojectGroupId=uk.gov.hmrc \
  -DprojectArtifactId=vpd-domain-api \
  -Dextensions="camel-quarkus-rest,camel-quarkus-http,camel-quarkus-jackson"

2.2. Implement REST endpoints (pass-through to SAS initially):

POST /duty/vpd/submission-returns/v1 → forwards to SAS
GET /duty/vpd/submission-returns/v1 → forwards to SAS

2.3. Add header propagation:

Extract and forward X-Correlation-Id
Forward X-Idempotency-Key to tax-platform (POST only)

2.4. Add distributed tracing instrumentation:

Add Quarkus OpenTelemetry extension (quarkus-opentelemetry)
Configure OTLP exporter to send traces to Tempo (via LGTM stack)
Configure trace context propagation to backend calls
Emit spans for orchestration steps (validation, calculation, storage)
Structured logging with correlation IDs to Loki

2.5. Implement fault injection passthrough to Envoy:

Read fault configuration from application.yaml:

fault-injection:
  enabled: true
  excise:
    delay: { min: 100, max: 500 }
    error-rate: 5
  customer:
    delay: { min: 50, max: 300 }
    error-rate: 5
  tax-platform:
    delay: { min: 100, max: 400 }
    error-rate: 5

Generate random delay within configured range per backend call
Determine if request should error based on error-rate percentage
Inject Envoy fault headers on outgoing requests:
- x-envoy-fault-delay-request: <ms> - delay to apply
- x-envoy-fault-abort-request: 503 - when error injection triggered
Log injected faults for debugging/analysis
Support disabling via config (fault-injection.enabled: false)

2.6. Add health check endpoint (/health)

2.7. Create Dockerfile for domain-api

2.8. Add to docker-compose.yaml

Success Criteria

Deliverables

File	Description
`domain-api/pom.xml`	Maven project (with OpenTelemetry extension)
`domain-api/src/main/java/.../VpdRoute.java`	Main Camel routes
`domain-api/src/main/java/.../fault/FaultInjector.java`	Fault injection header generator
`domain-api/src/main/resources/application.yaml`	Config including fault injection settings
`domain-api/Dockerfile`	Container build

Phase 3: Orchestration

Objective: Implement sequential orchestration with validation gates and XML→JSON content type transformation.

Note: The excise backend returns XML (legacy system). The domain API must parse XML responses and transform them to JSON for the unified API response.

Tasks

3.1. Implement POST orchestration:

rest("/duty/vpd/submission-returns/v1")
    .post()
    .route()
    // Step 1: Validate with APS
    .to("direct:validateWithAPS")
    .choice()
        .when(header("APS-Error").isNotNull())
            .to("direct:returnAPSError")
        .otherwise()
    // Step 2: Calculate with RCV
            .to("direct:calculateWithRCV")
            .choice()
                .when(simple("${body[valid]} == false"))
                    .to("direct:returnRCVError")
                .otherwise()
    // Step 3: Store with SAS
                    .to("direct:storeWithSAS")
                    .to("direct:returnAcknowledgement");

3.2. Implement XML→JSON transformation for excise responses:

Add XML parsing library (Jackson XML or similar)
Create transformer component to convert XML responses to JSON POJOs
Handle excise-specific XML schema
Validate transformation with unit tests
Log transformation for debugging

3.3. Implement excise backend client:

Call GET /excise/vpd/registrations/{vpdApprovalNumber} (returns XML)
Call GET /excise/vpd/periods/{periodKey} (returns XML)
Call POST /excise/vpd/validate-and-calculate (returns XML)
Set Accept: application/xml header
Transform XML responses to JSON using transformer from 3.2
Short-circuit on 404, 409, 422

3.4. Implement customer backend client:

Call GET /customers/{customerId} (returns JSON)
Set Accept: application/json header
Parse JSON response directly

3.5. Implement tax-platform backend client:

Call POST /submissions/vpd (accepts/returns JSON)
Call GET /submissions/vpd/{acknowledgementReference} (returns JSON)
Call GET /submissions/vpd?vpdApprovalNumber=...&periodKey=... (returns JSON)
Forward X-Idempotency-Key
Extract acknowledgement reference

3.6. Implement GET orchestration:

By acknowledgementReference: tax-platform → excise (XML) → customer
By approval+period: excise validation (XML) → parallel (tax-platform + customer)

3.7. Implement error mapping:

excise 404 → Domain 404
excise 409 → Domain 409
excise 422 → Domain 422 with details
tax-platform 409 → Domain 409 (idempotency conflict)
Preserve error context through XML transformation

3.8. ~~Implement configurable delay injection~~ → Moved to Envoy configuration (Phase 1a)

Fault injection (delays and errors) handled by Envoy proxies, not domain API
See Phase 1a for Envoy fault filter configuration

3.9. Integration tests for all test scenarios from spec:

Test XML transformation: excise XML → domain JSON
Test mixed content types: excise (XML) + customer (JSON) + tax-platform (JSON)
Test all error scenarios with XML error responses

Success Criteria

POST orchestrates excise (XML) → tax-platform (JSON) + customer (JSON) correctly
XML responses from excise successfully transformed to JSON
GET by ack ref works with XML transformation
GET by approval+period validates then retrieves
All error scenarios handled correctly (including XML errors)
Integration tests pass for all scenarios
Domain API handles Envoy-injected delays and errors gracefully
Unit tests verify XML→JSON transformation accuracy

Deliverables

File	Description
`domain-api/src/main/java/.../orchestration/SubmissionOrchestrator.java`	Orchestration logic
`domain-api/src/main/java/.../orchestration/ExciseClient.java`	Excise backend client (XML)
`domain-api/src/main/java/.../orchestration/CustomerClient.java`	Customer backend client (JSON)
`domain-api/src/main/java/.../orchestration/TaxPlatformClient.java`	Tax-platform backend client (JSON)
`domain-api/src/main/java/.../transformation/XmlToJsonTransformer.java`	XML→JSON transformation
`domain-api/src/main/resources/application.yaml`	Application configuration
`domain-api/src/test/java/.../OrchestrationTest.java`	Integration tests
`domain-api/src/test/java/.../XmlTransformationTest.java`	XML transformation unit tests

Phase 4: Sparse Fieldsets

Objective: Implement fields[resource] query parameter filtering.

Tasks

4.1. Parse fields query parameter:

Extract field list for submission resource
Handle missing parameter (return all fields)

4.2. Implement field mask builder:

Create mask from field list
Handle nested fields (e.g., basicInformation.returnType)

4.3. Apply field mask to response:

Filter after orchestration completes
Preserve structure for nested fields

4.4. Add validation:

Return 400 for unknown field names
Include valid field list in error response

4.5. Unit tests for field filtering

4.6. Integration tests:

Filter submission fields
Nested field filtering
Invalid field name handling

Success Criteria

?fields[submission]=acknowledgementReference,totalDutyDue returns only those fields
Nested filtering works
Invalid field names return 400 with helpful error
Missing fields param returns all fields
Tests pass

Deliverables

File	Description
`domain-api/src/main/java/.../fieldfilter/FieldFilter.java`	Field filtering logic
`domain-api/src/test/java/.../FieldFilterTest.java`	Unit tests

Phase 5: Kong Integration

Objective: Integrate domain API with existing Kong instance in k8s cluster (auth, rate limiting already configured).

Note: Kong is already running in the k8s cluster with authentication and rate limiting configured. This phase is about integrating the domain API, not setting up Kong from scratch.

Tasks

5.1. Create Kong route configuration for domain API:

Service definition pointing to domain-api deployment
Route for /duty/vpd/*
Reference existing authentication plugin
Reference existing rate limiting configuration
Ensure correlation ID plugin is enabled

5.2. Document Kong integration points:

Authentication: OAuth2 token validation (already configured)
Rate limiting: Per-consumer limits (already configured)
Correlation ID: Auto-injection if not present
Path routing: /duty/vpd/* → domain-api service

5.3. Create k8s manifests for Kong integration:

Service definition for domain-api
Kong Ingress annotations or KongIngress CRD

5.4. End-to-end tests through Kong (in k8s):

Authenticated request succeeds
Unauthenticated request returns 401
Rate limit behavior (if testable in non-prod)
Correlation ID present in responses
Request tracing through Kong → domain-api → backends

5.5. Update deployment documentation

Success Criteria

Requests through Kong reach domain-api in k8s
Authentication enforced (using existing OAuth2 config)
Rate limiting active (using existing config)
Correlation ID injected if missing
Traces show full request path: Kong → domain-api → backends
E2E tests pass in k8s environment

Deliverables

File	Description
`kustomize/base/kong-route.yaml`	Kong route configuration
`docs/KONG_INTEGRATION.md`	Kong integration guide
`tests/e2e/kong-integration.test.js`	E2E tests (run in k8s)

Phase 6: Kubernetes Deployment

Objective: Integrate domain API and mocks with existing LOB repository and Helm charts, deploy to sandbox cluster.

Note: Domain API will be integrated into the existing Line of Business (LOB) repository alongside other microservices, using existing Helm chart templates.

Tasks

6.1. Integrate with existing LOB repository and Helm charts:

Identify applicable existing chart in LOB repo (standard microservice chart)
Create values file for vpd-domain-api and mocks
Configure deployment, service, configmap, secrets
Follow LOB repository structure and conventions

6.2. Configure sandbox cluster environment values:

Sandbox cluster configuration
Backend service URLs (point to mock services or backends)
Observability endpoints (Grafana, Tempo, Loki in cluster)
Resource limits and requests appropriate for sandbox

6.3. Configure Kong ingress routing:

Create Kong route configuration for /duty/vpd/*
Reference existing authentication plugin
Reference existing rate limiting configuration
Ensure correlation ID plugin enabled

6.4. Deploy and verify in sandbox cluster:

Helm install/upgrade to sandbox cluster
Deploy both domain API and mock backends
Verify all pods running and healthy
Check service discovery and internal networking
Validate health check endpoints
Confirm Kong routing works

6.5. End-to-end testing:

Test full flow: Kong → domain-api → backends
Verify authentication and authorization
Confirm tracing and logging in Grafana
Load test with k6 (adapted for cluster endpoints)

6.6. Document deployment process:

Helm values configuration guide
Deployment commands and rollback procedures
Troubleshooting common issues
Observability dashboards and queries

Success Criteria

Domain API and mocks integrated into LOB repository structure
Helm charts configured for sandbox cluster deployment
All pods deploy successfully and remain healthy
Domain API accessible via Kong ingress in sandbox
Mock backends reachable from domain API (internal service discovery)
Full test scenario works: Kong → domain-api → mocks
Traces visible in Grafana/Tempo for sandbox traffic
Logs aggregated in Loki with correct labels
Deployment documentation complete and tested

Deliverables

File	Description
`helm/values-sandbox.yaml`	Sandbox cluster Helm values
`helm/values-prod.yaml`	Production environment Helm values (template)
`k8s/kong-route.yaml`	Kong route configuration for `/duty/vpd/*`
`docs/DEPLOYMENT.md`	Helm deployment guide for LOB repo integration
`docs/TROUBLESHOOTING.md`	Common issues and solutions
`docs/LOB_INTEGRATION.md`	Guide for integrating with LOB repository

Phase 7a: Camel YAML DSL - Simple Pass-Through

Objective: Validate JBang + Camel YAML DSL setup with a simple pass-through route.

Approach: Use JBang with Camel to run YAML DSL routes directly, without requiring a full Quarkus/Maven project. Routes are defined in integration.yaml files mounted as ConfigMaps.

Tasks

7a.1. Setup JBang + Camel container in docker-compose:

domain-api:
  image: ghcr.io/jbangdev/jbang-action:latest
  command: >
    camel run /routes/integration.yaml
    --dep=org.apache.camel:camel-http
    --dep=org.apache.camel:camel-jackson
  volumes:
    - ./routes:/routes
  ports:
    - "8080:8080"
  environment:
    - CAMEL_HEALTH_ENABLED=true
  depends_on:
    - tax-platform-mock

7a.2. Create simple GET pass-through route (routes/integration.yaml):

- rest:
    path: /duty/vpd/submission-returns/v1
    get:
      - to: direct:getSubmission
 
- route:
    id: getSubmission
    from:
      uri: direct:getSubmission
    steps:
      # Extract query parameter
      - setHeader:
          name: acknowledgementRef
          simple: "${header.CamelHttpQuery}"
 
      # Propagate correlation ID
      - setHeader:
          name: X-Correlation-Id
          simple: "${header.X-Correlation-Id}"
 
      # Call tax-platform backend
      - toD:
          uri: "http://tax-platform-mock:4010/submissions/vpd/${header.acknowledgementRef}?bridgeEndpoint=true"
 
      # Pass through response
      - setHeader:
          name: Content-Type
          constant: application/json

7a.3. Add header propagation:

Forward X-Correlation-Id to backend
Forward X-Idempotency-Key to tax-platform (POST only)
Pass through Envoy fault injection headers (x-envoy-fault-delay-request, x-envoy-fault-abort-request)

Note: Fault injection is controlled via header pass-through rather than Domain API configuration. This simplifies the Domain API (no fault logic in Camel routes) while enabling per-request fault control for targeted testing. See docs/vpd-testing.md for usage.

7a.4. Configure health endpoint:

Camel health check at /health
Ready when routes are started

7a.5. Integration test:

# Test pass-through
curl "http://localhost:8080/duty/vpd/submission-returns/v1?acknowledgementReference=ACK-2026-01-26-000123"
 
# Verify correlation ID propagated
curl -H "X-Correlation-Id: test-123" "http://localhost:8080/duty/vpd/submission-returns/v1?acknowledgementReference=ACK-2026-01-26-000123"

Success Criteria

JBang + Camel container starts successfully
GET request passes through to tax-platform mock
Response returned to client unchanged
X-Correlation-Id header propagated to backend
Health endpoint responds at /health
Fault injection headers (x-envoy-fault-delay-request, x-envoy-fault-abort-request) passed through to Envoy proxy

Deliverables

File	Description
`routes/integration.yaml`	Simple pass-through route
`docker-compose.yaml`	Updated with JBang service
`docs/JBANG_SETUP.md`	JBang + Camel setup guide

Phase 7b: Camel YAML DSL - Full Orchestration

Objective: Implement complete orchestration flows in Camel YAML DSL, including XML transformation, parallel calls, and validation gates.

Note: This phase implements the three flows from the spec’s sequence diagrams.

Tasks

7b.1. Implement GET by acknowledgementReference flow:

Sequential: tax-platform → excise (×2) → customer

- route:
    id: getByAcknowledgement
    from:
      uri: direct:getByAcknowledgement
    steps:
      # 1. Get submission from tax-platform
      - toD:
          uri: "http://tax-platform-proxy:4012/submissions/vpd/${header.ackRef}"
          id: getTaxPlatform
      - unmarshal:
          json: {}
      - setProperty:
          name: submission
          simple: "${body}"
      - setProperty:
          name: vpdApprovalNumber
          simple: "${body[vpdApprovalNumber]}"
      - setProperty:
          name: periodKey
          simple: "${body[periodKey]}"
 
      # 2. Get registration from excise (XML)
      - setHeader:
          name: Accept
          constant: application/xml
      - toD:
          uri: "http://excise-proxy:4010/excise/vpd/registrations/${exchangeProperty.vpdApprovalNumber}"
          id: getRegistration
      - unmarshal:
          jacksonXml: {}
      - setProperty:
          name: customerId
          simple: "${body[customerId]}"
      - setProperty:
          name: registration
          simple: "${body}"
 
      # 3. Get period from excise (XML)
      - toD:
          uri: "http://excise-proxy:4010/excise/vpd/periods/${exchangeProperty.periodKey}"
          id: getPeriod
      - unmarshal:
          jacksonXml: {}
      - setProperty:
          name: period
          simple: "${body}"
 
      # 4. Get customer details (JSON)
      - removeHeader:
          name: Accept
      - toD:
          uri: "http://customer-proxy:4011/customers/${exchangeProperty.customerId}"
          id: getCustomer
      - unmarshal:
          json: {}
      - setProperty:
          name: customer
          simple: "${body}"
 
      # 5. Assemble response
      - to: direct:assembleGetResponse

7b.2. Implement GET by vpdApprovalNumber+periodKey flow:

Sequential validation → Parallel fetch: excise (×2) → parallel(tax-platform, customer)

- route:
    id: getByApprovalPeriod
    from:
      uri: direct:getByApprovalPeriod
    steps:
      # 1. Validate via excise - get registration (XML)
      - setHeader:
          name: Accept
          constant: application/xml
      - toD:
          uri: "http://excise-proxy:4010/excise/vpd/registrations/${header.vpdApprovalNumber}"
      - unmarshal:
          jacksonXml: {}
      - setProperty:
          name: customerId
          simple: "${body[customerId]}"
      - setProperty:
          name: registration
          simple: "${body}"
 
      # 2. Get period from excise (XML)
      - toD:
          uri: "http://excise-proxy:4010/excise/vpd/periods/${header.periodKey}"
      - unmarshal:
          jacksonXml: {}
      - setProperty:
          name: period
          simple: "${body}"
 
      # 3. Parallel fetch: tax-platform + customer
      - removeHeader:
          name: Accept
      - multicast:
          parallelProcessing: true
          strategyRef: aggregateResults
          steps:
            # 3a. Get submission from tax-platform
            - toD:
                uri: "http://tax-platform-proxy:4012/submissions/vpd?vpdApprovalNumber=${header.vpdApprovalNumber}&periodKey=${header.periodKey}"
            # 3b. Get customer details
            - toD:
                uri: "http://customer-proxy:4011/customers/${exchangeProperty.customerId}"
 
      # 4. Assemble response
      - to: direct:assembleGetResponse

7b.3. Implement POST submission flow:

Sequential validation → Parallel store+enrich: excise → parallel(tax-platform, customer)

- route:
    id: postSubmission
    from:
      uri: direct:postSubmission
    steps:
      # 1. Validate and calculate with excise (XML response)
      - setHeader:
          name: Accept
          constant: application/xml
      - setHeader:
          name: Content-Type
          constant: application/json
      - to:
          uri: "http://excise-proxy:4010/excise/vpd/validate-and-calculate"
          id: validateAndCalculate
      - unmarshal:
          jacksonXml: {}
      - setProperty:
          name: validationResult
          simple: "${body}"
      - setProperty:
          name: customerId
          simple: "${body[customerId]}"
 
      # 2. Check validation result
      - choice:
          when:
            - simple: "${body[valid]} == false"
              steps:
                - setHeader:
                    name: CamelHttpResponseCode
                    constant: 422
                - to: direct:returnValidationError
          otherwise:
            steps:
              # 3. Parallel: store submission + get customer
              - multicast:
                  parallelProcessing: true
                  strategyRef: aggregateResults
                  steps:
                    # 3a. Store in tax-platform
                    - setHeader:
                        name: X-Idempotency-Key
                        simple: "${header.X-Idempotency-Key}"
                    - to:
                        uri: "http://tax-platform-proxy:4012/submissions/vpd"
                    # 3b. Get customer details
                    - toD:
                        uri: "http://customer-proxy:4011/customers/${exchangeProperty.customerId}"
 
              # 4. Assemble response
              - setHeader:
                  name: CamelHttpResponseCode
                  constant: 201
              - to: direct:assemblePostResponse

7b.4. Implement response assembly:

- route:
    id: assembleGetResponse
    from:
      uri: direct:assembleGetResponse
    steps:
      - setBody:
          simple: |
            {
              "acknowledgementReference": "${exchangeProperty.submission[acknowledgementReference]}",
              "vpdApprovalNumber": "${exchangeProperty.vpdApprovalNumber}",
              "periodKey": "${exchangeProperty.periodKey}",
              "trader": ${exchangeProperty.customer},
              "submission": ${exchangeProperty.submission}
            }
      - marshal:
          json: {}
      - setHeader:
          name: Content-Type
          constant: application/json

7b.5. Add fault injection header generation:

# Processor to inject fault headers before each backend call
- route:
    id: injectFaultHeaders
    from:
      uri: direct:injectFaultHeaders
    steps:
      - process:
          ref: faultInjector  # Custom processor reads config, generates headers
      - setHeader:
          name: x-envoy-fault-delay-request
          simple: "${exchangeProperty.faultDelay}"
      - choice:
          when:
            - simple: "${exchangeProperty.shouldInjectError} == true"
              steps:
                - setHeader:
                    name: x-envoy-fault-abort-request
                    constant: 503

7b.6. Add error handling:

- onException:
    - exception: org.apache.camel.http.base.HttpOperationFailedException
      handled: true
      steps:
        - choice:
            when:
              - simple: "${exception.statusCode} == 404"
                steps:
                  - setHeader:
                      name: CamelHttpResponseCode
                      constant: 404
                  - setBody:
                      constant: '{"error": "Not found"}'
              - simple: "${exception.statusCode} == 409"
                steps:
                  - setHeader:
                      name: CamelHttpResponseCode
                      constant: 409
                  - setBody:
                      constant: '{"error": "Conflict"}'
            otherwise:
              steps:
                - setHeader:
                    name: CamelHttpResponseCode
                    constant: 500
                - setBody:
                    simple: '{"error": "${exception.message}"}'

7b.7. Integration tests for all flows:

Test POST with validation pass/fail
Test GET by acknowledgement
Test GET by approval+period
Verify parallel calls (check timing)
Verify XML→JSON transformation
Verify error propagation

Success Criteria

All three orchestration flows work correctly
XML responses from excise transformed to JSON
Parallel calls execute concurrently (measurable latency improvement)
Validation failures short-circuit with 422
Backend errors mapped correctly (404, 409, 422)
Fault injection headers generated and passed to Envoy
Response assembly produces correct JSON structure

Deliverables

File	Description
`routes/integration.yaml`	Complete orchestration routes (single file, refactored in 7c)
`docs/CAMEL_YAML_EVALUATION.md`	Findings and recommendation

Note: Route splitting into separate files happens in Phase 7c (Reusability Patterns).

Phase 7c: Reusability Patterns

Objective: Establish reusable components and file organization patterns for scaling to multiple domain API integrations.

Context: We expect many similar domain API integrations. Each integration should be high-level, focused on its specifics, and leverage reusable pieces for common patterns (sparse fieldsets, error handling, header propagation, etc.).

Tasks

7c.1. Spike: Validate Kamelets with JBang (1-2 hours):

Before committing to Kamelets as the reusability pattern, verify they work with our JBang setup.

Test scenario:

# test-kamelet.kamelet.yaml
apiVersion: camel.apache.org/v1
kind: Kamelet
metadata:
  name: test-kamelet
spec:
  definition:
    title: Test Kamelet
    required:
      - message
    properties:
      message:
        type: string
  template:
    from:
      uri: kamelet:source
      steps:
        - log:
            message: "Kamelet received: {{message}}"
        - setBody:
            simple: "Response: {{message}}"

# test-route.yaml
- route:
    from:
      uri: timer:test?repeatCount=1
      steps:
        - setProperty:
            name: testProp
            constant: "hello from property"
        - kamelet:
            name: test-kamelet
            parameters:
              message: "${exchangeProperty.testProp}"
        - log:
            message: "Result: ${body}"

Run:

docker run --rm -v $(pwd):/routes apache/camel-jbang:4.16.0 \
  run /routes/test-route.yaml /routes/test-kamelet.kamelet.yaml \
  --dep=org.apache.camel:camel-kamelet

Verify:

Kamelet loads without errors
Parameters resolve from exchange properties
Required parameter validation works
Can call Kamelet multiple times in same route

Fallback if Kamelets don’t work: Use direct routes with header-based args (documented below as alternative).

7c.2. Target route structure (aspirational - get as close as possible):

The goal is routes that read as pure orchestration flow, with all mechanics abstracted away.

Target: routes/get-by-ack.yaml

# GET /duty/vpd/submission-returns/v1?acknowledgementReference=...
#
# Orchestration:
#   1. Get submission from tax-platform (includes customerId)
#   2. Get trader details from customer service
#   3. Combine into enriched response
#   4. Apply sparse fieldsets if requested
 
- route:
    id: getByAcknowledgement
    from:
      uri: direct:getByAcknowledgement
      steps:
        # 1. Get submission from tax-platform
        - kamelet:
            name: backend-taxplatform-byack
            parameters:
              ackRef: "${exchangeProperty.ackRef}"
              # Outputs: submission, customerId (extracted to properties)
 
        # 2. Get trader details
        - kamelet:
            name: backend-customer
            parameters:
              customerId: "${exchangeProperty.customerId}"
              # Outputs: merged into response.trader
 
        # 3. Apply sparse fieldsets (if requested)
        - kamelet:
            name: sparse-fieldsets
            parameters:
              fields: "${exchangeProperty.fieldsParam}"
 
        # 4. Return response
        - kamelet:
            name: http-response
            parameters:
              correlationId: "${exchangeProperty.correlationId}"

Target: routes/get-by-approval.yaml

# GET /duty/vpd/submission-returns/v1?vpdApprovalNumber=...&periodKey=...
 
- route:
    id: getByApprovalAndPeriod
    from:
      uri: direct:getByApprovalAndPeriod
      steps:
        # 1. Get submission by approval + period
        - kamelet:
            name: backend-taxplatform-byapproval
            parameters:
              approvalNumber: "${exchangeProperty.approvalNumber}"
              periodKey: "${exchangeProperty.periodKey}"
 
        # 2. Get trader details
        - kamelet:
            name: backend-customer
            parameters:
              customerId: "${exchangeProperty.customerId}"
 
        # 3. Apply sparse fieldsets
        - kamelet:
            name: sparse-fieldsets
            parameters:
              fields: "${exchangeProperty.fieldsParam}"
 
        # 4. Return response
        - kamelet:
            name: http-response
            parameters:
              correlationId: "${exchangeProperty.correlationId}"

Target: routes/post-submission.yaml

# POST /duty/vpd/submission-returns/v1
 
- route:
    id: postSubmission
    from:
      uri: direct:postSubmission
      steps:
        # 1. Store submission in tax-platform
        - kamelet:
            name: backend-taxplatform-post
            parameters:
              idempotencyKey: "${exchangeProperty.idempotencyKey}"
              body: "${exchangeProperty.requestBody}"
 
        # 2. Get trader details for response enrichment
        - kamelet:
            name: backend-customer
            parameters:
              customerId: "${exchangeProperty.customerId}"
 
        # 3. Return 201 response
        - kamelet:
            name: http-response
            parameters:
              statusCode: 201
              correlationId: "${exchangeProperty.correlationId}"

Target: routes/rest-config.yaml

# REST endpoint definitions - thin routing layer
 
- restConfiguration:
    component: platform-http
    bindingMode: off
    port: 8080
 
- rest:
    path: /duty/vpd/submission-returns/v1
    get:
      - to: direct:routeGetRequest
    post:
      - to: direct:postSubmission
 
- rest:
    path: /health
    get:
      - to: direct:health
 
# Route incoming GET to correct handler based on query params
- route:
    id: routeGetRequest
    from:
      uri: direct:routeGetRequest
      steps:
        - kamelet:
            name: extract-request-context
            # Extracts: correlationId, ackRef, approvalNumber, periodKey, fieldsParam
 
        - choice:
            when:
              - simple: "${exchangeProperty.ackRef} != null"
                steps:
                  - to: direct:getByAcknowledgement
              - simple: "${exchangeProperty.approvalNumber} != null"
                steps:
                  - to: direct:getByApprovalAndPeriod
            otherwise:
              steps:
                - kamelet:
                    name: http-error
                    parameters:
                      statusCode: 400
                      message: "Either acknowledgementReference OR (vpdApprovalNumber AND periodKey) required"

What this achieves:

Routes are ~20-30 lines, not 200+
Pure orchestration - no HTTP mechanics, header manipulation, JSON parsing
Self-documenting - comments describe flow, Kamelets describe parameters
Consistent pattern - all routes follow same structure
Testable - each Kamelet can be tested independently

Builder task: Get as close to this target as possible. Document any deviations and why they were necessary (e.g., if Kamelets don’t support X, or JBang requires Y).

7c.3. Split routes into separate files per endpoint (implementing target above):

routes/rest-config.yaml - REST configuration and endpoint definitions
routes/get-by-ack.yaml - GET by acknowledgementReference route
routes/get-by-approval.yaml - GET by vpdApprovalNumber+periodKey route
routes/post-submission.yaml - POST submission route
routes/common.yaml - Shared subroutes (health, error responses)

Benefits:

Easier code review (one route per file)
Clear separation of concerns
Routes can be developed/tested independently
Simpler diffs when modifying specific endpoints

7c.4. Implement reusable Kamelets (wrapping Java/Groovy implementations):

Each reusable component uses a different mechanism to provide practical experience with all three options:

A. Sparse Fieldsets → Groovy Script (lib/sparse-fieldsets.groovy):

Best fit: Complex JSON transformation logic with dynamic field access.

import groovy.json.JsonSlurper
import groovy.json.JsonOutput
 
class SparseFieldsets {
 
    static def getNestedValue(obj, String path) {
        def parts = path.split('\\.')
        def current = obj
        for (part in parts) {
            if (current == null || !(current instanceof Map) || !current.containsKey(part)) {
                return null
            }
            current = current[part]
        }
        return current
    }
 
    static void setNestedValue(Map obj, String path, value) {
        def parts = path.split('\\.')
        def current = obj
        for (int i = 0; i < parts.size() - 1; i++) {
            if (!current.containsKey(parts[i])) {
                current[parts[i]] = [:]
            }
            current = current[parts[i]]
        }
        current[parts[-1]] = value
    }
 
    static Map apply(String jsonBody, String fieldsParam) {
        if (!fieldsParam) {
            return [filtered: false, body: jsonBody]
        }
 
        def slurper = new JsonSlurper()
        def response = slurper.parseText(jsonBody)
        def requestedFields = fieldsParam.split(',').collect { it.trim() }
 
        // Validate fields exist
        def invalidFields = requestedFields.findAll { field ->
            getNestedValue(response, field) == null &&
            !(response instanceof Map && response.containsKey(field))
        }
 
        if (invalidFields) {
            return [error: true, invalidFields: invalidFields.join(', ')]
        }
 
        // Build filtered response
        def filtered = [:]
        requestedFields.each { field ->
            def value = field.contains('.') ? getNestedValue(response, field) : response[field]
            if (field.contains('.')) {
                setNestedValue(filtered, field, value)
            } else if (response.containsKey(field)) {
                filtered[field] = value
            }
        }
 
        return [filtered: true, body: JsonOutput.toJson(filtered)]
    }
}

Usage in route:

- setBody:
    groovy: |
      def result = SparseFieldsets.apply(body, exchange.getProperty('fieldsParam', String))
      if (result.error) {
        exchange.setProperty('sparseFieldsetError', true)
        exchange.setProperty('invalidFields', result.invalidFields)
        return body
      }
      return result.body

B. Backend Client → Java Processor (lib/BackendClient.java):

Best fit: Encapsulates the full pattern of calling a backend and merging its result into shared state.

//DEPS com.fasterxml.jackson.core:jackson-databind:2.17.0
//DEPS org.apache.camel:camel-core:4.4.0
//DEPS org.apache.camel:camel-http:4.4.0
 
import org.apache.camel.Exchange;
import org.apache.camel.Processor;
import org.apache.camel.ProducerTemplate;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;
 
/**
 * Calls a backend service and merges the response into the exchange state.
 *
 * Encapsulates:
 * 1. URL construction from template + exchange properties
 * 2. Header propagation (correlation ID, etc.)
 * 3. HTTP call execution
 * 4. Response storage in named property
 * 5. Optional field extraction into additional properties
 * 6. Merging specific fields into a shared response object
 */
public class BackendClient {
    private static final ObjectMapper mapper = new ObjectMapper();
 
    private final String name;           // e.g., "customer", "taxPlatform"
    private final String urlTemplate;    // e.g., "http://customer-proxy:4010/customers/{customerId}"
    private final String responseProperty; // Where to store full response
    private final String[] extractFields;  // Fields to extract into separate properties
    private final MergeConfig mergeConfig; // How to merge into shared response
 
    public BackendClient(String name, String urlTemplate, String responseProperty,
                         String[] extractFields, MergeConfig mergeConfig) {
        this.name = name;
        this.urlTemplate = urlTemplate;
        this.responseProperty = responseProperty;
        this.extractFields = extractFields;
        this.mergeConfig = mergeConfig;
    }
 
    /**
     * Call the backend and merge result into exchange state.
     */
    public void call(Exchange exchange) throws Exception {
        // 1. Build URL from template
        String url = resolveUrl(exchange);
 
        // 2. Prepare headers
        exchange.getIn().removeHeaders("CamelHttp*");
        exchange.getIn().setHeader("CamelHttpMethod", "GET");
        String correlationId = exchange.getProperty("correlationId", String.class);
        if (correlationId != null) {
            exchange.getIn().setHeader("X-Correlation-Id", correlationId);
        }
 
        // 3. Make HTTP call
        ProducerTemplate producer = exchange.getContext().createProducerTemplate();
        String response = producer.requestBody(url + "?bridgeEndpoint=true", null, String.class);
 
        // 4. Store full response
        exchange.setProperty(responseProperty, response);
 
        // 5. Extract specific fields into properties
        if (extractFields != null && extractFields.length > 0) {
            JsonNode json = mapper.readTree(response);
            for (String field : extractFields) {
                JsonNode value = json.path(field);
                if (!value.isMissingNode()) {
                    exchange.setProperty(name + "_" + field, value.asText());
                }
            }
        }
 
        // 6. Merge into shared response if configured
        if (mergeConfig != null) {
            mergeIntoResponse(exchange, response);
        }
    }
 
    private String resolveUrl(Exchange exchange) {
        String url = urlTemplate;
        // Replace {property} placeholders with exchange property values
        for (String prop : exchange.getProperties().keySet()) {
            url = url.replace("{" + prop + "}",
                String.valueOf(exchange.getProperty(prop)));
        }
        return url;
    }
 
    private void mergeIntoResponse(Exchange exchange, String response) throws Exception {
        // Get or create shared response object
        ObjectNode sharedResponse = (ObjectNode) exchange.getProperty("response", JsonNode.class);
        if (sharedResponse == null) {
            sharedResponse = mapper.createObjectNode();
        }
 
        JsonNode backendResponse = mapper.readTree(response);
 
        // Apply merge configuration
        if (mergeConfig.asNested != null) {
            // Merge as nested object (e.g., customer -> trader)
            ObjectNode nested = mapper.createObjectNode();
            for (String field : mergeConfig.fields) {
                JsonNode value = backendResponse.path(field);
                if (!value.isMissingNode()) {
                    nested.set(mergeConfig.fieldMapping.getOrDefault(field, field), value);
                }
            }
            sharedResponse.set(mergeConfig.asNested, nested);
        } else {
            // Merge fields directly into response
            for (String field : mergeConfig.fields) {
                JsonNode value = backendResponse.path(field);
                if (!value.isMissingNode()) {
                    sharedResponse.set(mergeConfig.fieldMapping.getOrDefault(field, field), value);
                }
            }
        }
 
        exchange.setProperty("response", sharedResponse);
    }
 
    // Configuration for how to merge backend response
    public static class MergeConfig {
        String asNested;          // If set, merge as nested object with this name
        String[] fields;          // Which fields to include
        java.util.Map<String, String> fieldMapping; // Rename fields (optional)
 
        public static MergeConfig nested(String name, String... fields) {
            MergeConfig config = new MergeConfig();
            config.asNested = name;
            config.fields = fields;
            config.fieldMapping = new java.util.HashMap<>();
            return config;
        }
 
        public static MergeConfig flat(String... fields) {
            MergeConfig config = new MergeConfig();
            config.fields = fields;
            config.fieldMapping = new java.util.HashMap<>();
            return config;
        }
 
        public MergeConfig rename(String from, String to) {
            fieldMapping.put(from, to);
            return this;
        }
    }
}
 
// Pre-configured clients for each backend
class Backends {
    public static final BackendClient CUSTOMER = new BackendClient(
        "customer",
        "http://customer-proxy:4010/customers/{customerId}",
        "customerResponse",
        new String[]{"customerId"},
        BackendClient.MergeConfig.nested("trader", "name", "type", "registeredAddress")
            .rename("registeredAddress", "address")
    );
 
    public static final BackendClient TAX_PLATFORM_BY_ACK = new BackendClient(
        "taxPlatform",
        "http://tax-platform-proxy:4010/submissions/vpd/{ackRef}",
        "taxPlatformResponse",
        new String[]{"customerId", "vpdApprovalNumber", "periodKey"},
        BackendClient.MergeConfig.flat("acknowledgementReference", "vpdApprovalNumber",
                                        "periodKey", "submittedAt", "status")
    );
}

Usage in route - before:

# Verbose: 8 steps per backend call
- removeHeaders:
    pattern: "CamelHttp*"
- setHeader:
    name: CamelHttpMethod
    constant: "GET"
- setHeader:
    name: X-Correlation-Id
    simple: "${exchangeProperty.correlationId}"
- toD:
    uri: "http://customer-proxy:4010/customers/${exchangeProperty.customerId}?bridgeEndpoint=true"
- setProperty:
    name: customerResponse
    simple: "${body}"
- setBody:
    groovy: |
      // ... 20 lines to merge into response ...

Usage in route - after:

# Concise: 1 step per backend call
- process:
    groovy: "Backends.CUSTOMER.call(exchange)"
 
# Or with inline configuration:
- process:
    groovy: |
      new BackendClient("customer",
          "http://customer-proxy:4010/customers/{customerId}",
          "customerResponse", null, null).call(exchange)

Clean YAML via Kamelets (native Camel reusable components with parameters):

Kamelets are YAML-defined reusable components with declared parameters - native to Camel since 3.8.

Kamelet definitions (kamelets/backend-customer.kamelet.yaml):

apiVersion: camel.apache.org/v1
kind: Kamelet
metadata:
  name: backend-customer
  labels:
    camel.apache.org/kamelet.type: action
spec:
  definition:
    title: Customer Backend
    description: Call customer service and merge trader into response
    required:
      - customerId
    properties:
      customerId:
        title: Customer ID
        description: The customer ID to look up
        type: string
  template:
    from:
      uri: kamelet:source
      steps:
        - process:
            groovy: |
              exchange.setProperty('customerId', '{{customerId}}')
              Backends.CUSTOMER.call(exchange)

Kamelet (kamelets/backend-taxplatform-byack.kamelet.yaml):

apiVersion: camel.apache.org/v1
kind: Kamelet
metadata:
  name: backend-taxplatform-byack
  labels:
    camel.apache.org/kamelet.type: action
spec:
  definition:
    title: Tax Platform Backend (by Acknowledgement)
    description: Retrieve submission by acknowledgement reference
    required:
      - ackRef
    properties:
      ackRef:
        title: Acknowledgement Reference
        description: The acknowledgement reference to look up
        type: string
  template:
    from:
      uri: kamelet:source
      steps:
        - process:
            groovy: |
              exchange.setProperty('ackRef', '{{ackRef}}')
              Backends.TAX_PLATFORM_BY_ACK.call(exchange)

Kamelet (kamelets/backend-taxplatform-byapproval.kamelet.yaml):

apiVersion: camel.apache.org/v1
kind: Kamelet
metadata:
  name: backend-taxplatform-byapproval
  labels:
    camel.apache.org/kamelet.type: action
spec:
  definition:
    title: Tax Platform Backend (by Approval + Period)
    description: Retrieve submission by approval number and period key
    required:
      - approvalNumber
      - periodKey
    properties:
      approvalNumber:
        title: VPD Approval Number
        type: string
      periodKey:
        title: Period Key
        type: string
  template:
    from:
      uri: kamelet:source
      steps:
        - process:
            groovy: |
              exchange.setProperty('approvalNumber', '{{approvalNumber}}')
              exchange.setProperty('periodKey', '{{periodKey}}')
              Backends.TAX_PLATFORM_BY_APPROVAL.call(exchange)

Route becomes pure orchestration with clean Kamelet calls:

- route:
    id: getByAcknowledgement
    from:
      uri: direct:getByAcknowledgement
      steps:
        - to: direct:extractStandardHeaders
 
        # Backend calls with explicit parameters - clean and self-documenting
        - kamelet:
            name: backend-taxplatform-byack
            parameters:
              ackRef: "${exchangeProperty.ackRef}"
 
        - kamelet:
            name: backend-customer
            parameters:
              customerId: "${exchangeProperty.customerId}"
 
        # Response already assembled, just apply fieldsets and return
        - setBody:
            groovy: "exchange.getProperty('response').toString()"
        - to: direct:applySparseFieldsets
        - to: direct:injectResponseHeaders
 
- route:
    id: getByApprovalAndPeriod
    from:
      uri: direct:getByApprovalAndPeriod
      steps:
        - to: direct:extractStandardHeaders
 
        - kamelet:
            name: backend-taxplatform-byapproval
            parameters:
              approvalNumber: "${exchangeProperty.approvalNumber}"
              periodKey: "${exchangeProperty.periodKey}"
 
        - kamelet:
            name: backend-customer
            parameters:
              customerId: "${exchangeProperty.customerId}"
 
        - setBody:
            groovy: "exchange.getProperty('response').toString()"
        - to: direct:applySparseFieldsets
        - to: direct:injectResponseHeaders

JBang command to load Kamelets:

camel run routes/*.yaml kamelets/*.yaml lib/*.java lib/*.groovy \
  --dep=org.apache.camel:camel-kamelet \
  --dep=org.apache.camel:camel-groovy \
  --dep=org.apache.groovy:groovy-json:4.0.29

Benefits of Kamelets:

Native Camel feature - well supported and documented
Self-documenting with title, description, and typed parameters
Required parameters validated automatically
Reusable across integrations - can be shared as a library
Clean calling syntax with explicit parameters
IDE support for Kamelet catalogs

Fallback: Direct routes with header args (if Kamelets don’t work with JBang):

If the spike reveals Kamelets aren’t viable with JBang, use direct routes with a consistent header convention:

# Wrapper route
- route:
    id: backend-customer
    from:
      uri: direct:backend-customer
      steps:
        - process:
            groovy: |
              exchange.setProperty('customerId', exchange.in.getHeader('param.customerId'))
              Backends.CUSTOMER.call(exchange)
 
# Calling syntax (slightly less clean but workable)
- setHeader:
    name: param.customerId
    simple: "${exchangeProperty.customerId}"
- to: direct:backend-customer

This is more verbose than Kamelets but achieves the same goal of explicit parameters.

C. Header Propagation → YAML Direct Route (routes/common.yaml):

Best fit: Simple, declarative header manipulation that’s easy to understand and modify.

# Reusable header extraction - call at start of each route
- route:
    id: extractStandardHeaders
    from:
      uri: direct:extractStandardHeaders
      steps:
        - setProperty:
            name: correlationId
            simple: "${header.X-Correlation-Id}"
        - setProperty:
            name: idempotencyKey
            simple: "${header.X-Idempotency-Key}"
        - log:
            message: "Request correlationId=${exchangeProperty.correlationId}"
            loggingLevel: DEBUG
 
# Reusable header injection - call before returning response
- route:
    id: injectResponseHeaders
    from:
      uri: direct:injectResponseHeaders
      steps:
        - setHeader:
            name: Content-Type
            constant: "application/json"
        - setHeader:
            name: X-Correlation-Id
            simple: "${exchangeProperty.correlationId}"
        - removeHeader:
            name: "fields[submission-returns]"
        - removeHeader:
            name: "fields[:submission-returns]"
 
# Reusable HTTP cleanup before backend calls
- route:
    id: prepareBackendCall
    from:
      uri: direct:prepareBackendCall
      steps:
        - removeHeaders:
            pattern: "CamelHttp*"
        - setHeader:
            name: CamelHttpMethod
            simple: "${exchangeProperty.httpMethod}"
        - setHeader:
            name: X-Correlation-Id
            simple: "${exchangeProperty.correlationId}"

Usage in route:

- route:
    id: getByAcknowledgement
    from:
      uri: direct:getByAcknowledgement
      steps:
        # Extract headers at start
        - to: direct:extractStandardHeaders
 
        # Prepare and make backend call
        - setProperty:
            name: httpMethod
            constant: "GET"
        - to: direct:prepareBackendCall
        - toD:
            uri: "http://tax-platform-proxy:4010/submissions/vpd/${exchangeProperty.ackRef}?bridgeEndpoint=true"
 
        # ... processing ...
 
        # Inject headers before response
        - to: direct:injectResponseHeaders

7c.5. Add unit tests for each reusable component:

A. Groovy Unit Tests (tests/unit/SparseFieldsetsTest.groovy):

import spock.lang.Specification
 
class SparseFieldsetsTest extends Specification {
 
    def "should filter top-level fields"() {
        given:
        def json = '{"name": "Test", "value": 123, "extra": "ignored"}'
        def fields = "name,value"
 
        when:
        def result = SparseFieldsets.apply(json, fields)
 
        then:
        result.filtered == true
        result.body == '{"name":"Test","value":123}'
    }
 
    def "should handle nested fields"() {
        given:
        def json = '{"trader": {"name": "Acme", "address": {"city": "London"}}}'
        def fields = "trader.name,trader.address.city"
 
        when:
        def result = SparseFieldsets.apply(json, fields)
 
        then:
        result.filtered == true
        result.body.contains('"name":"Acme"')
    }
 
    def "should return error for invalid fields"() {
        given:
        def json = '{"name": "Test"}'
        def fields = "nonexistent"
 
        when:
        def result = SparseFieldsets.apply(json, fields)
 
        then:
        result.error == true
        result.invalidFields == "nonexistent"
    }
 
    def "should pass through when no fields param"() {
        given:
        def json = '{"name": "Test"}'
 
        when:
        def result = SparseFieldsets.apply(json, null)
 
        then:
        result.filtered == false
        result.body == json
    }
}

Run with: groovy -cp lib/ tests/unit/SparseFieldsetsTest.groovy

B. Java Unit Tests (tests/unit/BackendClientTest.java):

//DEPS org.junit.jupiter:junit-jupiter:5.10.0
//DEPS org.apache.camel:camel-core:4.4.0
//DEPS com.fasterxml.jackson.core:jackson-databind:2.17.0
 
import org.junit.jupiter.api.Test;
import org.apache.camel.Exchange;
import org.apache.camel.impl.DefaultCamelContext;
import org.apache.camel.support.DefaultExchange;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import static org.junit.jupiter.api.Assertions.*;
 
class BackendClientTest {
    private static final ObjectMapper mapper = new ObjectMapper();
 
    @Test
    void shouldResolveUrlFromExchangeProperties() {
        Exchange exchange = new DefaultExchange(new DefaultCamelContext());
        exchange.setProperty("customerId", "CUST789");
        exchange.setProperty("ackRef", "ACK-123");
 
        BackendClient client = new BackendClient(
            "test",
            "http://example.com/customers/{customerId}/ack/{ackRef}",
            "response", null, null
        );
 
        // Use reflection or expose for testing
        String url = client.resolveUrl(exchange);
        assertEquals("http://example.com/customers/CUST789/ack/ACK-123", url);
    }
 
    @Test
    void shouldMergeAsNestedObject() throws Exception {
        Exchange exchange = new DefaultExchange(new DefaultCamelContext());
 
        // Simulate backend response already stored
        String customerJson = """
            {
                "customerId": "CUST789",
                "name": "Example Vapes Ltd",
                "type": "ORG",
                "registeredAddress": {"line1": "123 High St"}
            }
            """;
 
        BackendClient.MergeConfig config = BackendClient.MergeConfig
            .nested("trader", "name", "type", "registeredAddress")
            .rename("registeredAddress", "address");
 
        // Test merge logic directly
        ObjectNode response = mapper.createObjectNode();
        exchange.setProperty("response", response);
 
        // Call internal merge method (would need to expose for testing)
        JsonNode customerNode = mapper.readTree(customerJson);
        ObjectNode trader = mapper.createObjectNode();
        trader.put("name", customerNode.path("name").asText());
        trader.put("type", customerNode.path("type").asText());
        trader.set("address", customerNode.path("registeredAddress"));
        response.set("trader", trader);
 
        // Verify
        assertTrue(response.has("trader"));
        assertEquals("Example Vapes Ltd", response.get("trader").get("name").asText());
        assertEquals("ORG", response.get("trader").get("type").asText());
        assertTrue(response.get("trader").has("address"));
    }
 
    @Test
    void shouldExtractFieldsIntoProperties() throws Exception {
        Exchange exchange = new DefaultExchange(new DefaultCamelContext());
 
        String response = """
            {"customerId": "CUST789", "vpdApprovalNumber": "VPD123"}
            """;
 
        // Simulate field extraction
        JsonNode json = mapper.readTree(response);
        exchange.setProperty("taxPlatform_customerId", json.path("customerId").asText());
        exchange.setProperty("taxPlatform_vpdApprovalNumber", json.path("vpdApprovalNumber").asText());
 
        assertEquals("CUST789", exchange.getProperty("taxPlatform_customerId"));
        assertEquals("VPD123", exchange.getProperty("taxPlatform_vpdApprovalNumber"));
    }
 
    @Test
    void shouldCreatePreConfiguredBackends() {
        // Verify pre-configured backends have correct settings
        assertNotNull(Backends.CUSTOMER);
        assertNotNull(Backends.TAX_PLATFORM_BY_ACK);
 
        assertEquals("customer", Backends.CUSTOMER.name);
        assertEquals("taxPlatform", Backends.TAX_PLATFORM_BY_ACK.name);
    }
}

Run with: jbang tests/unit/BackendClientTest.java

Note: Full integration testing of BackendClient.call() requires a running Camel context with HTTP component. The unit tests focus on URL resolution, merge logic, and configuration. The acceptance tests verify end-to-end behavior.

C. YAML Route Testing (integration-style with camel test):

Pure YAML routes can’t be unit tested in isolation, but Camel JBang provides a test runner:

Test file (tests/integration/common-routes-test.yaml):

# Test the header extraction/injection routes
- route:
    id: testExtractHeaders
    from:
      uri: direct:testExtractHeaders
      steps:
        - setHeader:
            name: X-Correlation-Id
            constant: "test-corr-123"
        - setHeader:
            name: X-Idempotency-Key
            constant: "test-idemp-456"
        - to: direct:extractStandardHeaders
        - to: mock:extractResult
 
- route:
    id: testInjectHeaders
    from:
      uri: direct:testInjectHeaders
      steps:
        - setProperty:
            name: correlationId
            constant: "injected-123"
        - to: direct:injectResponseHeaders
        - to: mock:injectResult

Run with: camel test tests/integration/common-routes-test.yaml --local

Alternative: Jest integration tests (already in place): The existing acceptance tests (tests/acceptance/) exercise the full routes through HTTP and verify correct behavior end-to-end. These provide confidence in the YAML routes without unit-level testing.

7c.6. Document when to use each approach:

Approach	Best For	Example	Testability
Groovy Script	JSON transformations, dynamic field access	Sparse fieldsets filtering	Spock/JUnit via `groovy` CLI
Java Processor	Multi-step patterns, type safety, reusable config	Backend call + response merge	Full JUnit, IDE integration
YAML Direct Route	Header manipulation, simple declarative flows	Extract/inject standard headers	Integration tests (`camel test`)

Decision guide:

Use Groovy when: Logic is self-contained, operates on JSON, benefits from dynamic typing
Use Java when: Pattern spans multiple concerns (HTTP + parsing + state), needs configuration, team prefers Java
Use YAML when: Logic is purely declarative (set header, remove header, simple choice)

7c.7. Wire Kamelets into refactored routes:

Routes call SparseFieldsets.apply() for field filtering (Groovy)
Backend calls use Backends.CUSTOMER.call(exchange) etc. (Java)
All routes use common header extraction/injection subroutes (YAML)

7c.8. Document the integration template pattern: Create a template structure for new domain API integrations:

domain-apis/specs/{domain}/
├── routes/
│   ├── rest-config.yaml      # REST endpoints → direct routes
│   ├── get-{resource}.yaml   # One file per GET variant
│   ├── post-{resource}.yaml  # POST route
│   └── common.yaml           # Health, shared subroutes
├── lib/
│   └── {domain}-utils.groovy # Domain-specific utilities (if any)
└── tests/
    └── integration/          # Per-route integration tests

7c.9. Evaluate reuse mechanisms:

Mechanism	Use Case	Pros	Cons
External Groovy scripts	Complex transformations (sparse fieldsets)	Testable, clean YAML	Extra file, class loading
Direct routes (`direct:`)	Reusable subroutes within same integration	Simple, no extra files	Limited to same file/context
Route templates	Parameterized patterns across integrations	Very reusable	More complex, less readable
Java processors	Complex logic, type safety	Full IDE support, testable	Heavier, requires Java knowledge

7c.10. Refactor current integration.yaml:

Split into separate route files
Extract sparse fieldsets to Groovy utility
Verify all tests still pass after refactoring

Success Criteria

Deliverables

File	Description
Route Files
`routes/rest-config.yaml`	REST configuration and endpoint definitions
`routes/get-by-ack.yaml`	GET by acknowledgementReference route
`routes/get-by-approval.yaml`	GET by vpdApprovalNumber+periodKey route
`routes/post-submission.yaml`	POST submission route
`routes/common.yaml`	Header propagation subroutes (YAML approach)
Kamelets
`kamelets/backend-customer.kamelet.yaml`	Customer backend Kamelet
`kamelets/backend-taxplatform-byack.kamelet.yaml`	Tax platform (by ack) Kamelet
`kamelets/backend-taxplatform-byapproval.kamelet.yaml`	Tax platform (by approval+period) Kamelet
Reusable Components
`lib/sparse-fieldsets.groovy`	Sparse fieldsets utility (Groovy approach)
`lib/BackendClient.java`	Backend client with response merge (Java approach)
Unit Tests
`tests/unit/SparseFieldsetsTest.groovy`	Spock tests for Groovy sparse fieldsets
`tests/unit/BackendClientTest.java`	JUnit tests for Java backend client
`tests/integration/common-routes-test.yaml`	Camel test for YAML routes
Documentation
`docs/INTEGRATION_TEMPLATE.md`	Template for new domain API integrations
`docs/REUSE_PATTERNS.md`	When to use Groovy vs Java vs YAML

Phase 7d: Evaluation and Documentation

Objective: Document findings comparing YAML DSL approach to traditional Java/Quarkus.

Tasks

7d.1. Compare approaches:

Aspect	Java/Quarkus	JBang + YAML DSL
Setup complexity	Maven project, dependencies	Single command, inline deps
Lines of code	~500 LOC	~300 LOC
Readability	Familiar to Java devs	More declarative, less boilerplate
Debugging	Full IDE support	Limited, log-based
Testing	JUnit, MockEndpoints	Integration tests only
Hot reload	Quarkus dev mode	`camel run --dev`
Production readiness	Proven	Emerging

7d.2. Document trade-offs:

When YAML DSL excels (simple orchestration, rapid prototyping)
When Java is better (complex logic, custom processors, extensive testing)
Hybrid approach possibilities

7d.3. Make recommendation:

For this POC: YAML DSL sufficient
For production: Consider based on team skills and complexity

Success Criteria

Comparison document complete
Clear recommendation with rationale
Trade-offs documented for future reference

Deliverables

File	Description
`docs/CAMEL_YAML_EVALUATION.md`	Full comparison and recommendation

Implementation Order

Option A: YAML DSL First (Recommended for POC)

Phase 0: API Specifications
    │
    ▼
Phase 1: Foundation (mocks with JSON, docker-compose, LGTM stack, k6)
    │
    ▼
Phase 1a: XML Content Type & Observability (excise XML/WireMock, Envoy proxies)
    │
    ▼
Phase 7a: JBang + YAML DSL Setup (simple pass-through)
    │
    ▼
Phase 7b: YAML DSL Orchestration (full flows, XML transform, parallel)
    │
    ▼
Phase 4: Sparse Fieldsets (field filtering)
    │
    ▼
Phase 5: Kong Integration
    │
    ▼
Phase 6: Kubernetes Deployment
    │
    ▼
Phase 7c: Reusability Patterns (split routes, extract utilities)
    │
    ▼
Phase 7d: Evaluation and Documentation

Option B: Java/Quarkus First (Traditional)

Phase 0 → Phase 1 → Phase 1a → Phase 2 → Phase 3 → Phase 4 → Phase 5 → Phase 6 → Phase 7

Recommendation: Option A for this POC - validates YAML DSL viability faster with less setup.

Each phase produces a committable, testable increment.

Open Design Decisions (from spec)

Decision	Resolution Timing
Domain API base path	Phase 5 (Kong config) - likely `/duty/vpd/*`
Sparse fieldsets implementation location	Phase 4
Error response format	Phase 3 - RFC 7807 vs custom

Test Data

Sample IDs

Resource	Pattern	Example
Approval Number	`VPD[0-9]{6}`	`VPD123456`
Period Key	`[0-9]{2}[A-Z][0-9]`	`24A1`
Acknowledgement	`ACK-YYYY-MM-DD-NNNNNN`	`ACK-2026-01-26-000123`

Mock Data Seeding

APS:

VPD123456 - ACTIVE account
- 24A1 - OPEN period
- 24A2 - FILED period
VPD999999 - Returns 404 (not found)

SAS:

Existing submission: (VPD123456, 24A2) with ack ACK-2026-01-01-000001

Test Scenarios Matrix

Scenario	APS	RCV	SAS	Expected
New filing, open period	200	200 valid	201	201 + ack
~~Replay same key + same payload~~	~~200~~	~~200~~	~~200 cached~~	Deferred
~~Replay same key + diff payload~~	~~200~~	~~200~~	~~409~~	Deferred
Unknown approval	404	–	–	404
Period already filed	409	–	–	409
Rule failure	200	422	–	422
GET by ack, exists	–	–	200	200 + enriched
GET by approval+period	200	–	200	200 + enriched
GET not found	–	–	404	404

Fault Injection via Envoy

Purpose

Simulate realistic network conditions (latency and errors) to verify the domain API handles slow/failing backend responses gracefully and that parallel orchestration provides actual latency benefits.

Implementation Approach

Two-layer architecture: Domain API controls faults, Envoy executes them.

Domain API (Phase 2):
- Reads fault configuration from application.yaml
- Generates random delay values based on per-backend ranges
- Determines if error should be injected based on error-rate percentage
- Passes fault instructions via headers to Envoy
Envoy proxies (Phase 1a):
- Configured in header-controlled mode (header_delay, header_abort)
- Respects x-envoy-fault-delay-request: <ms> for delays
- Respects x-envoy-fault-abort-request: <status> for errors
- No static fault config - fully controlled by domain API

Benefits of this approach:

Centralized fault configuration in domain API’s application.yaml
Per-request control over faults
Easy to disable faults without restarting Envoy
Works with any mock technology (Prism, WireMock, custom)
Faults logged by domain API for analysis

Domain API configuration (application.yaml):

fault-injection:
  enabled: true
  excise:
    delay: { min: 100, max: 500 }
    error-rate: 5
  customer:
    delay: { min: 50, max: 300 }
    error-rate: 5
  tax-platform:
    delay: { min: 100, max: 400 }
    error-rate: 5

Envoy fault filter (header-controlled mode):

- name: envoy.filters.http.fault
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.fault.v3.HTTPFault
    delay:
      header_delay: {}  # Use x-envoy-fault-delay-request header
      percentage:
        numerator: 100
        denominator: HUNDRED
    abort:
      header_abort: {}  # Use x-envoy-fault-abort-request header
      percentage:
        numerator: 100
        denominator: HUNDRED

Per-Backend Fault Profiles

Backend	Delay Range	Error Rate	Notes
excise	100-500ms	5%	Legacy XML system, slower
customer	50-300ms	5%	Fast lookup service
tax-platform	100-400ms	5%	Storage operations

Test Scenarios with Fault Injection

Baseline (faults disabled) - Verify correct behavior with no delays/errors
Default faults (5% errors, random delays) - Normal operation with realistic conditions
High delays (500-2000ms) - Verify timeout handling and resilience
High error rate (20-50%) - Verify circuit breaker and retry logic
Header-controlled faults - Inject specific delays/errors per request for targeted testing

Dynamic Fault Control via Headers

# Inject specific delay
curl -H "x-envoy-fault-delay-request: 2000" http://localhost:4010/excise/vpd/registrations/VPD123456
 
# Inject specific error
curl -H "x-envoy-fault-abort-request: 503" http://localhost:4010/excise/vpd/registrations/VPD123456
 
# Bypass faults for debugging
curl -H "x-envoy-fault-abort-request: 0" -H "x-envoy-fault-delay-request: 0" http://localhost:4010/...

Benefits

Works with any mock technology (Prism, WireMock, custom)
Simulates network-level issues more realistically than application-level injection
No domain API code changes needed
Centralized configuration in Envoy
Can inject errors that mocks themselves can’t simulate
Header-controlled faults enable targeted testing
Delay values logged for analysis

Load Testing with k6

Tool Choice (see spec Design Decision #6): k6 chosen for modern JavaScript API, developer-friendly scripting, and Grafana integration.

Purpose

Validate domain API performance under load and measure the impact of:

Sequential vs parallel orchestration
Backend delay injection on throughput
Error rate under load
Response time percentiles (p50, p95, p99)

Test Scenarios

Scenario	VUs	Duration	Description
Baseline	10	1 minute	Normal load, no delays
Load test	50	5 minutes	Sustained load, delays enabled
Spike test	0→100→0	3 minutes	Sudden traffic spike
Stress test	100+	Until failure	Find breaking point

Implementation

k6 script (tests/load/submit-return.js):

import http from 'k6/http';
import { check, sleep } from 'k6';
import { Rate, Trend } from 'k6/metrics';
 
const errorRate = new Rate('errors');
const submissionTime = new Trend('submission_duration');
 
export const options = {
  scenarios: {
    baseline: {
      executor: 'constant-vus',
      vus: 10,
      duration: '1m',
      startTime: '0s',
    },
    load: {
      executor: 'constant-vus',
      vus: 50,
      duration: '5m',
      startTime: '2m',
    },
  },
  thresholds: {
    'http_req_duration': ['p(95)<2000'], // 95% under 2s
    'errors': ['rate<0.1'],               // Error rate < 10%
  },
};
 
export default function () {
  const payload = JSON.stringify({
    vpdApprovalNumber: 'VPD123456',
    periodKey: '24A1',
    basicInformation: {
      returnType: 'ORIGINAL',
      submittedBy: {
        type: 'ORG',
        name: 'Example Vapes Ltd',
      },
    },
    dutyProducts: [],
  });
 
  const params = {
    headers: {
      'Content-Type': 'application/json',
      'X-Correlation-Id': `test-${__VU}-${__ITER}`,
      'X-Idempotency-Key': `test-${__VU}-${__ITER}`,
    },
  };
 
  const res = http.post('http://localhost:8080/duty/vpd/submission-returns/v1', payload, params);
 
  const success = check(res, {
    'status is 201': (r) => r.status === 201,
    'has acknowledgementReference': (r) => JSON.parse(r.body).acknowledgementReference !== undefined,
    'response time < 2s': (r) => r.timings.duration < 2000,
  });
 
  errorRate.add(!success);
  submissionTime.add(res.timings.duration);
 
  sleep(1);
}

Running Load Tests

# Install k6
brew install k6
 
# Run baseline scenario
k6 run tests/load/submit-return.js
 
# Run with custom VUs
k6 run --vus 100 --duration 2m tests/load/submit-return.js
 
# Generate HTML report
k6 run --out json=results.json tests/load/submit-return.js
k6-reporter results.json --output report.html

Success Criteria

p95 response time < 2 seconds under normal load (50 VUs)
Error rate < 1% under normal load
Throughput > 40 requests/second at 50 VUs
No memory leaks - stable memory usage over 5 minute test
Graceful degradation - error rate increases predictably under stress

Deliverables

File	Description
`tests/load/submit-return.js`	k6 POST submission test
`tests/load/get-return.js`	k6 GET retrieval test
`tests/load/README.md`	Load testing guide
`tests/load/run-all.sh`	Script to run all load scenarios

Techcle Wiki

Explorer

Plan

0009: VPD Submission Returns Domain API POC - Implementation Plan

Reference

Overview

Phase 0: API Specifications

Tasks

Success Criteria

Deliverables

Phase 1: Foundation

Tasks

Success Criteria

Deliverables

Phase 1a: XML Content Type & Observability Enhancement

Tasks

Success Criteria

Deliverables

Docker Compose Architecture

Testing the Compose Stack

Phase 2: Camel Domain Service

Tasks

Success Criteria

Deliverables

Phase 3: Orchestration

Tasks

Success Criteria

Deliverables

Phase 4: Sparse Fieldsets

Tasks

Success Criteria

Deliverables

Phase 5: Kong Integration

Tasks

Success Criteria

Deliverables

Phase 6: Kubernetes Deployment

Tasks

Success Criteria

Deliverables

Phase 7a: Camel YAML DSL - Simple Pass-Through

Tasks

Success Criteria

Deliverables

Phase 7b: Camel YAML DSL - Full Orchestration

Tasks

Success Criteria

Deliverables

Phase 7c: Reusability Patterns

Tasks

Success Criteria

Deliverables

Phase 7d: Evaluation and Documentation

Tasks

Success Criteria

Deliverables

Implementation Order

Option A: YAML DSL First (Recommended for POC)

Option B: Java/Quarkus First (Traditional)

Open Design Decisions (from spec)

Test Data

Sample IDs

Mock Data Seeding

Test Scenarios Matrix

Fault Injection via Envoy

Purpose

Implementation Approach

Per-Backend Fault Profiles

Test Scenarios with Fault Injection

Dynamic Fault Control via Headers

Benefits

Load Testing with k6

Purpose

Test Scenarios

Implementation

Running Load Tests

Success Criteria

Deliverables

Definition of Done

Graph View