0009: Vaping Duty Domain API POC

Summary

Implement a proof-of-concept Domain API for the Vaping Duty Submission Returns domain that orchestrates three backend APIs through a single unified interface. The POC validates the Domain API pattern using Apache Camel microservices behind Kong Enterprise gateway on Kubernetes, demonstrating:

Multi-backend sequential orchestration with validation gates
Idempotency handling via X-Idempotency-Key
Header propagation (CorrelationId, IdempotencyKey)
CorrelationId tracing through all backend calls
Sparse fieldsets (soft filtering) for efficient data retrieval

Problem Statement

Historically, programmes and projects have created ‘single use’ APIs in the HODs for their particular use case and exposed these via HIP. This leads to API proliferation, inconsistent interaction patterns, and tight coupling between consumers and backend implementations.

The Domain API programme aims to re-model the landscape with domain-aligned, reusable APIs that can be extended over time and consumed as needed. Vaping Duty, as a new domain, provides an ideal opportunity to establish this pattern from the outset rather than retrofitting existing APIs.

Without domain APIs, future Vaping Duty consumers would need to:

Integrate with multiple backend APIs directly, each with different interaction patterns
Request bespoke APIs from HODs or handle varying response formats (XML from legacy systems, JSON from modern)
Manually correlate data across systems (e.g., traders, products, declarations)
Over-fetch data when only specific fields are needed

The Domain API pattern addresses these issues by providing a unified, orchestrated interface that abstracts backend complexity while offering modern API capabilities like sparse fieldsets and consistent resource traversal.

Goals

Domain API Orchestration - Single Camel-based microservice that orchestrates calls to 3 backend APIs (excise, customer, tax-platform) with sequential validation gates for VPD Submission Returns
XML Content Type Transformation - Transform XML responses from legacy excise backend into JSON for unified API response
CorrelationId Tracing - Propagate X-Correlation-Id through all backend calls for end-to-end traceability
Sparse Fieldsets (Soft Filtering) - Allow clients to request only specific fields via fields[:resource] query parameter
Kong Enterprise Integration - Integrate with existing Kong gateway (auth and rate limiting already configured)
Kubernetes Deployment - Deploy all components to k8s sandbox cluster
Camel YAML DSL Exploration - Validate whether orchestration can be defined in YAML instead of Java code

Non-Goals

Idempotency Handling - Full idempotency via X-Idempotency-Key requires stateful backend; deferred to integration with real backends. POC will forward the header but stateless mocks won’t validate idempotency semantics.
Resource Traversal - Deferred to future (see Future Features)
Field Filtering on Traversed Resources - Deferred to future
Record Filtering/Sorting - Deferred to future (JSON:API syntax)
GraphQL Endpoint - Deferred to future
Real Backend Integration - Using mock backends for this POC
Field Level Scopes (Hard Filtering) - Scope-based field filtering to prevent sensitive information being disclosed to unauthorised clients. Deferred to future.
Optimised HOD Requests - Intelligently reducing backend calls based on sparse fieldset. For this POC, all backends are called regardless of field selection.
Egress Testing - Mocks run locally/in-cluster without going through egress. Egress configuration is out of scope for this POC.
Full Amendment Flow - PUT/PATCH for amending submissions. POST handles initial filing; amendment semantics deferred.
ETag/Optimistic Locking - Domain API aggregates data from multiple backends; a single backend’s ETag doesn’t represent the composite response. Aggregated ETag approach deferred to future.
OAuth Scope Enforcement - Mocks accept any valid token; scope-based authorisation is simulated but not enforced.

Open Design Decisions

Note: Kong plugins are listed as options but may not be preferred given uncertainty about Kong’s future in the platform.

Decisions

Decision	Options	Status
Domain API base path?	A) `/duty/vpd/` (as per OAS) B) `/api/duty/vpd/` C) Other convention	Leaning A - matches OAS
Where to implement sparse fieldsets?	A) In Domain API (Camel processor) - backend optimization B) Kong plugin - simpler but less efficient C) Shared library/adapter	Decided: A - Domain API level for backend optimization and orchestration awareness (see rationale below)
Error response format?	A) RFC 7807 Problem Details B) Custom error schema matching OAS	Open - align with HIP standards
Mock implementation?	A) Prism (OpenAPI-native, example-driven) B) WireMock (more flexible, requires stubs)	Decided: A - Prism for stateless POC; WireMock for future stateful scenarios
Mock observability?	A) Envoy proxy (native OTel) B) NGINX proxy (simpler config) C) Direct Prism (limited observability)	Recommended: A - Envoy for OTel support; NGINX acceptable fallback
Observability stack?	A) LGTM all-in-one (Loki, Grafana, Tempo, Mimir) B) Separate tools (Jaeger, Prometheus, etc.)	Decided: A - LGTM for dev/test simplicity; separate for production
Domain API framework?	A) Quarkus + Camel (declarative DSL) B) Spring Boot (familiar, larger ecosystem)	Decided: A - Camel’s orchestration DSL and cloud-native design preferred
Load testing tool?	A) k6 (JavaScript, dev-friendly) B) JMeter (GUI, XML) C) Gatling (Scala)	Decided: A - k6 for developer experience and scriptability
API Explorer UI?	A) Swagger UI (mature, reliable) B) Stoplight Elements (modern UX)	Recommended: A - Swagger UI default; either acceptable
OpenAPI version?	A) 3.0.3 (tool compatibility) B) 3.1 (full JSON Schema)	Decided: A - 3.0.3 until tool ecosystem catches up
Advanced API vs Simple API?	A) Advanced API (custom Java/Camel) B) Simple API with Camel YAML DSL	Leaning A - POC validates if B is viable (stretch goal)
Orchestration pattern?	A) Sequential only B) Parallel only C) Hybrid (sequential validation, parallel fetch)	Decided: C - Validation sequentially, parallelize where safe
Fragment versioning?	A) Centrally-versioned with $ref B) Inlined copies per API	Decided: A - Central versioning prevents drift, enables gradual migration
Stateful vs stateless mocks?	A) Stateless (Prism examples) B) Stateful (custom logic)	Decided: A - Stateless for POC; idempotency deferred to integration

Decision Rationale: Sparse Fieldsets at Domain API Level

Why Domain API (not gateway)?

Backend optimization: Avoid fetching unnecessary data from backends based on requested fields
Orchestration awareness: Domain API knows which backends provide which fields
Flexible filtering: Can optimize parallel fetch strategy based on requested fields
Not just projection: Filtering affects backend call decisions, not just response filtering

Gateway-level alternative: Simpler (just filter response JSON) but always fetches full data from all backends. Can work with any maturity of backend.

Domain Model: VPD Submission Returns

Domain Overview

Vaping Products Duty (VPD) is a UK excise duty on vaping products. The primary domain resource is Submission Returns - periodic duty returns submitted by registered traders.

Domain API resource:

Submission Returns - VPD duty returns submitted by traders, containing duty calculations, VAT, and declarations

Endpoint:

POST /duty/vpd/submission-returns/v1  - Submit a return
GET  /duty/vpd/submission-returns/v1  - Retrieve a return

Backend Services (Systems of Record)

The Domain API orchestrates three backend services, each owning distinct domain data:

Backend Service	System of Record For	Key Capabilities	Content Type
excise	VPD registrations, periods, duty rules	Manages VPD approval numbers, period state, validation rules, calculations. Bridges VPD registrations to customer IDs.	XML (legacy system)
customer	Trader/organization master data	Provides trader names, types, addresses. Uses generic customer IDs (not VPD-specific).	JSON
tax-platform	Submission records and acknowledgements	Stores submissions, handles idempotency, retrieves by acknowledgement or approval+period.	JSON

Content Type Transformation: The domain API transforms XML responses from the excise backend into JSON for the unified domain API response. This demonstrates handling legacy systems with different content types.

Orchestration Flows

flowchart TB
    subgraph POST["POST /submission-returns/v1"]
        direction TB
        P1[1. excise: Validate + calculate<br/>Returns XML → Transform to JSON<br/>customerId, calculations] --> P2[2. PARALLEL]
        P2 --> P3[2a. tax-platform: Store JSON]
        P2 --> P4[2b. customer: Get trader details JSON]
        P3 --> P5[Return acknowledgement<br/>Unified JSON response]
        P4 --> P5
    end

    subgraph GET_ACK["GET ?acknowledgementReference=..."]
        direction TB
        G1[1. tax-platform: Get submission JSON<br/>Returns: vpdApprovalNumber, periodKey] --> G2[2. excise: Get registration/period<br/>Returns XML → Transform to JSON<br/>customerId]
        G2 --> G3[3. customer: Get trader details JSON]
        G3 --> G4[Return enriched submission<br/>Unified JSON response]
    end

    subgraph GET_PERIOD["GET ?vpdApprovalNumber=...&periodKey=..."]
        direction TB
        G5[1. excise: Validate access + get period<br/>Returns XML → Transform to JSON<br/>customerId] --> G6[2. PARALLEL]
        G6 --> G7[2a. tax-platform: Find submission JSON]
        G6 --> G8[2b. customer: Get trader details JSON]
        G7 --> G9[Return enriched submission<br/>Unified JSON response]
        G8 --> G9
    end

POST Flow (Submit Return):

excise (XML) - Validate vpdApprovalNumber exists and is active; validate periodKey is open; validate submission payload; calculate duty totals and VAT. Returns customerId and calculations in XML format. Domain API transforms XML→JSON.
Parallel:
- tax-platform (JSON) - Store submission idempotently using X-Idempotency-Key; return acknowledgement reference
- customer (JSON) - Get trader details for response enrichment

GET Flow (by acknowledgement reference):

tax-platform (JSON) - Lookup submission by acknowledgement reference; returns vpdApprovalNumber, periodKey, submission payload
excise (XML) - Get VPD registration and period context; returns customerId, approval status, period state in XML format. Domain API transforms XML→JSON.
customer (JSON) - Get trader details for response enrichment

GET Flow (by approval + period):

excise (XML) - Validate caller can access this approval/period combination; get period state; returns customerId in XML format. Domain API transforms XML→JSON.
Parallel:
- tax-platform (JSON) - Find submission by approval + period
- customer (JSON) - Get trader details for response enrichment

Orchestration Sequence Diagrams

POST /duty/vpd/submission-returns/v1

sequenceDiagram
    participant Client
    participant Domain API
    participant Excise (XML)
    participant Tax Platform (JSON)
    participant Customer (JSON)

    Client->>Domain API: POST /submission-returns/v1<br/>(JSON payload + X-Idempotency-Key)

    Note over Domain API: 1. Validate and Calculate
    Domain API->>Excise (XML): POST /validate-and-calculate<br/>Accept: application/xml<br/>(vpdApprovalNumber, periodKey, submission)
    Excise (XML)-->>Domain API: XML response<br/>(valid, customerId, calculations)
    Note over Domain API: Transform XML→JSON

    alt Validation Failed
        Domain API-->>Client: 422 Unprocessable Entity<br/>(validation errors)
    else Validation Passed
        Note over Domain API: 2. Parallel Orchestration
        par Store Submission
            Domain API->>Tax Platform (JSON): POST /submissions/vpd<br/>(X-Idempotency-Key, submission)
            Tax Platform (JSON)-->>Domain API: JSON response<br/>(acknowledgementRef)
        and Get Trader Details
            Domain API->>Customer (JSON): GET /customers/{customerId}
            Customer (JSON)-->>Domain API: JSON response<br/>(trader name, type, address)
        end

        Note over Domain API: Assemble unified JSON response
        Domain API-->>Client: 201 Created<br/>(acknowledgement + enriched data)
    end

GET /duty/vpd/submission-returns/v1?acknowledgementReference=…

sequenceDiagram
    participant Client
    participant Domain API
    participant Tax Platform (JSON)
    participant Excise (XML)
    participant Customer (JSON)

    Client->>Domain API: GET ?acknowledgementReference=ACK-123

    Note over Domain API: 1. Retrieve Submission
    Domain API->>Tax Platform (JSON): GET /submissions/vpd/{acknowledgementRef}
    Tax Platform (JSON)-->>Domain API: JSON response<br/>(submission, vpdApprovalNumber, periodKey)

    Note over Domain API: 2. Get Registration/Period Context
    Domain API->>Excise (XML): GET /registrations/{vpdApprovalNumber}<br/>Accept: application/xml
    Excise (XML)-->>Domain API: XML response<br/>(customerId, status, registeredDate)
    Note over Domain API: Transform XML→JSON

    Domain API->>Excise (XML): GET /periods/{periodKey}<br/>Accept: application/xml
    Excise (XML)-->>Domain API: XML response<br/>(startDate, endDate, state)
    Note over Domain API: Transform XML→JSON

    Note over Domain API: 3. Get Trader Details
    Domain API->>Customer (JSON): GET /customers/{customerId}
    Customer (JSON)-->>Domain API: JSON response<br/>(trader name, type, address)

    Note over Domain API: Assemble enriched JSON response
    Domain API-->>Client: 200 OK<br/>(enriched submission)

GET /duty/vpd/submission-returns/v1?vpdApprovalNumber=…&periodKey=…

sequenceDiagram
    participant Client
    participant Domain API
    participant Excise (XML)
    participant Tax Platform (JSON)
    participant Customer (JSON)

    Client->>Domain API: GET ?vpdApprovalNumber=VPD123&periodKey=24A1

    Note over Domain API: 1. Validate Access & Get Context
    Domain API->>Excise (XML): GET /registrations/{vpdApprovalNumber}<br/>Accept: application/xml
    Excise (XML)-->>Domain API: XML response<br/>(customerId, status, registeredDate)
    Note over Domain API: Transform XML→JSON

    Domain API->>Excise (XML): GET /periods/{periodKey}<br/>Accept: application/xml
    Excise (XML)-->>Domain API: XML response<br/>(startDate, endDate, state)
    Note over Domain API: Transform XML→JSON

    Note over Domain API: 2. Parallel Orchestration
    par Find Submission
        Domain API->>Tax Platform (JSON): GET /submissions/vpd?approval=...&period=...
        Tax Platform (JSON)-->>Domain API: JSON response<br/>(submission, acknowledgementRef)
    and Get Trader Details
        Domain API->>Customer (JSON): GET /customers/{customerId}
        Customer (JSON)-->>Domain API: JSON response<br/>(trader name, type, address)
    end

    Note over Domain API: Assemble enriched JSON response
    Domain API-->>Client: 200 OK<br/>(enriched submission)

Backend API Specifications

excise - Excise Duty System

System of record for: VPD registrations, periods, duty calculation rules

Endpoint	Description
`GET /excise/vpd/registrations/{vpdApprovalNumber}`	Returns VPD registration with customerId, approval status, registration date
`GET /excise/vpd/periods/{periodKey}`	Returns period definition (dates, duty rates) and state (OPEN/FILED/CLOSED)
`POST /excise/vpd/validate-and-calculate`	Validates submission payload against business rules; calculates duty and VAT; returns customerId, calculations, warnings

POST /excise/vpd/validate-and-calculate request:

{
  "vpdApprovalNumber": "VPD123456",
  "periodKey": "24A1",
  "submission": { /* full submission payload */ }
}

POST response:

{
  "valid": true,
  "customerId": "CUST789",
  "calculations": {
    "totalDutyDue": { "amount": 12345.67, "currency": "GBP" },
    "vat": { "amount": 2469.13, "rate": 0.20 },
    "calculationHash": "sha256:abc123..."
  },
  "warnings": [
    { "code": "WARN-VD-001", "text": "Spoilt product close to threshold" }
  ]
}

Error responses:

404 - Unknown approval number or period
409 - Period already filed or not in OPEN state
422 - Rule validation failures with structured error list
400 - Malformed payload

customer - Customer Master Data

System of record for: Trader/organization information

Endpoint	Description
`GET /customers/{customerId}`	Returns trader details (name, type, address)

GET response:

{
  "customerId": "CUST789",
  "name": "Example Vapes Ltd",
  "type": "ORG",
  "registeredAddress": {
    "line1": "123 High Street",
    "postcode": "AB1 2CD"
  }
}

Error responses:

404 - Customer not found

tax-platform - Submissions & Acknowledgements

System of record for: Submission records, acknowledgement references

Endpoint	Description
`POST /submissions/vpd`	Store submission idempotently (by X-Idempotency-Key); returns acknowledgement reference
`GET /submissions/vpd/{acknowledgementReference}`	Retrieve submission by acknowledgement reference
`GET /submissions/vpd?vpdApprovalNumber=...&periodKey=...`	Find submission by approval number and period

POST request headers:

X-Idempotency-Key - Required for idempotent storage
X-Correlation-Id - Propagated for tracing

POST request body:

{
  "vpdApprovalNumber": "VPD123456",
  "periodKey": "24A1",
  "customerId": "CUST789",
  "submission": { /* full payload */ },
  "calculations": { /* from excise */ },
  "warnings": [ /* from excise */ ]
}

POST response (201 Created):

{
  "acknowledgementReference": "ACK-2026-01-26-000123",
  "storedAt": "2026-01-26T14:51:02Z"
}

GET response:

{
  "acknowledgementReference": "ACK-2026-01-26-000123",
  "vpdApprovalNumber": "VPD123456",
  "periodKey": "24A1",
  "customerId": "CUST789",
  "submission": { /* full payload */ },
  "calculations": { /* stored from excise */ },
  "warnings": [],
  "submittedAt": "2026-01-26T14:51:02Z",
  "status": "RECEIVED"
}

Error responses:

404 - Submission not found
409 - Idempotent replay with different payload (same key, different content)

Domain API Endpoints

Endpoint	Method	Query Parameters	Description
`/duty/vpd/submission-returns/v1`	POST	-	Submit a VPD return; returns acknowledgement
`/duty/vpd/submission-returns/v1`	GET	`acknowledgementReference`	Retrieve by acknowledgement reference
`/duty/vpd/submission-returns/v1`	GET	`vpdApprovalNumber`, `periodKey`	Retrieve by approval + period

Headers

Header	Direction	Description
`X-Correlation-Id`	Request/Response	Correlation ID propagated to all backend calls
`X-Idempotency-Key`	Request (POST)	Idempotency key for safe retries

Example Responses

POST Response (201 Created):

{
  "acknowledgementReference": "ACK-2026-01-26-000123",
  "vpdApprovalNumber": "VPD123456",
  "periodKey": "24A1",
  "status": "RECEIVED",
  "storedAt": "2026-01-26T14:51:02Z",
  "warnings": [
    {"code": "WARN-VD-001", "text": "Spoilt product close to upper threshold"}
  ]
}

Headers: X-Correlation-Id: <echoed>

GET Response (200 OK):

{
  "acknowledgementReference": "ACK-2026-01-26-000123",
  "vpdApprovalNumber": "VPD123456",
  "periodKey": "24A1",
  "basicInformation": {
    "returnType": "ORIGINAL",
    "submittedBy": { "type": "ORG", "name": "Example Vapes Ltd" }
  },
  "totalDutyDue": { "amount": 12345.67, "currency": "GBP" },
  "vat": { "rate": 0.20, "amount": 2469.13 }
}

Headers: X-Correlation-Id: <echoed>

Technical Approach

The VPD Submission Returns Domain API is a microservice within HIP. It uses HIP’s existing infrastructure (Kong, k8s) while adding domain-specific orchestration.

C4Container
    title VPD Submission Returns Domain API

    Person(consumer, "API Consumer", "Client application")

    System_Boundary(hip, "HIP Platform") {
        Container(kong, "Kong Enterprise", "API Gateway", "Auth, rate limiting, routing")
        Container(domain_api, "VPD Domain API", "Camel/Quarkus", "Orchestration, parallel fetching, field filtering")
    }

    System_Boundary(backends, "Backend Services (Mocked)") {
        Container(excise, "excise", "Mock/JSON", "VPD registrations, periods, calculations")
        Container(customer, "customer", "Mock/JSON", "Trader master data")
        Container(tax_platform, "tax-platform", "Mock/JSON", "Submission storage")
    }

    Rel(consumer, kong, "HTTPS", "/duty/vpd/*")
    Rel(kong, domain_api, "HTTP")
    Rel(domain_api, excise, "HTTP/JSON", "Validate, calculate")
    Rel(domain_api, customer, "HTTP/JSON", "Enrich trader details")
    Rel(domain_api, tax_platform, "HTTP/JSON", "Store/retrieve")

Key points:

Domain API handles orchestration with parallel fetching where possible, idempotency, and client-facing concerns (sparse fieldsets, response shaping)
Kong provides cross-cutting concerns (auth, rate limiting, correlation ID injection) as it does for all HIP APIs
Backend Services are systems of record: excise owns VPD registrations/rules, customer owns trader data, tax-platform owns submissions
For this POC, mock backends simulate these systems
Parallel fetching after validation gates improves latency
This pattern demonstrates real enterprise integration patterns (idempotency, correlation, parallel orchestration)

Key Components

1. Apache Camel Domain Service

A Quarkus-based Camel application providing:

REST DSL Endpoints - Defining the domain API routes
Orchestration with Parallel Fetching - Validation gates with short-circuit, then parallel backend calls where possible
Header Propagation - CorrelationId, IdempotencyKey forwarding
Field Filtering - Post-processing to apply sparse fieldsets

// Example Camel route structure for POST
rest("/duty/vpd/submission-returns/v1")
    .post()
    .route()
    .setHeader("X-Correlation-Id", simple("${header.X-Correlation-Id}"))
    // Step 1: Validate and calculate with excise
    .to("direct:validateAndCalculateWithExcise")
    .choice()
        .when(simple("${body[valid]} == false"))
            .to("direct:returnValidationError")
        .otherwise()
            // Step 2: Parallel - store submission AND fetch trader details
            .multicast()
                .parallelProcessing()
                .to("direct:storeInTaxPlatform", "direct:fetchTraderFromCustomer")
            .end()
            .to("direct:returnAcknowledgement");

2. Kong Enterprise Gateway

Handles cross-cutting concerns:

Authentication - OAuth2 validation
Rate Limiting - Per-consumer limits
Correlation ID - Inject X-Correlation-Id if not present
Request Routing - Path-based routing to domain service

Kong routes /duty/vpd/* to the Camel domain service.

3. Idempotency Handling (Header Forwarding Only)

Note: Full idempotency validation is deferred to integration with real backends. The POC forwards the header but stateless Prism mocks don’t validate idempotency semantics.

The X-Idempotency-Key header enables safe retries:

POST /duty/vpd/submission-returns/v1
X-Idempotency-Key: abc-123-def

POC Behaviour:

Domain API forwards X-Idempotency-Key to tax-platform
Stateless mock returns consistent example responses (doesn’t validate key/payload matching)

Future Behaviour (with real backends):

First request: Full orchestration, return acknowledgement
Replay with same key + same payload hash: tax-platform returns cached acknowledgement (201)
Replay with same key + different payload: tax-platform returns 409 Conflict

4. Sparse Fieldsets (Soft Filtering)

Implements JSON:API-style sparse fieldsets, applied to the response.

Example - requesting only specific fields:

GET /duty/vpd/submission-returns/v1?acknowledgementReference=ACK-123&fields[submission]=acknowledgementReference,totalDutyDue,vat

Response with only requested fields:

{
  "acknowledgementReference": "ACK-2026-01-26-000123",
  "totalDutyDue": { "amount": 12345.67, "currency": "GBP" },
  "vat": { "rate": 0.20, "amount": 2469.13 }
}

5. Mock Backend Services

For this POC, all backends are mocked with Prism (stateless):

Service	Spec	Notes
excise-mock	`mocks/excise-api.yaml`	Stateless; returns deterministic validation/calculation responses
customer-mock	`mocks/customer-api.yaml`	Stateless; returns trader details
tax-platform-mock	`mocks/tax-platform-api.yaml`	Stateless; returns 201 for POST, 200 for GET with example data

Mock data in OAS examples:

excise: VPD123456 (ACTIVE, maps to CUST789) with periods 24A1 (OPEN), 24A2 (FILED)
customer: CUST789 → “Example Vapes Ltd” (ORG)
tax-platform: Returns acknowledgement references like ACK-2026-01-27-000123

Note on idempotency: Full idempotency testing (validating key/payload matching, conflict detection) is deferred to integration with real backends. Prism will return consistent responses but won’t validate idempotency semantics.

6. Observability Stack (LGTM)

Purpose: Full-stack observability for development and testing environments.

Components:

Grafana: Unified visualization UI (http://localhost:3000)
Loki: Log aggregation and querying
Tempo: Distributed tracing backend
Mimir: Prometheus-compatible metrics storage

Implementation: Single grafana/otel-lgtm Docker image provides all four components.

Logging Proxies:

Architecture: Lightweight proxy (Envoy or NGINX) in front of each Prism mock
Purpose: Prism doesn’t natively support tracing/structured logging
Proxy responsibilities:
- Log all requests (method, path, headers, status, timing)
- Extract and propagate trace context (W3C Trace Context)
- Emit structured JSON logs to Loki
- Emit OpenTelemetry traces to Tempo
- Forward requests to Prism unchanged
Benefits:
- Full observability without modifying Prism
- Consistent logging across all backend mocks
- Trace continuity: k6 → domain-api → proxy → mock
- Prism remains simple and replaceable

Domain API Instrumentation:

Quarkus OpenTelemetry extension for automatic instrumentation
Emit traces to Tempo (OTLP protocol)
Structured logging to Loki with correlation IDs
Metrics to Mimir (HTTP requests, latencies, error rates)

Trace Flow Example:

k6 load test
  → domain-api (span: POST /submission-returns)
    → excise-proxy (span: POST /validate-and-calculate)
      → excise-mock (Prism)
    → customer-proxy (span: GET /customers/{id})
      → customer-mock (Prism)
    → tax-proxy (span: POST /submissions)
      → tax-mock (Prism)

All spans visible in Grafana/Tempo with full request/response details.

7. Platform OAS Generation

Fragment Versioning:

Common fragments (headers, schemas, responses, parameters) are centrally versioned by the platform team
Both producer and platform OAS reference fragments via $ref (NOT inlined)
Platform generator:
1. Preserves all $ref to fragments
2. Injects sparse fieldsets for GET operations
3. Adds platform metadata
Benefits:
- Single source of truth for common elements
- Fragment changes propagate to all APIs
- No version drift or duplication
- API portal resolves $ref at runtime

Example:

# Producer OAS
responses:
  '400':
    $ref: '../fragments/responses.yaml#/BadRequest'
 
# Platform OAS (generated) - $ref preserved
responses:
  '400':
    $ref: '../fragments/responses.yaml#/BadRequest'

Directory Structure

repos/vaping-duty-poc/
├── docker-compose.yaml            # Run entire stack locally
├── domain-api/                    # Camel/Quarkus domain service
│   ├── specs/
│   │   └── vpd-submission-returns-api.yaml  # Domain API OAS
│   ├── src/main/java/
│   │   └── uk/gov/hmrc/vpd/
│   │       ├── VpdRoute.java
│   │       ├── orchestration/
│   │       └── fieldfilter/
│   ├── src/main/resources/
│   │   └── application.yaml
│   └── pom.xml
├── mocks/                         # Backend mocks (all Prism)
│   ├── excise-api.yaml            # Excise duty system spec
│   ├── customer-api.yaml          # Customer master data spec
│   ├── tax-platform-api.yaml      # Tax platform submissions spec
│   └── docker-compose.yaml        # Runs all Prism mock servers
├── kong/
│   └── kong.yaml
├── kustomize/
│   ├── base/
│   └── overlays/lab/
└── tests/
    ├── acceptance/
    └── integration/

Note: All mocks are stateless using Prism. Full idempotency testing (key/payload validation) is deferred to integration phase with real backends.

Success Criteria

Orchestration Works
- POST /duty/vpd/submission-returns/v1 orchestrates excise validation/calculation → parallel (tax-platform store + customer enrich) and returns acknowledgement
- GET by acknowledgementReference retrieves from tax-platform, enriches with excise registration/period and customer trader details
- GET by vpdApprovalNumber + periodKey validates via excise then parallelly fetches from tax-platform + customer
- Backend errors short-circuit the flow (fail fast, no partial processing)
- Parallel fetching improves latency where possible
- Mock backends include error simulation for testing all failure scenarios
XML Content Type Transformation Works
- Excise mock returns XML responses
- Domain API successfully parses XML and transforms to JSON
- Unified JSON response includes data from XML (excise) and JSON (customer, tax-platform) backends
- XML parsing errors handled gracefully with appropriate error responses
Header Propagation Works
- X-Correlation-Id propagated to all backend calls
- X-Idempotency-Key forwarded to tax-platform (validation deferred to real backend integration)
Sparse Fieldsets Work
- fields[resource] parameter filters response to requested fields only
- Invalid field names return 400 with helpful error
Kong Integration Works
- Traffic routes through existing Kong to domain service
- Authentication/rate limiting applied (using existing plugins)
- Correlation ID injected if not present
Kubernetes Deployment Works
- All components deploy to k8s sandbox cluster
- Services discoverable via internal DNS
- Health checks pass
Camel YAML DSL Evaluated
- Equivalent routes implemented in YAML DSL
- Comparison documented (complexity, maintainability, testability)
- Recommendation made for future domain APIs

Implementation Phases

Phase 0: API Specifications

Write backend API OAS specs (excise, customer, tax-platform)
Write domain API OAS (vpd-submission-returns-api.yaml)
Review and validate specs

Phase 1: Foundation

Create repository structure
Set up mocks with docker-compose (Prism for all backends)
Excise mock returns XML (not JSON) to validate content type transformation
Setup interactive API explorer (Swagger UI or Stoplight Elements):
- Accessible via browser (e.g., http://localhost:8090)
- Load all backend OAS specs (excise, customer, tax-platform)
- Enable “Try It” functionality to execute requests against Prism mocks
- Later: compare domain API responses vs direct mock calls
- Configure CORS to allow browser-based API calls
Setup observability with LGTM stack:
- Distributed tracing to show requests from client → domain-api → mocks
- Structured logging with correlation IDs
Setup k6 load testing:
- Basic smoke tests for each mock endpoint
- Correlation ID injection per request

Phase 2: Camel Domain Service

Create Quarkus/Camel project
Implement basic routes (pass-through to tax-platform)
Add header propagation (CorrelationId, IdempotencyKey forwarding)
Add OpenTelemetry instrumentation for tracing
Delay injection passed to Envoy (based on config and random timings)
Error injection passed to Envoy (based on config and random timings)

Phase 3: Orchestration

Implement orchestration with validation gates (excise validate/calculate)
Add XML→JSON transformation for excise responses
Add parallel fetching (tax-platform + customer after validation)
Add error mapping (backend errors → domain errors)
Integration tests for all orchestration flows

Phase 4: Sparse Fieldsets

Parse fields query parameters
Implement field masking
Add validation for unknown fields
Unit and integration tests

Phase 5: Kong Integration

Note: Kong is already running in the k8s cluster with authentication and rate limiting configured. This phase integrates the domain API, not sets up Kong from scratch.

Create Kong route configuration for /duty/vpd/*
Reference existing authentication plugin
Reference existing rate limiting configuration
Ensure correlation ID plugin is enabled
End-to-end tests through Kong in k8s

Phase 6: Kubernetes Deployment

Integrate mocks and domain API with existing LOB repo & Helm charts
Deploy to sandbox cluster
Verify end-to-end flow: Kong → domain-api → mock backends
Document deployment process

Phase 7: Camel YAML DSL Exploration

Create equivalent orchestration routes in Camel YAML DSL (instead of Java)
Compare complexity, maintainability, and testability
Document findings and recommendation for future domain APIs
Determine if YAML DSL is viable for complex orchestration patterns

Test Scenarios

The following scenarios should be covered by integration tests:

Scenario	excise	customer	tax-platform	Expected
New filing, open period	200 valid+calc (customerId)	200	201 stored	201 + acknowledgement
Unknown approval number	404	–	–	404
Period already filed	409	–	–	409
Rule failure (e.g., missing required field)	422 with details	–	–	422
GET by ack, exists	200 (registration+period)	200	200	200 + enriched
GET by approval+period, exists	200 (validates access)	200	200	200 + enriched
GET not found	200	(optional)	404	404

Deferred to integration with real backends:

Replay same idempotency key + same payload (expects cached 201 response)
Replay same key + different payload (expects 409 Conflict)
ETag validation and optimistic locking

Future Features

This POC establishes the foundation for additional capabilities:

1. Full Idempotency Handling

Complete idempotency validation when integrating with real backends:

POST /duty/vpd/submission-returns/v1
X-Idempotency-Key: abc-123-def

Full behaviour (requires stateful backend):

First request: Full orchestration, return acknowledgement
Replay with same key + same payload hash: tax-platform returns cached acknowledgement (201)
Replay with same key + different payload: tax-platform returns 409 Conflict
Idempotency survives service restarts

Testing requirements:

WireMock or similar stateful mock for idempotency testing
Test scenarios for key/payload matching, conflict detection, timeout handling

2. ETag/Optimistic Locking

Support optimistic concurrency control for PUT/PATCH operations:

Challenge: The domain API aggregates data from multiple backends (excise, customer, tax-platform). A single backend’s ETag doesn’t represent the composite response - if any backend’s data changes, the aggregated response changes.

Aggregated ETag approach:

GET /duty/vpd/submission-returns/v1?acknowledgementReference=ACK-123
→ Response includes: ETag: "agg-{hash-of-all-backend-etags}"

PUT /duty/vpd/submission-returns/v1?acknowledgementReference=ACK-123
If-Match: "agg-{hash}"
→ Domain API validates aggregated ETag before allowing update

Implementation considerations:

Generate composite ETag from hash of all backend ETags (or response content)
Store ETag-to-backend-ETags mapping (or recompute on each request)
On PUT/PATCH with If-Match: verify all backend data unchanged before proceeding
Return 412 Precondition Failed if any backend data changed

Complexity: High - requires careful handling of partial backend changes and retry semantics.

3. “Try It” Capability for HIP

Interactive API explorer allowing consumers to explore and test APIs before implementation:

Based on Swagger UI or similar
Supports sparse fieldsets, filtering, and other domain API features as they become available
Built into Integration Hub
Uses shared credentials in non-prod environments only
Enables rapid prototyping and validation of API queries

4. Resource Traversal (`include` parameter)

Allow embedding related resources via include query parameter:

GET /declarations/DECL001?include=trader,products

Returns declaration with full trader and products embedded, following links automatically.

Design considerations:

Limit traversal depth (max 2 levels)
Circular reference handling
Performance: parallel fetching of included resources

5. Field Filtering on Traversed Resources

Combine include with fields for precise control:

GET /declarations/DECL001?include=trader&fields[declaration]=id,status&fields[trader]=name

Only fetches and returns the specified fields from each resource type.

6. Record Filtering and Sorting (JSON:API Syntax)

Filter and sort collections using JSON:API query parameters:

GET /declarations?filter[status]=submitted&filter[period]=2024-Q1&sort=-submittedAt

Supported operations:

filter[field]=value - Exact match
filter[field][gt]=value - Greater than (and lt, gte, lte)
filter[field][like]=pattern - Pattern matching
sort=field - Ascending sort
sort=-field - Descending sort
page[offset]=0&page[limit]=20 - Pagination

Implementation approach:

Parse JSON:API filter/sort parameters
Translate to backend-specific query syntax where possible
Fall back to in-memory filtering/sorting when backends don’t support it

7. GraphQL Endpoint

Offer GraphQL as an alternative to REST for clients needing flexible queries:

Sparse fieldsets (only get what you need):

query {
  declaration(id: "DECL001") {
    id
    status
    totalDuty
  }
}

Resource traversal (nested data in one request):

query {
  declaration(id: "DECL001") {
    id
    status
    trader {
      name
      traderId
    }
    lineItems {
      productName
      quantity
      dutyAmount
    }
  }
}

Record filtering and sorting:

query {
  declarations(
    filter: { status: "submitted", period: "2024-Q1" }
    sort: { field: "submittedAt", order: DESC }
    first: 10
  ) {
    edges {
      node {
        id
        trader { name }
        totalDuty
      }
    }
  }
}

Implementation approach:

GraphQL schema generated from OAS
Resolver delegates to same Camel orchestration
Field selection maps to sparse fieldsets
Nested queries map to resource traversal

Benefits:

Clients specify exactly what they need
Single request for complex data requirements
Strong typing and introspection

8. Field Level Scopes (Hard Filtering)

Restrict field visibility based on OAuth scopes, preventing sensitive data disclosure:

# Client with scope "declarations:basic" sees:
{ "id": "DECL001", "status": "submitted" }

# Client with scope "declarations:full" sees:
{ "id": "DECL001", "status": "submitted", "totalDuty": 1234.56, "traderId": "TRD001" }

Implementation approach:

Define field-to-scope mappings in configuration
Extract scopes from Kong-validated JWT
Apply hard filter before soft filter (sparse fieldsets)

9. Optimised Backend Requests

Intelligently reduce backend calls based on sparse fieldset selection:

# Request: GET /declarations/DECL001?fields[declaration]=id,status
# Only calls Declaration API (skips Trader/Product APIs since no fields needed)

Implementation approach:

Analyse field selection to determine which backends are needed
Skip unnecessary backend calls
Significant latency and cost reduction for targeted queries

Testing Requirements

Fault Injection via Envoy Proxies

To simulate realistic network conditions in test environments, Envoy proxies in front of backend mocks provide fault injection capabilities:

Capabilities:

Delay injection: Add latency to simulate slow backends
Error injection: Return 5xx errors to test resilience
Percentage-based: Apply faults to a percentage of requests
Per-backend configuration: Different fault profiles per mock

Implementation: Envoy header-controlled fault injection with Domain API pass-through

Architecture:

Client (or test harness) passes fault headers with request
Domain API passes fault headers through to Envoy proxies unchanged
Envoy applies faults based on headers (header_delay, header_abort mode)

POC Simplification: For the POC, fault injection uses direct header pass-through rather than Domain API configuration. This keeps the Domain API simple while enabling per-request fault control for targeted testing. Future phases may add automated fault generation in the Domain API if needed.

Requirements:

Delay range: Configurable per-backend in domain API (default: 0-500ms)
Error rate: Configurable percentage in domain API (default: 5%)
Distribution: Uniform random within configured range
Control: Enable/disable via domain API config (fault-injection.enabled)

Use cases:

Verify domain API handles slow backend responses gracefully
Test timeout behavior and circuit breaker patterns
Validate error handling and retry logic
Ensure orchestration doesn’t assume instant responses
Validate that parallel calls actually provide latency benefits

Domain API configuration (application.yaml):

fault-injection:
  enabled: true
  excise:
    delay: { min: 100, max: 500 }
    error-rate: 5
  customer:
    delay: { min: 50, max: 300 }
    error-rate: 5
  tax-platform:
    delay: { min: 100, max: 400 }
    error-rate: 5

Envoy configuration (header-controlled mode):

# envoy-excise.yaml - fault injection filter
http_filters:
  - name: envoy.filters.http.fault
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.http.fault.v3.HTTPFault
      delay:
        header_delay: {}             # Delay from x-envoy-fault-delay-request header
        percentage:
          numerator: 100             # Apply to 100% of requests with header
          denominator: HUNDRED
      abort:
        header_abort: {}             # Abort from x-envoy-fault-abort-request header
        percentage:
          numerator: 100             # Apply to 100% of requests with header
          denominator: HUNDRED
  - name: envoy.filters.http.router
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

How it works (POC - pass-through mode):

Client/test harness includes fault headers in request:
- x-envoy-fault-delay-request: 500 (to inject 500ms delay)
- x-envoy-fault-abort-request: 503 (to inject 503 error)
Domain API passes these headers through to Envoy proxies unchanged
Envoy applies the fault before forwarding to mock

Future enhancement (automated fault generation):

Domain API generates random delay (e.g., 250ms) based on config
Domain API determines if error should be injected (5% chance)
Domain API adds headers to backend request
Envoy applies the fault before forwarding to mock

Manual testing with headers:

# Inject specific delay
curl -H "x-envoy-fault-delay-request: 500" http://localhost:4010/excise/vpd/registrations/VPD123456
 
# Inject error
curl -H "x-envoy-fault-abort-request: 503" http://localhost:4010/excise/vpd/registrations/VPD123456

Per-backend configuration:

Backend	Default Delay	Error Rate	Notes
excise	100-500ms	5%	Legacy XML system, slower
customer	50-300ms	5%	Fast lookup service
tax-platform	100-400ms	5%	Storage operations

Benefits over application-level injection:

Works with any mock technology (Prism, WireMock, custom)
Simulates network-level issues more realistically
No domain API code changes needed
Centralized configuration in Envoy
Can inject errors that mocks can’t simulate

Load Testing with k6

Purpose: Validate domain API performance under load and measure orchestration latency benefits.

Requirements:

Tool: k6 load testing framework
Scenarios:
- Baseline: 10 VUs (virtual users), 1 minute
- Load test: 50 VUs, 5 minutes
- Spike test: 0→100→0 VUs over 3 minutes
Metrics: Response time (p50, p95, p99), throughput, error rate
With delays: Test both with and without backend delay injection enabled

Example k6 script:

import http from 'k6/http';
import { check } from 'k6';
 
export const options = {
  scenarios: {
    baseline: {
      executor: 'constant-vus',
      vus: 10,
      duration: '1m',
    },
  },
};
 
export default function () {
  const payload = JSON.stringify({
    vpdApprovalNumber: 'VPD123456',
    periodKey: '24A1',
    basicInformation: { /* ... */ },
  });
 
  const res = http.post('http://localhost:8080/duty/vpd/submission-returns/v1', payload, {
    headers: {
      'Content-Type': 'application/json',
      'X-Idempotency-Key': `test-${__VU}-${__ITER}`,
    },
  });
 
  check(res, {
    'status is 201': (r) => r.status === 201,
    'response time < 2s': (r) => r.timings.duration < 2000,
  });
}

Risks

Risk	Impact	Mitigation
Camel/Quarkus learning curve	Medium	Use Camel’s REST DSL which is well-documented; start simple
Kong configuration complexity	Low	Use declarative config (deck) for reproducibility
Idempotency testing	Low	Deferred to integration with real backends; POC uses stateless Prism mocks
Orchestration latency	Low	Sequential calls are required by design; backends are fast (mocked)
Field filter complexity	Medium	Start with flat field filtering; defer nested filtering
Idempotency edge cases	Medium	Comprehensive test scenarios for replay, conflict, and timeout cases
Fault injection testing	Low	Use Envoy’s built-in fault filter for delays and error injection

References

Apache Camel REST DSL: https://camel.apache.org/components/latest/rest-component.html
Kong Enterprise: https://docs.konghq.com/enterprise/
JSON:API Sparse Fieldsets: https://jsonapi.org/format/#fetching-sparse-fieldsets
JSON:API Filtering: https://jsonapi.org/format/#fetching-filtering

Techcle Wiki

Explorer

Spec

0009: Vaping Duty Domain API POC

Summary

Problem Statement

Goals

Non-Goals

Open Design Decisions

Decisions

Decision Rationale: Sparse Fieldsets at Domain API Level

Domain Model: VPD Submission Returns

Domain Overview

Backend Services (Systems of Record)

Orchestration Flows

Orchestration Sequence Diagrams

POST /duty/vpd/submission-returns/v1

GET /duty/vpd/submission-returns/v1?acknowledgementReference=…

GET /duty/vpd/submission-returns/v1?vpdApprovalNumber=…&periodKey=…

Backend API Specifications

excise - Excise Duty System

customer - Customer Master Data

tax-platform - Submissions & Acknowledgements

Domain API Endpoints

Headers

Example Responses

Technical Approach

Key Components

1. Apache Camel Domain Service

2. Kong Enterprise Gateway

3. Idempotency Handling (Header Forwarding Only)

4. Sparse Fieldsets (Soft Filtering)

5. Mock Backend Services

6. Observability Stack (LGTM)

7. Platform OAS Generation

Directory Structure

Success Criteria

Implementation Phases

Phase 0: API Specifications

Phase 1: Foundation

Phase 2: Camel Domain Service

Phase 3: Orchestration

Phase 4: Sparse Fieldsets

Phase 5: Kong Integration

Phase 6: Kubernetes Deployment

Phase 7: Camel YAML DSL Exploration

Test Scenarios

Future Features

1. Full Idempotency Handling

2. ETag/Optimistic Locking

3. “Try It” Capability for HIP

4. Resource Traversal (include parameter)

5. Field Filtering on Traversed Resources

6. Record Filtering and Sorting (JSON:API Syntax)

7. GraphQL Endpoint

8. Field Level Scopes (Hard Filtering)

9. Optimised Backend Requests

Testing Requirements

Fault Injection via Envoy Proxies

Load Testing with k6

Risks

References

Graph View

Table of Contents

4. Resource Traversal (`include` parameter)