Domain APIs Repository Restructure

Status: Refined draft v4 Date: 2026-01-30 Updated: Kamelet routing bug investigation complete - intermittent under rapid load, workaround acceptable

This document captures observations from the Spec 0009 POC and proposes structural improvements for the domain-apis repository.

1. Repository Structure

Current State

specs/
└── vaping-duty/
    ├── domain/
    │   ├── fragments/
    │   ├── platform/
    │   │   ├── routes/
    │   │   ├── lib/
    │   │   └── vpd-submission-returns-api.yaml
    │   ├── producer/
    │   └── tools/
    └── mocks/
        ├── customer-api.yaml
        ├── excise-api.yaml
        ├── tax-platform-api.yaml
        └── wiremock/
            └── mappings/

Issues

Fragments are global, not VPD-specific - Platform standards (headers, error responses, common schemas) shared across ALL domains.
Mocks represent backends, not domains - customer-api, excise-api, tax-platform-api are external systems, not part of VPD.
Wiremock is an implementation detail - Only excise uses WireMock (for XML transformation - Prism doesn’t handle XML). Customer and tax-platform use Prism (OAS-driven JSON mocks).
Platform shared code inside domain - lib/SparseFieldsets.groovy and generic Camel code are reusable across all domains.
Unnecessary nesting - specs/ wrapper adds no value.

Proposed Structure

fragments/                          # GLOBAL OAS fragments
└── v1/
    ├── headers.yaml
    ├── parameters.yaml
    ├── responses.yaml
    └── schemas.yaml

platform/                           # SHARED Camel infrastructure
├── lib/
│   ├── SparseFieldsets.groovy
│   └── BackendClient.java          # Java processor example
└── camel/                          # Shared Camel definitions (not "routes" - OAS term)
    └── common.yaml                 # prepareGetRequest, prepareXmlGetRequest,
                                    # applySparseFieldsets, assembleGetResponse, health

domains/                            # Domain APIs
└── vaping-duty/                    # Reference implementation
    ├── fragments/                  # Domain-specific OAS fragments (if needed)
    │   └── v1/
    ├── rest-config.yaml            # REST config + endpoint definitions only
    ├── router.yaml                 # Entry point routing (getSubmission)
    ├── excise-parsers.yaml         # parseExciseRegistrationXml, parseExciseValidationXml, etc.
    ├── routes/
    │   ├── get-by-ack.yaml
    │   ├── get-by-approval.yaml
    │   └── post-submission.yaml
    ├── platform-oas.yaml           # Platform OAS
    └── producer-oas.yaml           # Producer OAS

backends/                           # External system specs/mocks
├── excise-api/
│   ├── oas.yaml
│   └── wiremock/                   # WireMock for XML (Prism doesn't support XML)
│       └── mappings/
├── customer-api.yaml               # OAS only - Prism handles JSON mocks
└── tax-platform-api.yaml           # OAS only - Prism handles JSON mocks

Key changes:

Removed specs/ wrapper - subfolders promoted to root
Domain-specific fragments live inside each domain (e.g., domains/vaping-duty/fragments/)
Renamed platform/routes/ to platform/camel/ - “routes” has OAS meaning
VPD is the reference implementation - no separate example needed
Global fragments + domain-specific fragments (both patterns supported)

2. Route File Issues

rest-config.yaml Contains Route Logic

Current: 105 lines mixing REST configuration AND getSubmission route.

Proposed: Split into:

rest-config.yaml - Just restConfiguration and rest blocks (~38 lines)
router.yaml - The getSubmission entry point route

common.yaml Mixes Generic and VPD-Specific

Current: 464 lines mixing platform patterns with domain-specific code.

Generic (move to platform/camel/common.yaml):

extractStandardHeaders
injectResponseHeaders
prepareGetRequest
preparePostRequest
prepareXmlGetRequest - generic pattern for XML backends
preparePostXmlRequest - generic pattern for XML backends
applySparseFieldsets
assembleGetResponse - generic response assembly pattern
health

VPD-specific (stay in domains/vaping-duty/excise-parsers.yaml):

parseExciseRegistrationXml
parseExciseValidationXml
parseExcisePeriodXml

3. Missing Phase 7c.4 Deliverables

Plan Specified Three Mechanisms

The plan stated: “Each reusable component uses a different mechanism to provide practical experience with all three options”

Mechanism	File	Purpose	Status
A. Groovy Script	`lib/SparseFieldsets.groovy`	Complex JSON transformation	Done
B. Java Processor	`lib/BackendClient.java`	Backend client pattern	Required
C. Kamelets	Wrapping A & B	Abstraction layer	Required (was supposed to be first)

Priority: Kamelets Spike

The plan (7c.1) specified validating Kamelets with JBang first, before implementing other patterns. This spike was skipped.

Action: Complete the Kamelets spike:

Create simple test Kamelet
Verify it loads with JBang
Test parameter resolution from exchange properties
Document findings (works/doesn’t work, limitations)

Required: BackendClient.java

The plan included a full Java example demonstrating:

URL template resolution from exchange properties
Header propagation (correlation ID, etc.)
HTTP call execution via ProducerTemplate
Response storage in named properties
Field extraction into additional properties
Merging into shared response object

Action: Implement this to complete the POC evaluation of Java vs Groovy vs YAML approaches.

Question: Are —dep Annotations Needed?

The plan showed:

//DEPS com.fasterxml.jackson.core:jackson-databind:2.17.0
//DEPS org.apache.camel:camel-core:4.4.0
//DEPS org.apache.camel:camel-http:4.4.0

With camel-jbang, standard Camel dependencies are auto-resolved. The argocd-eda example didn’t need explicit --dep annotations.

Action: Test with JBang to confirm which (if any) deps are actually required.

4. Documentation Structure

Current State (Scattered)

docs/
├── CAMEL_YAML_EVALUATION.md     # Platform
├── INTEGRATION_TEMPLATE.md      # Platform
├── REUSE_PATTERNS.md            # Platform
├── acceptance-testing.md        # Mixed audience
├── api-consumer-guide.md        # Consumer
├── api-producer-guide.md        # Producer
├── getting-started.md           # Unclear
├── mock-servers.md              # Mixed
├── vpd-testing.md               # VPD-specific (wrong location)
└── index.html                   # Existing entry point

Proposed: Audience-Based Structure

Three audiences:

API Consumers - Developers calling the APIs
API Producers - Teams building domain APIs
Platform Team - Maintaining the infrastructure

docs/
├── index.html                      # Existing - add audience sections to 'Additional Resources'
│
├── patterns/                       # Implementation patterns (what we follow)
│   ├── sparse-fieldsets.md         # Consumer + Producer
│   ├── error-handling.md           # Consumer + Producer
│   └── route-patterns.md           # Platform
│
├── how-to/                         # Task guides (how to do things)
│   ├── creating-a-domain.md        # Producer + Platform (was INTEGRATION_TEMPLATE)
│   ├── testing.md                  # Producer + Platform
│   └── mock-servers.md             # Producer + Platform
│
└── decisions/                      # Architecture decision records
    └── camel-yaml-evaluation.md    # Platform (was CAMEL_YAML_EVALUATION)

Domain-specific docs (like vpd-testing.md) move into domains/vaping-duty/docs/ with a link from index.html.

5. Library Structure (lib/)

Current State (Post-POC)

The lib/ folder has been restructured into granular, function-focused micro-libraries:

lib/transformers/
├── common/
│   └── src/MockExchange.groovy      # Shared test utility (duck-typed Camel Exchange)
├── body-utils/
│   ├── src/BodyUtils.groovy
│   ├── test/BodyUtilsTest.groovy
│   ├── build.gradle
│   └── settings.gradle
├── store-request-builder/
├── validation-error-builder/
├── submission-return-response-builder/
├── sparse-fieldsets/
├── Taskfile.yaml                    # Discovers and tests all transformers
└── .gitignore

Characteristics:

Each transformer is self-contained with its own tests
Uses Gradle for Groovy compilation (Docker: gradle:8.11-jdk21)
Shared test utilities in common/ (mounted read-only)
Taskfile auto-discovers all transformers for testing

GitHub CI dynamically finds all Gradle projects with test/ directories

Question: Where Does BackendClient.java Belong?

A BackendClient.java exists in spikes/ implementing:

URL template resolution from exchange properties
Header propagation (correlation ID, etc.)
HTTP call execution via ProducerTemplate
Response storage in named property

Option A: lib/transformers/backend-client/

Consistent with existing structure
But it’s not a “transformer” - it doesn’t transform data

Option B: lib/clients/backend-client/

Separate category for HTTP client utilities
Clearer semantic distinction from data transformers
Could house other client patterns (retry, circuit breaker, etc.)

Option C: Don’t create it at all

Kamelets already provide declarative backend calls
The customer-getCustomer.kamelet.yaml pattern works
Java processor adds complexity without clear benefit

Question: Is BackendClient the Right Java Example?

BackendClient demonstrates Java can be used, but not why you’d choose it. Better examples might be:

Scenario	Why Java?	Example
Complex parsing	Type safety, libraries	XML Schema validation with JAXB
Cryptography	Security-critical code	Signature verification, HMAC
External SDK	Vendor provides Java SDK	AWS SDK, cloud provider APIs
Performance-critical	JIT optimization	High-throughput transformations
Legacy integration	Existing Java code	Calling internal Java libraries

Question: Should we find a scenario where Java is genuinely the right choice, rather than forcing BackendClient?

Question: What Would Use BackendClient.java?

Current approach (Kamelets):

# Declarative, self-documenting
- kamelet:
    name: customer-getCustomer
    parameters:
      customerId: "${exchangeProperty.customerId}"

BackendClient.java approach:

# Imperative, requires understanding Java class
- bean:
    ref: "#class:BackendClient"
    method: "call"
  properties:
    urlTemplate: "http://customer-api:4011/customers/{customerId}"
    responseProperty: "customerResponse"

Trade-offs:

Aspect	Kamelets	Java BackendClient
Learning curve	Low (YAML config)	Higher (Java + Camel APIs)
Debugging	Harder (abstraction)	Easier (step through code)
Reusability	Per-endpoint Kamelet	Single generic class
Testing	Integration only	Unit testable
JBang compatibility	✅ Works (with routing bug workaround)	❓ Needs verification
IDE support	Limited (YAML)	Full (Java)

Question: What Is The Scope?

If we implement BackendClient.java, what should it cover?

Minimal scope (current spike):

URL template resolution
Header propagation
Response storage

Extended scope (from original plan):

Field extraction into additional properties
Merging into shared response object
Error handling and status code checking
Retry logic
Circuit breaker pattern

Question: Is extended scope needed, or do Camel’s built-in EIPs handle this better?

Findings: Kamelets Spike (Completed)

The Kamelets spike is now complete. Key findings:

Kamelets work with JBang - Load correctly, parameters resolve from exchange properties
Routing bug discovered - When the same Kamelet is used by multiple routes, control can jump to wrong route after Kamelet completes

⚠️ Kamelet Routing Bug - Investigation Complete

Problem: The current “workaround” (duplicate Kamelets per route) defeats the entire purpose of Kamelets as reusable components. If we need customer-getCustomer.kamelet.yaml AND customer-getCustomer-post.kamelet.yaml, we’ve just moved duplication from routes into Kamelets.

Investigation Results (2026-01-30):

A spike in spikes/kamelet-routing-bug/ tested multiple scenarios:

Simple shared Kamelet - PASSED (no bug reproduced)
HTTP-calling Kamelets - PARTIALLY FAILED

Test results from multi-kamelet-routes.yaml:

Test 1: Single call Multi-A ✅
Test 2: Single call Multi-B ✅
Test 3: Interleaved calls (A, B, A, B) ✅ all correct
Test 4: Rapid sequential calls - 1 FAILURE in 10 rounds
  Round 2: A=❌ B=✅  (Route A got wrong response)

Key Findings:

The bug IS real but intermittent
Occurs under rapid sequential load, not single requests
Pattern suggests exchange property leakage or HTTP component state not properly isolated
The workaround (duplicate Kamelets) works because separate Kamelet definitions create separate internal state

Root Cause Hypothesis: Not a fundamental Kamelet design issue. Likely one of:

Race condition in exchange property handling when multiple requests hit same Kamelet
HTTP component caching/pooling causing cross-request state leakage
JBang-specific behavior in how Kamelet templates are instantiated

Options:

A) Report upstream - File issue with Apache Camel, include spike reproduction

B) Use route-specific Kamelets - Accept duplication cost (current workaround)
C) Avoid rapid concurrent calls - Not realistic for production
D) Test in Quarkus - May not have same issue outside JBang

Recommendation: Use workaround (B) for now, file upstream issue (A), and verify behavior when moving to Quarkus deployment (D). The Kamelet pattern is still valuable for organizing code even if full reuse is limited.

Files for reproduction: spikes/kamelet-routing-bug/

Recommendation

Decision needed: Given that Kamelets work (with workaround), is BackendClient.java still needed?

Arguments for:

Demonstrates Java processor pattern for POC completeness
Provides alternative for teams preferring imperative style
Easier to unit test

Arguments against:

Kamelets already solve the problem
Adds maintenance burden (two patterns to support)
Java requires more Camel knowledge

Proposed: Document BackendClient.java as “alternative pattern” in spikes/, don’t promote to lib/. Focus on Kamelets as the primary pattern.

6. Java Implementation Question

Background

The original spec (Goal 7) stated:

“Camel YAML DSL Exploration - Validate whether orchestration can be defined in YAML instead of Java code”

The intended approach was:

Phase 2: Build with Java/Quarkus (the baseline)
Phase 7: Create YAML DSL equivalent to compare

The implementation jumped straight to YAML DSL without the Java baseline.

Current State

BackendClient.java exists in spikes/ as a working example
Kamelets provide an alternative declarative approach that works
No full Java route implementation exists for comparison

Decision Needed

POC Success Criteria

For Camel YAML DSL to be confirmed as suitable, we need to demonstrate:

Criterion	Status	Evidence
YAML routes work	✅ Done	40 acceptance tests passing
Can call external APIs	✅ Done	Excise, Customer, Tax-Platform backends
Can transform data	✅ Done	Groovy transformers with tests
Can extend with Groovy	✅ Done	5 transformers in `lib/transformers/`
Can extend with Java	⚠️ Partial	`BackendClient.java` exists but not integrated into working route
Reusable components	⚠️ Limited	Kamelets work but have intermittent bug under load - use route-specific copies as workaround

What’s Still Needed

1. Java integration proof (Required)

We have Groovy working end-to-end in routes. We need equivalent proof for Java:

A Java processor called from a YAML route
Actually exercised by acceptance tests
Not just existing in spikes/

2. Kamelet bug resolution (Required)

The routing bug makes Kamelets unsuitable for multi-route reuse. Options:

Fix the bug (investigate root cause)
Find alternative reuse pattern
Accept YAML DSL has this limitation

3. Full Java route (Optional)

A complete route in Java for direct comparison would be nice-to-have but not essential if we can prove Java processors work within YAML routes.

Summary of Actions

Current PR (Phase 7c)

#	Action	Priority	Status	Notes
1	Kamelets spike	High	⚠️ PARTIAL	Works, but workaround defeats reusability (see Review Follow-ups #1)
2	lib/ restructure	High	⚠️ PARTIAL	Structure exists but code NOT USED in routes (see Review Follow-ups #2, #7)
3	Taskfile hierarchy	High	⚠️ PARTIAL	Tasks exist but not integrated (see Review Follow-ups #13)
4	GitHub CI for Groovy tests	High	❌ FALSE CLAIM	CI has zero Groovy jobs (see Review Follow-ups #3, #9)
5	Investigate Kamelet routing bug	High	✅ Done	Intermittent under load - workaround acceptable
6	Java integration proof	High	❌ NOT DONE	No Java processor exists (see Review Follow-ups #1)
7	Find better Java example	Medium	✅ Correctly Pending	BackendClient may not be ideal

Code Review Follow-ups (AI) - 2026-01-30

Status: 10 HIGH, 4 MEDIUM, 3 LOW issues found

🔴 HIGH SEVERITY (Must Fix)

#	Issue	File(s)	Solution
1	Action #6 Falsely Marked Done	`_enhancements/restructure.md:477`	Either implement Java processor OR update table to mark as ❌ NOT DONE
2	lib/ Code Not Used in Production	`routes/post-submission-return.yaml:19,29,79,106,144`	DECISION REQUIRED: (A) Use Camel bean refs `bean: {ref: "#class:BodyUtils"}`, OR (B) Remove lib/ entirely and document inline-only approach, OR (C) Wait for JBang classloader fix
3	CI Doesn’t Test Groovy	`.github/workflows/ci.yaml`	Add job: `groovy-tests` with matrix strategy for each transformer dir
4	Kamelet Workaround Defeats Reusability	All routes using Kamelets	DECISION REQUIRED: (A) File Apache Camel bug report, OR (B) Test in Quarkus runtime, OR (C) Accept limitation and document
5	Hardcoded HTTP Not HTTPS	All `*.kamelet.yaml` files	Change `http://` to `https://` for all backend URIs (or document why HTTP is acceptable for POC)
6	No Error Handling in Kamelets	All Kamelets	Add `choice` step checking `${header.CamelHttpResponseCode}` after each HTTP call
7	Empty Sparse-Fieldsets File	`lib/transformers/sparse-fieldsets/src/SparseFieldsets.groovy`	Implement OR remove transformer directory
8	Duplicate Groovy Logic (Not DRY)	`routes/post-submission-return.yaml`	Extract repeated JSON parsing into reusable `direct:` routes OR fix classloader to use lib/
9	Action #4 False Claim (No Matrix)	`_enhancements/restructure.md:475`	Update table to ❌ FALSE until CI job added
10	Acceptance Tests Don’t Verify lib/	`tests/acceptance/`	Cannot fix until #2 resolved (lib code must be used first)

🟡 MEDIUM SEVERITY (Should Fix)

#	Issue	File(s)	Solution
11	Kamelet Docs Incomplete	`docs/api-producer-guide.md`	Add section explaining unique routeId requirement and routing bug workaround
12	No Kamelet Parameter Validation	All Kamelets	Add simple validation checking parameters are not null/empty before URI interpolation
13	Taskfile Not in Build Pipeline	Root `Taskfile.yaml`	Add `vpd:lib:test` to CI pipeline or main build task
14	No Dependency Version Management	All `build.gradle`	Create parent `gradle.properties` with shared version variables

🟢 LOW SEVERITY (Nice to Fix)

#	Issue	File(s)	Solution
15	Inconsistent Logging Levels	All routes	Standardize: DEBUG for entry/exit, INFO for backend calls, WARN for errors
16	No Performance Benchmarks	N/A	Optional: Add k6 script comparing Groovy vs inline performance
17	Spike TODOs for Quarkus	`spikes/kamelet-routing-bug/README.md:61`	Test Kamelet sharing in Quarkus runtime

Key Decisions Needed:

lib/ Strategy (#2): Use beans vs inline vs wait for fix?
Kamelet Bug (#4): File bug vs test Quarkus vs accept limitation?
Java Integration (#1, #6): Required for POC or defer?

Restructure Phase (Separate PR)

#	Action	Priority	Notes
8	Remove `specs/` wrapper	Medium	Promote subfolders to root
9	Move backends to root level	Medium	Separate from domains
10	Move global fragments to root	Medium	Share across domains
11	Extract platform shared code	Medium	lib + camel/common.yaml
12	Split rest-config.yaml	Low	Separate config from routing
13	Split common.yaml (generic vs VPD)	Low	excise-parsers.yaml for VPD
14	Restructure docs	Low	patterns/how-to/decisions

Resolved Questions

Question	Answer
Should `platform/` contain a reference implementation?	No - VPD is the reference
Domain-specific fragments vs global only?	Both - global at root, domain-specific inside domain
How to version platform vs domain code?	Git tree is the current version
Do Kamelets work with JBang?	Yes, with workaround for routing bug (use separate Kamelet per route)
lib/ structure?	`lib/transformers/<name>/` with individual Gradle builds and tests
Kamelet routing bug	Intermittent under rapid load. Use route-specific Kamelets as workaround. File upstream issue. Verify in Quarkus deployment.

Open Questions

Question	Options	Priority	Notes
Java integration proof	Need working example in route	🔴 High	Required for POC completion
Better Java example?	Crypto, SDK integration, parsing?	Medium	BackendClient not compelling
Where should Java code live?	`lib/clients/` vs `lib/processors/` vs `spikes/`	Low	Depends on above
Full Java route?	For comparison with YAML	Low	Nice-to-have

Techcle Wiki

Explorer

Restructure

Domain APIs Repository Restructure

1. Repository Structure

Current State

Issues

Proposed Structure

2. Route File Issues

rest-config.yaml Contains Route Logic

common.yaml Mixes Generic and VPD-Specific

3. Missing Phase 7c.4 Deliverables

Plan Specified Three Mechanisms

Priority: Kamelets Spike

Required: BackendClient.java

Question: Are —dep Annotations Needed?

4. Documentation Structure

Current State (Scattered)

Proposed: Audience-Based Structure

5. Library Structure (lib/)

Current State (Post-POC)

Question: Where Does BackendClient.java Belong?

Question: Is BackendClient the Right Java Example?

Question: What Would Use BackendClient.java?

Question: What Is The Scope?

Findings: Kamelets Spike (Completed)

⚠️ Kamelet Routing Bug - Investigation Complete

Recommendation

6. Java Implementation Question

Background

Current State

Decision Needed

POC Success Criteria

What’s Still Needed

Summary of Actions

Current PR (Phase 7c)

Code Review Follow-ups (AI) - 2026-01-30

🔴 HIGH SEVERITY (Must Fix)

🟡 MEDIUM SEVERITY (Should Fix)

🟢 LOW SEVERITY (Nice to Fix)

Restructure Phase (Separate PR)

Resolved Questions

Open Questions

Graph View

Table of Contents