Acceptance Testing Guide

Overview

Acceptance tests for the Domain API POC are maintained in separate test projects within this repository at tests/acceptance/:

UI tests (tests/acceptance/ui/) - Playwright tests for the documentation site and API explorer
Domain API tests (tests/acceptance/domain-api/) - Vitest tests for the VPD Domain API

These tests support both docker-compose and Kubernetes deployments, allowing validation in different environments.

Quick Start

Install Dependencies

From the main project directory:

# Install acceptance test dependencies
task test:acceptance:install

This will:

Install Playwright and dependencies in tests/acceptance/
Install Playwright browsers

Run Tests

# Run all acceptance tests (UI and Domain API)
task test:acceptance
 
# UI tests (documentation and explorer)
task test:acceptance:ui:all          # Run all UI tests
task test:acceptance:headed          # Run in headed mode (see browser)
task test:acceptance:ui              # Run in UI mode (interactive)
task test:acceptance:report          # View test report
 
# Domain API tests
task test:acceptance:domain-api      # Run against docker-compose (default)
task test:acceptance:domain-api:k8s  # Run against Kubernetes deployment
 
# Slow tests (Swagger UI rendering)
task test:acceptance:skipped         # Run slow tests
task test:acceptance:skipped:headed  # Run slow tests in headed mode

Project Structure

domain-apis/
├── tests/
│   └── acceptance/
│       ├── ui/                           # Playwright UI tests
│       │   ├── tests/
│       │   │   ├── api-explorer.spec.ts      # API explorer tests
│       │   │   └── documentation.spec.ts     # Documentation site tests
│       │   ├── playwright.config.ts          # Playwright configuration
│       │   ├── package.json                  # UI test dependencies
│       │   └── README.md                     # UI test documentation
│       └── domain-api/                   # Vitest Domain API tests
│           ├── domain-api.spec.ts            # Domain API orchestration tests
│           ├── vitest.config.ts              # Vitest configuration
│           └── package.json                  # Domain API test dependencies
├── docs/                        # Documentation site
│   ├── index.html              # Homepage
│   ├── explorer.html           # API explorer
│   └── ...
└── package.json                # Main project dependencies

Test Coverage

API Explorer Tests (`tests/api-explorer.spec.ts`)

Validates the interactive API explorer functionality:

✅ Page loads correctly with Swagger UI
✅ Default API specification loads (Taxpayer API)
✅ API selection dropdown works
✅ Switching between API specifications
✅ Endpoints are displayed
✅ Endpoint details can be expanded
✅ “Try it out” functionality is available
✅ Response schemas are displayed
✅ Navigation back to documentation works
✅ URL parameters preserve API selection
✅ Usage instructions are displayed

Requirements validated: 7.1, 8.5, 9.1, 9.2, 9.3, 9.4

Documentation Site Tests (`tests/documentation.spec.ts`)

Validates the documentation site functionality:

✅ Homepage loads with correct branding
✅ GOV.UK styled header and phase banner
✅ All three API cards are displayed
✅ Each API card has correct information
✅ Navigation to API explorer works
✅ Navigation to API documentation works
✅ Additional resources section is present
✅ Key features section is present
✅ Footer is displayed
✅ GOV.UK styling is consistent
✅ Responsive design on mobile
✅ Navigation between all API docs

Requirements validated: 7.1, 7.2, 9.1, 9.5, 9.6

Domain API Tests (`tests/acceptance/domain-api/domain-api.spec.ts`)

Validates the VPD Domain API orchestration and functionality:

✅ Health check endpoint
✅ GET by acknowledgement reference (orchestration: tax-platform → excise → customer)
✅ GET by approval number + period key (orchestration: excise validation → tax-platform → customer)
✅ POST submission (orchestration: excise validate → tax-platform store → customer enrich)
✅ Trader details from customer service (orchestrated)
✅ Registration details from excise service (XML transformed to JSON)
✅ Period details from excise service (XML transformed to JSON)
✅ Correlation ID propagation
✅ Fault injection (delay and abort headers)
✅ Error handling (400 responses)
✅ Sparse fieldsets (soft filtering)
- Single and multiple field requests
- Nested field filtering with dot notation
- Invalid field validation
- Mixing top-level and nested fields
✅ CORS support (preflight and headers)

Environment Support:

Docker Compose: Tests against http://localhost:8081 (default)
Kubernetes: Tests against https://vpd-domain-api.lab.local.ctoaas.co (via TEST_ENV=k8s)

Environment Variables:

DOMAIN_API_URL: Override default URL
TEST_ENV: Set to k8s or docker to force environment

Running Tests Locally

Prerequisites

Node.js 18+
npm or yarn

Setup

Install main project dependencies (if not already done):
```
task install
```
Install acceptance test dependencies:
```
task test:acceptance:install
```

Run Tests

UI Tests

The UI tests will automatically start the documentation server before running:

# Run all UI tests (fast tests only)
task test:acceptance:ui:all
 
# Run tests in headed mode (see browser)
task test:acceptance:headed
 
# Run tests in UI mode (interactive debugging)
task test:acceptance:ui
 
# Run only the slow tests (Swagger UI rendering)
task test:acceptance:skipped
 
# Run slow tests in headed mode (recommended to see what's happening)
task test:acceptance:skipped:headed
 
# View test report
task test:acceptance:report

Domain API Tests

The Domain API tests require either docker-compose or Kubernetes to be running:

# Docker Compose (default)
docker-compose up -d
task test:acceptance:domain-api
 
# Kubernetes
task test:acceptance:domain-api:k8s
 
# Custom URL
DOMAIN_API_URL=https://custom-domain.example.com task test:acceptance:domain-api

About Slow Tests: Some tests depend on Swagger UI fully rendering from CDN, which can be slow and inconsistent. These tests are tagged with @slow and excluded by default. Run them separately with task test:acceptance:skipped when you want to validate Swagger UI functionality.

Manual Server Start

If you prefer to start the server manually:

# Terminal 1: Start documentation server using docker-compose
docker-compose up docs
 
# Terminal 2: Run tests
cd tests/acceptance
npm test

Detailed Test Documentation

For detailed information about the acceptance tests, including:

Configuration options
Writing new tests
Debugging tests
CI/CD integration
Troubleshooting

See the Acceptance Tests README.

Test Configuration

The tests are configured in tests/acceptance/playwright.config.ts:

Base URL: http://localhost:8080 (configurable via BASE_URL env var)
Browsers: Chromium, Firefox, WebKit
Parallel execution: Enabled (except on CI)
Retries: 2 on CI, 0 locally
Screenshots: On failure only
Videos: Retained on failure
Traces: On first retry

Automatic Server Start

The configuration includes a webServer section that automatically starts the documentation server using docker-compose:

webServer: {
  command: 'docker-compose up -d docs && echo "Waiting for docs server..." && sleep 2',
  url: 'http://localhost:8080',
  reuseExistingServer: !process.env.CI,
}

This starts the docs service from docker-compose.yml, which uses a lightweight static file server to serve the documentation on port 8080. You don’t need to manually start the server before running tests.

CI/CD Integration

GitHub Actions Example

name: Acceptance Tests
 
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
 
jobs:
  acceptance-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '18'
      
      - name: Install main project dependencies
        run: npm ci
      
      - name: Install acceptance test dependencies
        working-directory: tests/acceptance
        run: npm ci
      
      - name: Install Playwright browsers
        working-directory: tests/acceptance
        run: npx playwright install --with-deps
      
      - name: Run acceptance tests
        run: task test:acceptance
        env:
          CI: true
      
      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-report
          path: tests/acceptance/playwright-report/

Benefits of In-Repo Separate Project

Independent Dependencies: Acceptance tests have their own package.json and dependencies
Version Control: Tests are versioned with the code they validate
Easy Discovery: Tests are part of the main repository
Isolated Execution: Tests run in their own context
Clear Separation: Testing concerns are separated from implementation
Flexible Testing: Can test against different environments

Debugging Tests

Visual Debugging

# Run in headed mode to see the browser
npm run test:acceptance:headed
 
# Run in UI mode for interactive debugging
npm run test:acceptance:ui

From Acceptance Test Directory

cd tests/acceptance
 
# Run specific test file
npm test tests/api-explorer.spec.ts
 
# Run in debug mode
npm run test:debug
 
# Generate test code
npm run codegen

Success Criteria

Acceptance tests are the final validation before considering work complete. All acceptance tests must pass before:

Merging pull requests
Deploying to production
Marking features as complete

Remember: Unit tests + Integration tests + Acceptance tests = Complete validation

Troubleshooting

Tests fail with “baseURL not accessible”

Ensure the documentation server can start:

npm run serve:docs

If port 8080 is in use, set a different port:

BASE_URL=http://localhost:3000 npm run test:acceptance

Browser installation issues

Install browsers manually:

cd tests/acceptance
npx playwright install --with-deps

Tests are flaky

Check network conditions
Increase timeouts in playwright.config.ts
Use proper waits instead of fixed timeouts
Run tests in headed mode to observe behavior

Next Steps

✅ Acceptance test project created at tests/acceptance/
✅ API explorer tests implemented
✅ Documentation site tests implemented
⏳ Set up CI/CD pipeline for automated testing
⏳ Add more test scenarios as features are added

Techcle Wiki

Explorer

Acceptance Testing

Acceptance Testing Guide

Overview

Quick Start

Install Dependencies

Run Tests

Project Structure

Test Coverage

API Explorer Tests (`tests/api-explorer.spec.ts`)

Documentation Site Tests (`tests/documentation.spec.ts`)

Domain API Tests (`tests/acceptance/domain-api/domain-api.spec.ts`)

Running Tests Locally

Prerequisites

Setup

Run Tests

UI Tests

Domain API Tests

Manual Server Start

Detailed Test Documentation

Test Configuration

Automatic Server Start

CI/CD Integration

GitHub Actions Example

Benefits of In-Repo Separate Project

Debugging Tests

Visual Debugging

From Acceptance Test Directory

Success Criteria

Troubleshooting

Tests fail with “baseURL not accessible”

Browser installation issues

Tests are flaky

Next Steps

Graph View

Table of Contents

Techcle Wiki

Explorer

Acceptance Testing

Acceptance Testing Guide

Overview

Quick Start

Install Dependencies

Run Tests

Project Structure

Test Coverage

API Explorer Tests (tests/api-explorer.spec.ts)

Documentation Site Tests (tests/documentation.spec.ts)

Domain API Tests (tests/acceptance/domain-api/domain-api.spec.ts)

Running Tests Locally

Prerequisites

Setup

Run Tests

UI Tests

Domain API Tests

Manual Server Start

Detailed Test Documentation

Test Configuration

Automatic Server Start

CI/CD Integration

GitHub Actions Example

Benefits of In-Repo Separate Project

Debugging Tests

Visual Debugging

From Acceptance Test Directory

Success Criteria

Troubleshooting

Tests fail with “baseURL not accessible”

Browser installation issues

Tests are flaky

Next Steps

Graph View

Table of Contents

API Explorer Tests (`tests/api-explorer.spec.ts`)

Documentation Site Tests (`tests/documentation.spec.ts`)

Domain API Tests (`tests/acceptance/domain-api/domain-api.spec.ts`)