gw/backend-testing.md

# Backend Testing

## Overview

The backend uses **pytest** with **pytest-asyncio** for all testing. Tests run against an in-memory SQLite database (via aiosqlite) so no PostgreSQL instance is required. The test suite is split into two layers:

| Layer | Location | Purpose |
|---|---|---|
| Functional | `tests/` | Happy-path and behavioural correctness |
| Security | `tests/security/` | OWASP ASVS control coverage + API Security Top 10 |

Current suite status, verified on **2026-03-30**:
- `122` tests collected
- `122` passed

---

## Prerequisites

Run backend tests from the `backend/` directory so `pytest.ini` is discovered and imports resolve the same way they do in CI:

```bash
cd backend
python -m pip install -r requirements.txt
```

Notes:
- Tests override `app.database.get_db`, so the PostgreSQL `DATABASE_URL` in `.env` is **not** used during test execution
- Required test dependencies already live in `requirements.txt` (`pytest`, `pytest-asyncio`, `pytest-cov`, `httpx`, `aiosqlite`)
- Interactive API docs are disabled by default via `ENABLE_DOCS=false`; set `ENABLE_DOCS=true` locally if you need `/docs` or `/redoc`

---

## Running tests

```bash
cd backend

# All tests
python -m pytest

# Collect test names without executing them
python -m pytest --collect-only -q

# Security tests only
python -m pytest tests/security/

# Functional tests only (exclude security)
python -m pytest tests/ --ignore=tests/security/

# Stop on first failure
python -m pytest -x

# With coverage report
python -m pytest --cov=app --cov-report=term-missing

# Single file
python -m pytest tests/security/test_authn.py -v

# Single test
python -m pytest tests/security/test_authn.py::TestJWTSecurity::test_alg_none_attack_rejected -v

# Name filter within a file/class
python -m pytest tests/security/test_authn.py -k alg_none -v
```

Path-based selection is the reliable split today. `backend/pytest.ini` declares a `security` marker, but the current suite is organized primarily by directory rather than by applied markers.

---

## Pytest configuration

`backend/pytest.ini` currently sets:

| Setting | Value | Effect |
|---|---|---|
| `asyncio_mode` | `auto` | Async tests run without needing per-test event-loop boilerplate |
| `testpaths` | `tests` | Test discovery is rooted under `backend/tests/` |
| `addopts` | `-v --tb=short` | Verbose test names and shortened tracebacks by default |
| `markers` | `security` | Reserved marker for hardening/security tests |

---

## Test infrastructure

### `tests/conftest.py`

Shared fixtures available to all tests:

| Fixture | Scope | Description |
|---|---|---|
| `setup_database` | session | Creates all ORM tables once at the start of the session |
| `clean_tables` | function (autouse) | Deletes all rows between each test |
| `reset_rate_limiter` | function (autouse) | Clears slowapi's in-memory counters — prevents 429 cascades |
| `client` | function | `httpx.AsyncClient` wired to the FastAPI app via ASGITransport |
| `admin_user` | function | Creates a live `User` row in the test DB |
| `admin_token` | function | Logs in as `admin_user` and returns the Bearer access token |
| `db_session` | function | Yields a live `AsyncSession` for direct DB state manipulation |
| `TestSessionLocal` | module helper | Async sessionmaker imported directly by some tests for manual inserts/queries |

### Database

Tests use `sqlite+aiosqlite:///:memory:` — a fresh in-memory SQLite instance for the entire pytest session. Tables are created once (`setup_database`), and every row is deleted between tests (`clean_tables`). This makes each test fully isolated without the overhead of dropping and recreating schema.

The app's normal `get_db()` dependency is overridden inside the `client` fixture, so HTTP requests made during tests always talk to SQLite even if local development is configured for PostgreSQL.

### Rate limiter reset

slowapi stores rate-limit counters in memory. Without resetting them, the 5/min limit on `/auth/login` exhausts after 5 tests, causing every subsequent `admin_token` fixture to fail with 429. The `reset_rate_limiter` autouse fixture calls `limiter._storage.reset()` before each test.

---

## Functional test suite

Functional tests live directly under `tests/` and currently cover the main happy-path API behavior:

| File | Coverage |
|---|---|
| `test_auth.py` | Login, invalid credentials, refresh-token rotation |
| `test_pages.py` | Page CRUD, public visibility rules, auth requirements |
| `test_analytics.py` | Authenticated analytics summary aggregation |
| `test_analytics_ingest.py` | Public analytics ingest, anon cookie creation, metadata filtering |

---

## Security test suite

Security tests live in `tests/security/` and are organised by OWASP control domain. Each test references the relevant **ASVS requirement** and **API Security Top 10 category** in its docstring.

### OWASP alignment matrix

| File | ASVS chapters | API Top 10 categories |
|---|---|---|
| `test_authn.py` | V2 Authentication, V3 Session Management | API2 Broken Authentication |
| `test_authz.py` | V4 Access Control | API1 BOLA, API3 Object Property Auth, API5 Function Level Auth |
| `test_input.py` | V5 Input Validation & Sanitization | API4 Resource Consumption, API8 Security Misconfiguration |
| `test_config.py` | V9 Communication, V14 Configuration | API8 Security Misconfiguration, API9 Improper Inventory |
| `test_rate_limit.py` | V13 API & Web Service | API4 Resource Consumption, API6 Business Flow, API7 SSRF |

---

### `test_authn.py` — Authentication

**ASVS V2.1 Credential validation**
- Wrong password → 401
- Unknown email → 401 (same response as wrong password; prevents user enumeration)
- Empty / null password → 401 or 422
- Missing fields → 422
- Unicode password → 401 (no crash)
- 1 000-char password → 401 (bcrypt `ValueError` is caught and treated as invalid credentials)
- Inactive account → 401

**ASVS V3.5 JWT security**
- No Authorization header → 401/403
- Garbage Bearer value → 401/403
- JWT signed with wrong key → 401/403
- Expired JWT → 401/403
- `alg: none` attack (unsigned token) → 401/403
- Tampered payload with valid signature → 401/403
- Token in query string → 401/403 (URL leaks tokens to logs and Referer)
- HTTP Basic Auth scheme → 401/403

**ASVS V3.5 Refresh token rotation**
- Token is rotated; old token is revoked and returns 401 on reuse
- Forged refresh token → 401
- Empty refresh token → 401/422
- New access token issued after rotation is accepted

---

### `test_authz.py` — Authorization

**ASVS V4.1 | API5 Unauthenticated write access**
- Parametrized over all 8 write/protected endpoints
- Verifies 401/403 for: `POST /pages`, `PUT /pages/:slug`, `DELETE /pages/:slug`, `POST /posts`, `PUT /posts/:slug`, `DELETE /posts/:slug`, `PUT /settings`, `GET /analytics/summary`

**ASVS V4.2 | API1 Object-level authorization (BOLA)**
- Non-existent slug → 404 (not 403, not 500)
- Path-traversal slugs (`../admin`, `%2e%2e/secret`) → 404/422
- DELETE/PUT on absent resource → 404

**ASVS V4.3 | API3 Mass assignment**
- Injected fields (`is_admin`, `hashed_password`, `id`, `internal_field`) are stripped from create and update requests
- Caller-supplied `id` is ignored; DB assigns it

---

### `test_input.py` — Input validation and sanitization

**ASVS V5.1 Schema validation**
- Required fields enforced on pages and posts
- Null required fields → 422
- `event_type`, `page`, `element` field max-lengths enforced
- Malformed JSON body → 422 (not 500)

**ASVS V5.2 | API8 HTML sanitization (nh3/ammonia)**
All vectors are tested at create and update time:
- `<script>alert()</script>` — stripped
- `onerror="..."`, `onclick="..."` event attributes — stripped
- `href="javascript:..."` — stripped
- `<iframe>` — stripped
- `<object>` — stripped
- Safe HTML (`<p>`, `<strong>`, `<a href>`, `<ul>`) — preserved

**ASVS V5.3 SQL injection**
- Parametrized over 5 SQL injection patterns in slug path parameter → 404, not 500
- Parametrized over 4 injection patterns in login email field → 401, not 200 or 500

**Analytics metadata sanitization**
- Unknown keys dropped silently
- Nested objects dropped
- `__proto__` / `constructor` pollution keys rejected by allowlist
- Oversized string values truncated to 120 chars
- Null metadata accepted
- 200-key object with mostly unknown keys → not 500

---

### `test_config.py` — Security configuration

**ASVS V14.4 | API8 HTTP security headers**

| Header | ASVS | Reason not set |
|---|---|---|
| `X-Content-Type-Options: nosniff` | 14.4.3 | Set by `SecurityHeadersMiddleware` |
| `X-Frame-Options: DENY` | 14.4.5 | Set by `SecurityHeadersMiddleware` |
| `Content-Security-Policy` | 14.4.6 | Set by `SecurityHeadersMiddleware` |
| `Strict-Transport-Security` | 9.2.2 | Set by `SecurityHeadersMiddleware` |
| `Referrer-Policy` | 14.4.4 | Set by `SecurityHeadersMiddleware` |

Implemented in `app/main.py`:
```python
from starlette.middleware.base import BaseHTTPMiddleware

class SecurityHeadersMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request, call_next):
        response = await call_next(request)
        response.headers["X-Content-Type-Options"] = "nosniff"
        response.headers["X-Frame-Options"] = "DENY"
        response.headers["Referrer-Policy"] = "strict-origin-when-cross-origin"
        return response

app.add_middleware(SecurityHeadersMiddleware)
```

**ASVS V14.3 Error handling**
- 404 on unknown route → no stack trace, no file paths, no `site-packages`
- 422 validation error → Pydantic JSON envelope only
- 401 response → `WWW-Authenticate: Bearer` header present (RFC 7235)
- Malformed JSON body → 422 not 500
- Wrong Content-Type → 415/422 not 500

**ASVS V14.5 CORS**
- `http://localhost:5173` → ACAO header echoed
- `https://www.goodwalk.co.nz` → ACAO header echoed
- `https://evil.example.com` → no ACAO header, not `*`

**API9 Inventory**
- `/health` → `{"status": "ok"}`
- `/openapi.json` → accessible (documented exposure; restrict in production)
- `/docs` and `/redoc` → disabled by default (`ENABLE_DOCS=false`)

---

### `test_rate_limit.py` — Rate limiting and SSRF

**ASVS V13.2 | API4 Rate limit headers**

slowapi is configured with `headers_enabled=True`, so rate-limited endpoints emit `X-RateLimit-*` response headers and clients can back off gracefully.

Current configuration in `app/middleware/rate_limit.py`:
```python
limiter = Limiter(key_func=get_remote_address, headers_enabled=True)
```

**API4 Payload size**
- Analytics `page` field > 255 chars → 422
- 100 KB `body` field → not 500
- Deeply nested JSON (50 levels) → not 500

**API7 SSRF — Private IP suppression**
Parametrized over 8 private/loopback IPs in `X-Forwarded-For`:
`127.0.0.1`, `10.0.0.1`, `10.255.255.255`, `192.168.1.100`, `172.16.0.1`, `172.31.255.255`, `::1`, `localhost`
→ event recorded (201); geo-lookup silently skipped (no outbound request to ip-api.com)

Parametrized over 5 malformed XFF values → not 500

**API6 Business flow**
- Analytics ingest is intentionally unauthenticated on both `/api/web/event` and the legacy alias `/api/analytics/event`
- Analytics summary requires authentication
- Session cookie is `HttpOnly`
- Session cookie is `SameSite=Lax`

---

## Coverage

Run with coverage to see which app paths are exercised:

```bash
python -m pytest --cov=app --cov-report=term-missing --cov-report=html
# Open htmlcov/index.html in a browser
```

Focus areas for future test expansion:
- `app/routers/sections.py` — legacy CMS endpoints not yet covered by security tests
- `app/services/analytics.py` — geo-lookup mock / retry behaviour
- Concurrency: two simultaneous refresh token rotations (race condition check)

---

## Troubleshooting

Common issues when running the suite locally:

| Symptom | Likely cause | What to do |
|---|---|---|
| `ModuleNotFoundError` / wrong rootdir | Running pytest outside `backend/` | `cd backend` first, then rerun |
| Login-related tests start returning `429` | Rate-limit state leaked between tests | Confirm `tests/conftest.py` is being loaded and `reset_rate_limiter` is still autouse |
| `PytestCacheWarning` about `.pytest_cache` on Windows | Local permission issue writing pytest cache files | Usually safe to ignore; test results are still valid |