gw/backend-testing.md

Backend Testing

Overview

The backend uses pytest with pytest-asyncio for all testing. Tests run against an in-memory SQLite database (via aiosqlite) so no PostgreSQL instance is required. The test suite is split into two layers:

Layer	Location	Purpose
Functional	tests/	Happy-path and behavioural correctness
Security	tests/security/	OWASP ASVS control coverage + API Security Top 10

Current suite status, verified on 2026-03-30:

• 122 tests collected
• 122 passed

---

Prerequisites

Run backend tests from the backend/ directory so pytest.ini is discovered and imports resolve the same way they do in CI:

cd backend
python -m pip install -r requirements.txt

Notes:

• Tests override app.database.get_db, so the PostgreSQL DATABASE_URL in .env is not used during test execution
• Required test dependencies already live in requirements.txt (pytest, pytest-asyncio, pytest-cov, httpx, aiosqlite)
• Interactive API docs are disabled by default via ENABLE_DOCS=false; set ENABLE_DOCS=true locally if you need /docs or /redoc

---

Running tests

cd backend

# All tests
python -m pytest

# Collect test names without executing them
python -m pytest --collect-only -q

# Security tests only
python -m pytest tests/security/

# Functional tests only (exclude security)
python -m pytest tests/ --ignore=tests/security/

# Stop on first failure
python -m pytest -x

# With coverage report
python -m pytest --cov=app --cov-report=term-missing

# Single file
python -m pytest tests/security/test_authn.py -v

# Single test
python -m pytest tests/security/test_authn.py::TestJWTSecurity::test_alg_none_attack_rejected -v

# Name filter within a file/class
python -m pytest tests/security/test_authn.py -k alg_none -v

Path-based selection is the reliable split today. backend/pytest.ini declares a security marker, but the current suite is organized primarily by directory rather than by applied markers.

---

Pytest configuration

backend/pytest.ini currently sets:

Setting	Value	Effect
asyncio_mode	auto	Async tests run without needing per-test event-loop boilerplate
testpaths	tests	Test discovery is rooted under backend/tests/
addopts	-v --tb=short	Verbose test names and shortened tracebacks by default
markers	security	Reserved marker for hardening/security tests

---

Test infrastructure

tests/conftest.py

Shared fixtures available to all tests:

Fixture	Scope	Description
setup_database	session	Creates all ORM tables once at the start of the session
clean_tables	function (autouse)	Deletes all rows between each test
reset_rate_limiter	function (autouse)	Clears slowapi's in-memory counters — prevents 429 cascades
client	function	httpx.AsyncClient wired to the FastAPI app via ASGITransport
admin_user	function	Creates a live User row in the test DB
admin_token	function	Logs in as admin_user and returns the Bearer access token
db_session	function	Yields a live AsyncSession for direct DB state manipulation
TestSessionLocal	module helper	Async sessionmaker imported directly by some tests for manual inserts/queries

Database

Tests use sqlite+aiosqlite:///:memory: — a fresh in-memory SQLite instance for the entire pytest session. Tables are created once (setup_database), and every row is deleted between tests (clean_tables). This makes each test fully isolated without the overhead of dropping and recreating schema.

The app's normal get_db() dependency is overridden inside the client fixture, so HTTP requests made during tests always talk to SQLite even if local development is configured for PostgreSQL.

Rate limiter reset

slowapi stores rate-limit counters in memory. Without resetting them, the 5/min limit on /auth/login exhausts after 5 tests, causing every subsequent admin_token fixture to fail with 429. The reset_rate_limiter autouse fixture calls limiter._storage.reset() before each test.

---

Functional test suite

Functional tests live directly under tests/ and currently cover the main happy-path API behavior:

File	Coverage
test_auth.py	Login, invalid credentials, refresh-token rotation
test_pages.py	Page CRUD, public visibility rules, auth requirements
test_analytics.py	Authenticated analytics summary aggregation
test_analytics_ingest.py	Public analytics ingest, anon cookie creation, metadata filtering

---

Security test suite

Security tests live in tests/security/ and are organised by OWASP control domain. Each test references the relevant ASVS requirement and API Security Top 10 category in its docstring.

OWASP alignment matrix

File	ASVS chapters	API Top 10 categories
test_authn.py	V2 Authentication, V3 Session Management	API2 Broken Authentication
test_authz.py	V4 Access Control	API1 BOLA, API3 Object Property Auth, API5 Function Level Auth
test_input.py	V5 Input Validation & Sanitization	API4 Resource Consumption, API8 Security Misconfiguration
test_config.py	V9 Communication, V14 Configuration	API8 Security Misconfiguration, API9 Improper Inventory
test_rate_limit.py	V13 API & Web Service	API4 Resource Consumption, API6 Business Flow, API7 SSRF

---

test_authn.py — Authentication

ASVS V2.1 Credential validation

• Wrong password → 401
• Unknown email → 401 (same response as wrong password; prevents user enumeration)
• Empty / null password → 401 or 422
• Missing fields → 422
• Unicode password → 401 (no crash)
• 1 000-char password → 401 (bcrypt ValueError is caught and treated as invalid credentials)
• Inactive account → 401

ASVS V3.5 JWT security

• No Authorization header → 401/403
• Garbage Bearer value → 401/403
• JWT signed with wrong key → 401/403
• Expired JWT → 401/403
• alg: none attack (unsigned token) → 401/403
• Tampered payload with valid signature → 401/403
• Token in query string → 401/403 (URL leaks tokens to logs and Referer)
• HTTP Basic Auth scheme → 401/403

ASVS V3.5 Refresh token rotation

• Token is rotated; old token is revoked and returns 401 on reuse
• Forged refresh token → 401
• Empty refresh token → 401/422
• New access token issued after rotation is accepted

---

test_authz.py — Authorization

ASVS V4.1 | API5 Unauthenticated write access

• Parametrized over all 8 write/protected endpoints
• Verifies 401/403 for: POST /pages, PUT /pages/:slug, DELETE /pages/:slug, POST /posts, PUT /posts/:slug, DELETE /posts/:slug, PUT /settings, GET /analytics/summary

ASVS V4.2 | API1 Object-level authorization (BOLA)

• Non-existent slug → 404 (not 403, not 500)
• Path-traversal slugs (../admin, %2e%2e/secret) → 404/422
• DELETE/PUT on absent resource → 404

ASVS V4.3 | API3 Mass assignment

• Injected fields (is_admin, hashed_password, id, internal_field) are stripped from create and update requests
• Caller-supplied id is ignored; DB assigns it

---

test_input.py — Input validation and sanitization

ASVS V5.1 Schema validation

• Required fields enforced on pages and posts
• Null required fields → 422
• event_type, page, element field max-lengths enforced
• Malformed JSON body → 422 (not 500)

ASVS V5.2 | API8 HTML sanitization (nh3/ammonia) All vectors are tested at create and update time:

• <script>alert()</script> — stripped
• onerror="...", onclick="..." event attributes — stripped
• href="javascript:..." — stripped
• <iframe> — stripped
• <object> — stripped
• Safe HTML (<p>, <strong>, <a href>, <ul>) — preserved

ASVS V5.3 SQL injection

• Parametrized over 5 SQL injection patterns in slug path parameter → 404, not 500
• Parametrized over 4 injection patterns in login email field → 401, not 200 or 500

Analytics metadata sanitization

• Unknown keys dropped silently
• Nested objects dropped
• __proto__ / constructor pollution keys rejected by allowlist
• Oversized string values truncated to 120 chars
• Null metadata accepted
• 200-key object with mostly unknown keys → not 500

---

test_config.py — Security configuration

ASVS V14.4 | API8 HTTP security headers

Header	ASVS	Reason not set
X-Content-Type-Options: nosniff	14.4.3	Set by SecurityHeadersMiddleware
X-Frame-Options: DENY	14.4.5	Set by SecurityHeadersMiddleware
Content-Security-Policy	14.4.6	Set by SecurityHeadersMiddleware
Strict-Transport-Security	9.2.2	Set by SecurityHeadersMiddleware
Referrer-Policy	14.4.4	Set by SecurityHeadersMiddleware

Implemented in app/main.py:

from starlette.middleware.base import BaseHTTPMiddleware

class SecurityHeadersMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request, call_next):
        response = await call_next(request)
        response.headers["X-Content-Type-Options"] = "nosniff"
        response.headers["X-Frame-Options"] = "DENY"
        response.headers["Referrer-Policy"] = "strict-origin-when-cross-origin"
        return response

app.add_middleware(SecurityHeadersMiddleware)

ASVS V14.3 Error handling

• 404 on unknown route → no stack trace, no file paths, no site-packages
• 422 validation error → Pydantic JSON envelope only
• 401 response → WWW-Authenticate: Bearer header present (RFC 7235)
• Malformed JSON body → 422 not 500
• Wrong Content-Type → 415/422 not 500

ASVS V14.5 CORS

• http://localhost:5173 → ACAO header echoed
• https://www.goodwalk.co.nz → ACAO header echoed
• https://evil.example.com → no ACAO header, not *

API9 Inventory

• /health → {"status": "ok"}
• /openapi.json → accessible (documented exposure; restrict in production)
• /docs and /redoc → disabled by default (ENABLE_DOCS=false)

---

test_rate_limit.py — Rate limiting and SSRF

ASVS V13.2 | API4 Rate limit headers

slowapi is configured with headers_enabled=True, so rate-limited endpoints emit X-RateLimit-* response headers and clients can back off gracefully.

Current configuration in app/middleware/rate_limit.py:

limiter = Limiter(key_func=get_remote_address, headers_enabled=True)

API4 Payload size

• Analytics page field > 255 chars → 422
• 100 KB body field → not 500
• Deeply nested JSON (50 levels) → not 500

API7 SSRF — Private IP suppression Parametrized over 8 private/loopback IPs in X-Forwarded-For: 127.0.0.1, 10.0.0.1, 10.255.255.255, 192.168.1.100, 172.16.0.1, 172.31.255.255, ::1, localhost → event recorded (201); geo-lookup silently skipped (no outbound request to ip-api.com)

Parametrized over 5 malformed XFF values → not 500

API6 Business flow

• Analytics ingest is intentionally unauthenticated on both /api/web/event and the legacy alias /api/analytics/event
• Analytics summary requires authentication
• Session cookie is HttpOnly
• Session cookie is SameSite=Lax

---

Coverage

Run with coverage to see which app paths are exercised:

python -m pytest --cov=app --cov-report=term-missing --cov-report=html
# Open htmlcov/index.html in a browser

Focus areas for future test expansion:

• app/routers/sections.py — legacy CMS endpoints not yet covered by security tests
• app/services/analytics.py — geo-lookup mock / retry behaviour
• Concurrency: two simultaneous refresh token rotations (race condition check)

---

Troubleshooting

Common issues when running the suite locally:

Symptom	Likely cause	What to do
ModuleNotFoundError / wrong rootdir	Running pytest outside backend/	cd backend first, then rerun
Login-related tests start returning 429	Rate-limit state leaked between tests	Confirm tests/conftest.py is being loaded and reset_rate_limiter is still autouse
PytestCacheWarning about .pytest_cache on Windows	Local permission issue writing pytest cache files	Usually safe to ignore; test results are still valid