0.2.64 update

server/internal/credits/README.md

@@ -0,0 +1,129 @@
+# Credits detection
+
+Where an episode's closing credits begin, discovered for the small number of episodes a
+household is actually about to watch.
+
+This is not a library scanner and the distinction is the whole design. A scanner asks "what
+is in the library" and answers by reading all of it. This asks "what will somebody press Play
+on over the next few evenings", which Tracearr already knows, and reads almost nothing:
+
+```
+Tracearr demand → priority queue → marker cached? → tiny tail scan → one write → never again
+```
+
+## Why it exists
+
+Emby's `CreditsStart` marker is in its `MarkerType` enumeration and Emby 4.10 does not write
+it. A survey of this household's 20,000-item library found `Chapter`, `IntroStart` and
+`IntroEnd` and **no `CreditsStart` at all**. The existing feature's coverage comes entirely
+from chapters *named* like credits — about 5% of items. This fills in the rest.
+
+Emby still wins wherever it has an answer. `api/intro.go` reads its chapters first and only
+falls through to a discovered marker when there is nothing to defer to.
+
+## What was reused rather than built
+
+The investigation that preceded this found more already in place than expected, and none of
+it was duplicated:
+
+| Needed | Already there |
+|---|---|
+| Tracearr ingestion | `internal/tracearr` + the `tracearr_sessions` table, imported by the For You pipeline |
+| Emby item ids for sessions | `emby_item_id` / `emby_series_id`, backfilled by `UpdateTracearrSessionMapping` |
+| Stop positions and next-episode transitions | `progress_ms`, `total_duration_ms`, `started_at`/`stopped_at` on the same table |
+| Episode numbering | `library_items`, via `ParentIndexNumber`/`IndexNumber` in the payload |
+| Delivery to Android TV | `introResponse.creditsAvailable` / `creditsStartMs` — **no client changes at all** |
+| Background scheduling | `internal/scheduler`, so the interval is the operator's and the last run is in the console |
+| Live playback signal | the playback reports televisions already send |
+
+Two database reads produce the entire candidate queue for the whole household, whatever it
+is watching. Nothing here is per-candidate and nothing polls Tracearr directly.
+
+## The files
+
+```
+service.go     the worker, the cache check, and when scanning is allowed
+queue.go       bounded in-RAM priority queue; nothing about it is persisted
+candidates.go  velocity, look-ahead, decay, multi-user merging — all pure
+tracearr.go    demand → candidates, and the in-memory episode index
+behaviour.go   stop clustering; can write a marker with no media access at all
+confidence.go  signal combination and the rule that stops repeated rewrites
+window.go      where to scan, and how season history narrows it
+detector.go    the two-stage sparse visual scan and its changepoint
+sampler.go     ffmpeg, used surgically
+fingerprint.go media version identity, so a replaced file invalidates itself
+resolver.go    one Emby request per candidate, cached
+postgres.go    the store adapter
+load.go        whether the server is too busy for speculative work
+```
+
+## Two detectors, and the cheap one is often better
+
+**Behavioural** clustering costs nothing: no file is opened, no decoder runs, no new row is
+written anywhere. Where three viewers independently stopped an episode within seconds of each
+other, near the end but not at it, that agreement is evidence no decoder can produce. Two
+viewers is enough only when both rolled into the next episode, which is unambiguous about why
+they left.
+
+**Visual** scanning finds a sustained structural transition — dark, flat, textured with thin
+text, and staying that way for a minute. Darkness is *multiplied* rather than added into the
+frame score, which is the one modelling decision worth defending: under a weighted sum a
+night exterior reaches the credit-like floor on darkness and flatness alone, which is exactly
+how a final scene comes to be reported as a credits roll. A unit test pins that case.
+
+The two fail in unrelated ways, so agreement between them is worth far more than either
+alone — hence a probabilistic union rather than an average. Disagreement beyond 20 seconds
+costs a 0.3 penalty, which usually means storing nothing. **Prefer no marker to a wrong
+marker** is the governing rule throughout: a missing Skip Credits button is an absence nobody
+notices, a button during the final scene is a fault they remember.
+
+## Cost, and what is still unmeasured
+
+**Derived arithmetic** (checkable without hardware):
+
+| | |
+|---|---|
+| Generic tail window | `clamp(runtime × 0.20, 5min, 12min)` — 8m50s on a 44-minute episode, **20% of the file** |
+| Season-narrowed window | ±90s around the expected position — **3 minutes, 6.8% of the file** |
+| Coarse pass | one 160×90 grey frame every 4s — ~45 frames over a narrowed window |
+| Fine pass | 750ms over ±30s — ~80 frames |
+| Frame buffer | 14,400 bytes, allocated once and reused for the whole pass |
+| Writes, cached marker | **0** |
+| Writes, successful scan | **1** |
+
+Against scanning the full library: 20,000 items read end to end versus a queue capped at 20
+candidates, most of which are rejected by the marker check before any media is touched. Once
+a household settles, the steady state is one indexed read per candidate and nothing else.
+
+**Not yet measured, and it needs the NAS.** Bytes actually read, wall-clock analysis time,
+CPU and peak RSS all depend on the container, the network path to Emby and the media itself,
+and no figure taken anywhere else would mean anything. The tool is built and ships in the
+image:
+
+```
+docker compose exec server /app/memby-credits benchmark <emby-item-id>
+```
+
+It prints the runtime, the scan region **and its provenance**, an estimated byte count,
+frames sampled, allocation, elapsed time, the detection and what would be stored. The
+provenance line is the one to read: a run reporting `generic-tail-window` every time is a run
+in which demand-driven narrowing is doing nothing, and the design would need revisiting.
+
+## Things to preserve
+
+- **The queue is deliberately not durable.** Candidate priorities are rebuilt from one
+  Tracearr query on restart, which is cheaper and simpler than a second persistent job
+  scheduler. The database holds markers and nothing else.
+- **A single worker, and it is not a placeholder for a pool.** Concurrent scans multiply the
+  two costs this exists to minimise on a machine whose real job is streaming video.
+- **Live playback does not scan immediately.** `livePlaybackDelay` (45s) is what stops a
+  curious button press becoming disk activity; `AbandonPlayback` withdraws it.
+- **The marker is keyed on a media fingerprint**, so a file Sonarr replaces stops matching
+  with nothing having to notice the swap. A fingerprint too weak to detect a replacement
+  (runtime only) causes the marker to be *withheld* rather than stored un-invalidatable.
+- **Season history needs two markers, not one.** One is an anecdote and may itself be the
+  mis-detection; narrowing a scan onto it is how one wrong marker propagates through a season.
+- **`ShouldRewrite` is the only thing between "one write per episode, ever" and a row updated
+  on every playback.** Readings wobble by seconds; a ±12s difference is not news.
+- **The detector never learns why an episode was chosen.** That boundary is what stops it
+  being tuned to agree with the predictor rather than with the media.