172 lines
10 KiB
Markdown
172 lines
10 KiB
Markdown
# Credits detection
|
||
|
||
Where an episode's closing credits begin, discovered for the small number of episodes a
|
||
household is actually about to watch.
|
||
|
||
This is not a library scanner and the distinction is the whole design. A scanner asks "what
|
||
is in the library" and answers by reading all of it. This asks "what will somebody press Play
|
||
on over the next few evenings", which Tracearr already knows, and reads almost nothing:
|
||
|
||
```
|
||
Tracearr demand → priority queue → marker/cooldown check → tiny tail scan → history + marker
|
||
```
|
||
|
||
## Why it exists
|
||
|
||
Emby's `CreditsStart` marker is in its `MarkerType` enumeration and Emby 4.10 does not write
|
||
it. A survey of this household's 20,000-item library found `Chapter`, `IntroStart` and
|
||
`IntroEnd` and **no `CreditsStart` at all**. The existing feature's coverage comes entirely
|
||
from chapters *named* like credits — about 5% of items. This fills in the rest.
|
||
|
||
Emby still wins wherever it has an answer. `api/intro.go` reads its chapters first and only
|
||
falls through to a discovered marker when there is nothing to defer to.
|
||
|
||
## What was reused rather than built
|
||
|
||
The investigation that preceded this found more already in place than expected, and none of
|
||
it was duplicated:
|
||
|
||
| Needed | Already there |
|
||
|---|---|
|
||
| Tracearr ingestion | `internal/tracearr` + the `tracearr_sessions` table, imported by the For You pipeline |
|
||
| Emby item ids for sessions | `emby_item_id` / `emby_series_id`, backfilled by `UpdateTracearrSessionMapping` |
|
||
| Stop positions and next-episode transitions | `progress_ms`, `total_duration_ms`, `started_at`/`stopped_at` on the same table |
|
||
| Episode numbering | `library_items`, via `ParentIndexNumber`/`IndexNumber` in the payload |
|
||
| Delivery to Android TV | `introResponse.creditsAvailable` / `creditsStartMs` — **no client changes at all** |
|
||
| Background scheduling | `internal/scheduler`, so the interval is the operator's and the last run is in the console |
|
||
| Live playback signal | the playback reports televisions already send |
|
||
|
||
Two database reads produce the entire candidate queue for the whole household, whatever it
|
||
is watching. Nothing here is per-candidate and nothing polls Tracearr directly.
|
||
|
||
## The files
|
||
|
||
```
|
||
service.go the worker, the cache check, and when scanning is allowed
|
||
queue.go bounded in-RAM priority queue; nothing about it is persisted
|
||
candidates.go velocity, look-ahead, decay, multi-user merging — all pure
|
||
tracearr.go demand → candidates, and the in-memory episode index
|
||
behaviour.go stop clustering; can write a marker with no media access at all
|
||
confidence.go signal combination and the rule that stops repeated rewrites
|
||
window.go where to scan, and how season history narrows it
|
||
detector.go the two-stage sparse visual scan and its changepoint
|
||
sampler.go ffmpeg, used surgically
|
||
fingerprint.go media version identity, so a replaced file invalidates itself
|
||
resolver.go one Emby request per candidate, cached
|
||
postgres.go the store adapter
|
||
load.go whether the server is too busy for speculative work
|
||
```
|
||
|
||
## Operator controls and history
|
||
|
||
Admin Console → Credits detection controls three different bounds: the number of candidates
|
||
allowed to wait, the ordinary and maximum episode look-ahead, and the retry delay after an
|
||
inconclusive or failed speculative scan. The environment values are first-run defaults; a
|
||
saved console choice is restored on restart.
|
||
|
||
The retry delay is important. A successful marker excludes an episode naturally, but an old
|
||
worker forgot a no-match result and selected the episode again at the next ten-minute
|
||
refresh. Completed attempts now have a small durable history and no-match/failed episodes
|
||
cool down before prediction may select them again. Live playback may still raise an episode
|
||
immediately because a viewer is waiting.
|
||
|
||
## Two detectors, and the cheap one is often better
|
||
|
||
**Behavioural** clustering costs nothing: no file is opened, no decoder runs, no new row is
|
||
written anywhere. Where three viewers independently stopped an episode within seconds of each
|
||
other, near the end but not at it, that agreement is evidence no decoder can produce. Two
|
||
viewers is enough only when both rolled into the next episode, which is unambiguous about why
|
||
they left.
|
||
|
||
**Visual** scanning finds a sustained structural transition — dark, flat, textured with thin
|
||
text, and staying that way for about half a minute. Darkness is *multiplied* rather than added
|
||
into the frame score, which is the one modelling decision worth defending: under a weighted
|
||
sum a night exterior reaches the credit-like floor on darkness and flatness alone, which is
|
||
exactly how a final scene comes to be reported as a credits roll. The candidate frame itself
|
||
must also be genuinely dark, and at least 65% of the remaining frames must stay credit-like;
|
||
those two guards stop a dim final scene or a temporary title card becoming an early marker.
|
||
Unit tests pin both cases.
|
||
|
||
The two fail in unrelated ways, so agreement between them is worth far more than either
|
||
alone — hence a probabilistic union rather than an average. Disagreement beyond 20 seconds
|
||
costs a 0.3 penalty, which usually means storing nothing. **Prefer no marker to a wrong
|
||
marker** is the governing rule throughout: a missing Skip Credits button is an absence nobody
|
||
notices, a button during the final scene is a fault they remember.
|
||
|
||
## Where the bytes come from
|
||
|
||
The gateway has no filesystem access to the media, so a scan reads Emby's own stream route
|
||
over HTTP and `-ss` ahead of `-i` is what makes that a ranged request rather than a download.
|
||
That optimisation is only worth anything if the range survives the trip, which is why the
|
||
address is configurable separately from every other way the gateway reaches Emby.
|
||
|
||
`MEMBY_EMBY_MEDIA_URL` is the address media bytes are read from; unset it falls back to
|
||
`MEMBY_EMBY_URL` and nothing changes. The distinction is not cosmetic in this deployment: the
|
||
stack runs on the NAS, Emby and the media live on the HTPC, and `MEMBY_EMBY_URL` is a public
|
||
DDNS name. Left to fall back, every ranged read leaves the host for the internet-facing edge,
|
||
pays TLS and returns through a reverse proxy — and a proxy that buffers its upstream turns
|
||
the ranged read into a whole-file read, which is the single thing this package is built not to
|
||
do. Naming the LAN address instead keeps the read on the wire between the two machines.
|
||
|
||
It is **stated rather than inferred** for the usual reason: there is no way to look at a URL
|
||
and tell whether it happens to resolve locally, and guessing wrong either sends scans the long
|
||
way round or points them at a host that is not there. `media_url` is on the `credits detection
|
||
enabled` start-up line so an operator can read back which path is in force, and the benchmark
|
||
below sets it too — measuring the public route and deploying the internal one would report a
|
||
number belonging to neither.
|
||
|
||
## Cost, and what is still unmeasured
|
||
|
||
**Derived arithmetic** (checkable without hardware):
|
||
|
||
| | |
|
||
|---|---|
|
||
| Generic tail window | `clamp(runtime × 0.20, 5min, 12min)` — 8m50s on a 44-minute episode, **20% of the file** |
|
||
| Season-narrowed window | ±90s around the expected position — **3 minutes, 6.8% of the file** |
|
||
| Coarse pass | one 160×90 grey frame every 4s — ~45 frames over a narrowed window |
|
||
| Fine pass | 750ms over ±30s — ~80 frames |
|
||
| Frame buffer | 14,400 bytes, allocated once and reused for the whole pass |
|
||
| Writes, cached marker | **0** |
|
||
| Writes, completed scan | **1 history row**, plus **1 marker** only when accepted |
|
||
|
||
Against scanning the full library: 20,000 items read end to end versus a queue capped at 20
|
||
candidates, most of which are rejected by the marker check before any media is touched. Once
|
||
a household settles, the steady state is one indexed read per candidate and nothing else.
|
||
|
||
**Not yet measured, and it needs the NAS.** Bytes actually read, wall-clock analysis time,
|
||
CPU and peak RSS all depend on the container, the network path to Emby and the media itself,
|
||
and no figure taken anywhere else would mean anything. The tool is built and ships in the
|
||
image:
|
||
|
||
```
|
||
docker compose exec server /app/memby-credits benchmark <emby-item-id>
|
||
```
|
||
|
||
It prints the runtime, the scan region **and its provenance**, an estimated byte count,
|
||
frames sampled, allocation, elapsed time, the detection and what would be stored. The
|
||
provenance line is the one to read: a run reporting `generic-tail-window` every time is a run
|
||
in which demand-driven narrowing is doing nothing, and the design would need revisiting.
|
||
|
||
## Things to preserve
|
||
|
||
- **The queue is deliberately not durable.** Candidate priorities are rebuilt from one
|
||
Tracearr query on restart, which is cheaper and simpler than a second persistent job
|
||
scheduler. Completed scan history is durable because it is an operator record and the
|
||
source of retry cooldowns; it does not preserve or resume queue state.
|
||
- **A single worker, and it is not a placeholder for a pool.** Concurrent scans multiply the
|
||
two costs this exists to minimise on a machine whose real job is streaming video.
|
||
- **Live playback does not scan immediately.** `livePlaybackDelay` (45s) is what stops a
|
||
curious button press becoming disk activity; `AbandonPlayback` withdraws it.
|
||
- **The marker is keyed on a media fingerprint**, so a file Sonarr replaces stops matching
|
||
with nothing having to notice the swap. A fingerprint too weak to detect a replacement
|
||
(runtime only) causes the marker to be *withheld* rather than stored un-invalidatable.
|
||
- **Season history needs two markers, not one.** One is an anecdote and may itself be the
|
||
mis-detection; narrowing a scan onto it is how one wrong marker propagates through a season.
|
||
- **`ShouldRewrite` is the only thing between "one write per episode, ever" and a row updated
|
||
on every playback.** Readings wobble by seconds; a ±12s difference is not news.
|
||
- **The detector never learns why an episode was chosen.** That boundary is what stops it
|
||
being tuned to agree with the predictor rather than with the media.
|
||
- **An `ffmpeg` decoder crash gets one conservative retry.** The retry is single-threaded and
|
||
discards corrupt packets; ordinary network, authentication and timeout failures are not
|
||
retried by the sampler.
|