Files
memby/server/README.md
T

746 lines
44 KiB
Markdown
Raw Normal View History

# Memby gateway
A small Go service that sits between the Memby Android TV client and Emby. It owns
authentication, caching, search and the shaping of TV screens, so the client can stay a
thin renderer.
Video never passes through here. `/v1/items/{id}/playback` returns a direct-play URL
pointing at Emby itself; only metadata and artwork traverse the gateway.
## Run it
```bash
cp .env.example .env # from the repo root
$EDITOR .env # set MEMBY_EMBY_URL and POSTGRES_PASSWORD
docker compose up -d --build
2026-07-29 15:26:27 +12:00
curl localhost:32768/readyz
```
Then build the TV app against it:
```powershell
2026-07-29 15:26:27 +12:00
.\gradlew.bat assembleDebug -Pmemby.gatewayUrl=https://mserver.sublogue.com
```
Leaving `memby.gatewayUrl` blank keeps the app on its original direct-to-Emby path, so a
gateway outage is one rebuild away from being routed around.
## Local development
```bash
go build ./...
go test ./...
go run ./cmd/memby-server # needs Postgres + Redis reachable
```
On Windows, `go` may need `-buildvcs=false` when the working tree has no usable `.git`.
2026-07-29 15:26:27 +12:00
## Deploy to the NAS
```powershell
.\deploy-server.ps1 # from the repo root; PowerShell 7
```
Streams the local `server/`, `docker-compose.yml` and `.env.example` to the NAS over one
SSH connection, builds there, and swaps the release in only once
Postgres, Redis and the server all report healthy — otherwise the previous release is put
back. The Postgres volume survives every deployment.
Configuration comes from the local `.env.example`, which holds real values: each deployment
installs it as the NAS's `.env`, keeping the previous one as `.env.previous`. It validates
the fixed `32768` reverse-proxy port and required admin/Emby secrets before activation.
Changing `POSTGRES_PASSWORD` intentionally recreates the PostgreSQL volume with an empty
database and the new password.
It deploys the **local working tree**, including uncommitted server changes.
`-SourceDirectory`, `-Destination`, `-RemoteHost` and `-HealthTimeoutSeconds` cover the rest.
2026-07-27 21:06:51 +12:00
## Logs
2026-08-06 22:33:56 +12:00
The server writes one aligned line per event, designed for
`docker compose logs -f server` and for the admin page's live log:
2026-07-27 21:06:51 +12:00
2026-08-06 22:33:56 +12:00
```
2026-08-14 11:47:32 +12:00
2026-08-03 11:33:40 INFO playback requested component=playback user=matt device="Living room" client=0.1.60 correlation=req-8F2A1C play_session_id=8F2A1C title="Severance Good News About Hell" item=184223 type=Episode play_method=DirectStream resume=12m0s runtime=57m0s version=0.1.0
2026-08-03 11:33:40 INFO request component=home user=matt device="Living room" client=0.1.60 correlation=req-4D91B7 protocol=1 method=GET path=/v1/home status=200 duration=412ms cache=miss version=0.1.0
2026-08-06 22:33:56 +12:00
```
The timestamp is a column rather than a `time=` field, and the fields are always in the
same order: **who and where** first (`component`, `user`, `device`, `client`), then what
the event is about, with `version` (the gateway build) and `error` last. `MEMBY_LOG_FORMAT`
switches the whole stream to `logfmt` (slog's own text) or `json` for a collector.
2026-08-12 09:57:56 +12:00
The same structured records are appended to `MEMBY_LOG_HISTORY_PATH` as JSONL and restored
into the admin log when the gateway starts. Compose mounts that path from the persistent
`memby-logs` volume, so replacing the container for a new version keeps the previous
history. The archive is compacted to the configured buffer capacity and remains bounded.
2026-08-14 11:47:32 +12:00
Every line from a request carries the viewer, the television, the app build, a correlation
identifier and the `component` — the part of the app the call came from, derived from the
route, so it is right even for an APK too old to report anything about itself. The same
identifier is returned as `X-Memby-Correlation`; the TV's network diagnostics record it
beside the request outcome. Playback events also carry the Emby play-session identifier.
2026-08-06 22:33:56 +12:00
Beyond the per-request line, these are logged as events in their own right: sign-in,
sign-out and rejected sign-ins; a device removed or renamed from another TV; what
somebody asked to play and what happened to it (`playback requested`, `playback
started`, `playback stopped` with how much was watched, `next episode resolved`); an
update offered or forced; a media request; maintenance and feature-policy changes; and
library imports (start, progress per 500-item page, final counts and duration).
At the default `MEMBY_LOG_LEVEL=INFO`, successful health checks, live maintenance polls,
artwork requests, search terms and ten-second playback progress reports are hidden;
2026-08-14 11:47:32 +12:00
warnings and failures from those routes are still shown. Set `MEMBY_LOG_LEVEL=DEBUG` for
diagnostic request and playback detail, or `TRACE` for the deepest negotiation and request
trace, then recreate the server container. URLs and sensitive structured attributes are
redacted before they reach either the console or admin log history.
2026-08-06 22:33:56 +12:00
The build that wrote a line is `internal/buildinfo/VERSION`, embedded at compile time —
bump it with a meaningful server change. It is also on `/healthz` and in the admin rail.
2026-07-27 21:06:51 +12:00
## API
All `/v1` routes need `Authorization: Bearer <token>` from `/v1/auth/login`. Image URLs
accept `?t=<token>` instead, because the client's image loader fetches plain URLs with no
headers attached.
| Method | Path | Purpose |
| --- | --- | --- |
2026-07-27 21:06:51 +12:00
| POST | `/v1/auth/login` | Emby credentials + device identity in, gateway token out |
| POST | `/v1/auth/logout` | Retire this device's token |
| GET | `/v1/auth/session` | Confirm a stored token is still valid |
2026-08-02 22:10:19 +12:00
| GET | `/v1/auth/devices` | List this user's signed-in TVs |
| PUT | `/v1/auth/devices/{deviceId}` | Rename a signed-in TV |
| DELETE | `/v1/auth/devices/{deviceId}` | Revoke another signed-in TV |
2026-07-29 15:26:27 +12:00
| GET | `/v1/status` | Lightweight live maintenance state plus informational alerts; remains available during maintenance |
2026-08-02 22:10:19 +12:00
| GET | `/v1/features` | Versioned, capability-gated feature document evaluated for this TV |
| GET | `/v1/home?limit=` | **Every launcher row in one response** |
| GET | `/v1/recommendations?refresh=1` | Recommendation rows alone; `refresh` forces a rebuild |
2026-07-29 15:26:27 +12:00
| GET | `/v1/for-you?minutes=` | Tracearr-powered, explainable picks for a 0/30/60/120-minute viewing window |
| GET | `/v1/update` | Whether this client (per `X-Memby-Version`) must update |
| GET | `/v1/screensaver?limit=` | Backdrop pool, cached and shuffled per request |
2026-07-29 15:26:27 +12:00
| GET | `/v1/search?q=&limit=` | Library search (title, series name, year, genre, studio) |
| GET | `/v1/search/history` | Current user's distinct searches from the last 30 days |
| POST | `/v1/search/history` | Record a successful search for the current user |
| GET | `/v1/items/{id}` | Full metadata for one item |
2026-08-03 08:52:55 +12:00
| GET | `/v1/items/{id}/ratings` | Optional server-filtered MDBList movie ratings; empty on disable/unavailable/failure |
2026-07-29 15:26:27 +12:00
| GET | `/v1/items/{id}/episodes` | Complete episode browser for a series, grouped into seasons by the TV |
| GET | `/v1/items/{id}/playback` | Resolves series → episode, returns a direct-play URL |
2026-07-29 15:26:27 +12:00
| GET | `/v1/items/{id}/next` | Episode following this one, or 404 when nothing does |
| GET | `/v1/items/{id}/trailer` | First local trailer, or 404 |
| POST | `/v1/items/{id}/favorite` | `{"value":true}` |
| POST | `/v1/items/{id}/played` | `{"value":true}` |
| POST | `/v1/playback/{started\|progress\|stopped}` | Progress reporting |
| POST | `/v1/analytics/rows` | Batched row engagement from a TV |
| GET | `/v1/images/{itemId}/{backdrop\|primary\|logo\|thumb}` | Artwork proxy |
| GET | `/healthz`, `/readyz` | Liveness, readiness |
### Server-driven rows
`/v1/home` returns a `rows` array — order, titles and kinds all decided here — plus the
2026-07-29 15:26:27 +12:00
three fixed rows repeated flat for the client's offline cache:
```json
{
"rows": [
{"id": "continue", "title": "Continue Watching", "kind": "continue", "items": [...]},
2026-08-02 22:10:19 +12:00
{"id": "for-you:pick-up", "title": "Pick this show up again", "kind": "for-you", "items": [...]},
{"id": "sonarr-airing-today", "title": "Shows airing in the next 5 days", "kind": "schedule", "items": [...]},
{"id": "favorites", "title": "Favourites", "kind": "favorites", "items": [...]},
2026-07-29 15:26:27 +12:00
{"id": "latest-movies", "title": "Recent New Releases", "kind": "latest", "items": [...]},
{"id": "similar:sev", "title": "Because you watched Severance", "kind": "similar", "items": [...]},
2026-07-27 21:06:51 +12:00
{"id": "recommended", "title": "Recommended from your watching history", "kind": "recommended", "items": [...]},
{"id": "curated:apple-tv", "title": "Apple TV+ Shows", "kind": "shows", "items": [...]},
{"id": "curated:drama-shows", "title": "Drama TV Shows", "kind": "shows", "items": [...]},
{"id": "curated:comedy-shows", "title": "Comedy TV Shows", "kind": "shows", "items": [...]}
],
2026-07-29 15:26:27 +12:00
"continueWatching": [...], "favorites": [...], "latestMovies": [...],
"partial": false
}
```
The TV renders whatever arrives, so a new row ships without an app release. `kind` picks
the card shape; an unrecognised kind falls back to poster cards rather than being dropped.
2026-08-06 22:33:56 +12:00
There is no Next Up row: those episodes are interleaved into Continue Watching, most
recently watched first, so the episode after one that just finished is the first card
rather than a card in another row. The response still carries a flat `nextUp` array for
televisions running a build that predates the merge.
2026-07-29 15:26:27 +12:00
The user's own section toggles still hide the three fixed rows, but never rows the server
invented — nobody opted out of a row that did not exist when they last opened Settings.
### Recommendations
`internal/recommend` builds rows from viewing history. Two kinds:
- **"Because you watched X"** — Emby's own `/Items/{id}/Similar` for the most recent
distinct titles, filtered down to what the user has not seen. Emby's similarity ranking
is better than anything worth reimplementing here; this only removes the already-watched.
- **"Recommended from your watching history"** — genre and studio affinity. History is
weighted by recency (0.94 per position, so the 12th item counts about half the most
recent), favourites add a smaller fixed weight, and candidates are unplayed titles in the
top three genres scored by affinity + a mild community-rating nudge. Titles tagged with
many genres get a `sqrt(n)` penalty so genre-stuffing cannot buy a top slot.
2026-07-27 21:06:51 +12:00
- **Curated TV shelves** — Apple TV+, Drama and Comedy membership is filtered from the
imported Postgres catalogue. Within each shelf, unseen shows are ranked using the same
personal genre/studio profile; the shelves themselves are ordered by affinity too.
A new profile falls back to community rating until it has viewing history.
Rows shorter than four items are dropped, and a user with no history gets no rows at all
2026-07-27 21:06:51 +12:00
rather than a strip of noise, except curated shelves: these remain useful with their
quality-ranked fallback.
2026-08-02 22:10:19 +12:00
**The home screen never waits on the engine.** Rows live in their own `r:<userId>:rows:v3`
cache key with a long TTL (24h). A cache miss serves home immediately without them and
triggers a background rebuild — deduplicated per user, so four TVs waking together do the
2026-08-02 22:10:19 +12:00
work once. Playback and favourite mutations do not discard this slow-moving taste cache.
The scoring is pure and unit-tested (`profile_test.go`), and the row assembly runs against
a fake Emby (`engine_test.go`), so neither needs a server to verify.
2026-07-27 21:06:51 +12:00
Curated shelves require at least one completed library import because their genre/studio
filter runs against Postgres. Emby's exact studio naming is preserved during import;
Apple matching accepts `Apple TV+`, `Apple TV Plus` and `Apple Studios`.
2026-07-29 15:26:27 +12:00
### Tracearr-powered For You
`/v1/for-you` is separate from `/v1/home`, so the launcher remains a single fast request
and never waits for analytics. The Android rail loads For You only when opened and lets
the viewer choose 30 minutes, one hour, two hours or any length.
The gateway combines:
- Emby history, favourites and catalogue metadata;
- Tracearr's read-only public `/api/v1/public/history` data, including completion,
aggregate watch time, device/platform, codecs and direct-play/transcode outcomes;
- Memby's own recent card focus, dwell and selection events;
- the requested time window and the item's Emby runtime.
Tracearr currently exposes no per-user or `since` history query. A background importer
therefore pages newest-first and upserts recommendation-relevant fields by Tracearr's
stable `(serverId, sessionId)` key. Incremental passes run every five minutes and stop
after two unchanged pages (at most ten); a daily full pass catches late or out-of-order
updates and reconciles deletions. Imports are idempotent.
2026-08-06 22:33:56 +12:00
Whether a pass is owed is decided by the timestamps recorded in `app_settings`, not by
how long this process has been up: the scheduler ticks, asks, and usually does nothing.
That is what makes the cadence survive a restart. Driving it from tickers alone meant a
redeploy or a bounced container started the interval again and imported immediately, so a
gateway that restarted often imported far more often than configured — and, in the other
direction, one restarted daily never reached its 24-hour full reconciliation at all. The
startup pass follows the same rule; `/admin` still forces either kind on demand.
2026-07-29 15:26:27 +12:00
Postgres retains the compact source sessions, one derived profile per enabled Emby user,
2026-08-02 22:10:19 +12:00
and at most 750 ranked Movie/Series candidates per user. The background builder reads Tracearr's
2026-07-29 15:26:27 +12:00
public users endpoint and exact-matches usernames against Emby's user list. Unmatched
Emby users still receive Emby-history recommendations; similar-looking names are never
fuzzily joined without an explicit operator decision.
Keeping the pool deliberately much larger than the cards shown gives the request-time
`minutes` filter enough headroom. `/v1/for-you` reads up to 240 ranked candidates and
derives several de-duplicated shelves: top picks, up to two distinct “Because you
finished …” shelves, up to two genre shelves, and a television-compatible shelf when
there is enough playback evidence. Completed-title evidence is distributed
deterministically across relevant candidates rather than allowing the newest Drama
title to explain every Drama recommendation.
2026-08-02 22:10:19 +12:00
The endpoint remains one indexed PostgreSQL read plus JSON enrichment. Tracearr imports
stay frequent but do not rebuild pools. A session's first transition to stopped/completed
dirties only its matched user, and all pools rebuild once daily at the configured local
off-peak hour. A stored algorithm version triggers a one-time startup migration only when
2026-08-06 22:33:56 +12:00
ranking behaviour changes. A cold or failed rebuild uses the original live Tracearr/Emby
2026-08-02 22:10:19 +12:00
path, so persistence cannot blank the area.
2026-07-29 15:26:27 +12:00
Every returned item is enriched with `MembyRecommendationReason` and
`MembyCompatibility`. The TV shows the reason on the card and in the focused metadata
panel. Missing Tracearr or sparse codec evidence degrades to Emby/Memby signals rather
than blanking the area; Tracearr errors never affect the essential home rows.
2026-08-02 22:10:19 +12:00
Tracearr episode history also identifies shows abandoned in season one. A show qualifies
after 21 days without activity only when Emby still returns a season-one `Next Up`
episode, which prevents completed shows and later-season pauses from leaking into the
row. Genuine matches appear on Home as **Pick this show up again**, even when only one
show qualifies; generic recommendations are never used as padding.
Item payloads are Emby's own JSON, forwarded verbatim. That is deliberate: the Android
client already models this shape, so there is no second schema to keep in sync.
`app/src/test/.../GatewayPayloadTest.kt` and `internal/api/api_test.go` pin the envelope
around it from both sides.
2026-07-27 21:06:51 +12:00
### Sonarr schedule
When `MEMBY_SONARR_URL` and `MEMBY_SONARR_API_KEY` are set, the gateway reads Sonarr's
2026-08-02 22:10:19 +12:00
v3 calendar and inserts **Shows airing in the next 5 days** near Continue Watching.
The window is today plus the following four local calendar days. Card timing reads
`In 8 hours (4:00 PM)` for later today, `Tomorrow: 4:00 PM` for tomorrow, and then the
weekday and local time. Cards also show season/episode number, episode title and one of:
upcoming, downloading, awaiting download, unmonitored, or the exact time Sonarr added
the episode file.
2026-07-27 21:06:51 +12:00
2026-08-16 09:02:17 +12:00
Only shows the household actually follows appear. Sonarr's calendar is read with
`unmonitored=true`, because the calendar page and the aired banners want the whole
picture, but an unmonitored show is never coming and so is dropped from this row. An
episode arriving with no expanded series record is kept — that is no evidence either way.
2026-07-27 21:06:51 +12:00
This row is informational. An episode listed before it is downloaded is not an Emby
item, so the TV deliberately does not offer Play, Favourite or Watched actions on it.
Sonarr poster and fanart requests are proxied through the gateway; its API key is never
sent to the TV. The shared Redis entry is refreshed every five minutes by default, not
once per user.
2026-07-29 15:26:27 +12:00
### Aired banners
The same calendar data drives a slide-in banner on open clients: when an episode's air
time passes and Sonarr has not imported it yet, `/v1/status` starts returning an alert —
```json
{"maintenance": false, "message": "", "alerts": [
{"id": "sonarr:7:42:aired", "kind": "sonarr-aired", "title": "Northbound",
"message": "S02E04 — The Crossing aired at 9:00 PM and will be in Emby soon.",
"itemId": "sonarr:7:42", "imageTag": "sonarr", "airedAt": "2026-07-27T21:00:00+12:00"}]}
```
`/v1/status` is the poll the app already runs every ten seconds, so this needs no push
channel and no second connection. Alerts read from the cached calendar, so a polling
client never costs a Sonarr request of its own; they are newest-first and capped at three.
Episodes already downloaded and unmonitored ones say nothing — the first is on the home
screen already, the second is never coming.
The server has no idea which TVs have seen what, so the client owns that: it persists the
ids it has shown and displays each alert once, for ten seconds, without taking focus — a
ring on the banner counts that down, since the viewer cannot dismiss it by remote.
Clients poll only while a Memby screen is in the foreground, and record an alert as shown
only when the banner is actually on screen. So keep offering an alert for the whole window
rather than once: a TV that was showing its screensaver when the episode aired will pick
the alert up when someone comes back to it, and one that never does simply lets the alert
expire with the window.
`MEMBY_SONARR_ALERT_WINDOW` bounds how long after air time an alert stays current;
2026-08-02 22:10:19 +12:00
`0` disables banners while leaving the five-day schedule row alone.
### Radarr digital-release schedule
When `MEMBY_RADARR_URL` and `MEMBY_RADARR_API_KEY` are set, the gateway reads Radarr's
v3 calendar and inserts **Upcoming Movie releases** beside the Sonarr schedule near
Continue Watching. The row covers today plus the following four local calendar days.
Radarr's `digitalRelease` date is preferred. When it is missing, the row uses an
estimated digital availability date 30 days after `inCinemas`; physical-disc dates are
ignored. A known digital date always wins, including for older films with later cinema
re-releases.
Movie posters and fanart are proxied through the gateway so the Radarr API key never
leaves the server. `MEMBY_RADARR_TTL` controls the shared calendar cache lifetime.
2026-07-29 15:26:27 +12:00
## Admin interface
2026-08-10 20:39:26 +12:00
`https://mserver.sublogue.com/admin/` — a self-contained console for library imports,
maintenance, engagement and journeys. Set `MEMBY_ADMIN_TOKEN` to enable it; unset, every
2026-08-02 22:10:19 +12:00
`/admin` route 404s so it cannot be left exposed by accident. The page first uses the same
discreet Emby login gate as the private installer, and additionally requires that the
account **administers Emby**`Policy.IsAdministrator`, which Emby returns with the
authentication itself, so the check costs no extra request. Knowing a household password
is not the qualification for reaching the console: `/install` is public by design, and any
viewer may sign in there. A refusal is worded exactly as a wrong password, so nothing
discloses that the account was right but the person is not an administrator; the operator
sees the reason in the log. An Emby that returns no policy at all is asked directly rather
than read as a refusal, and one that will not answer refuses the sign-in — silence is
never taken as permission.
The two sign-ins issue the *same* cookie, so they are separated by being signed with their
own purpose: an installer session cannot satisfy the console's gate, while an
administrator's session satisfies the installer's. After successful verification the page
2026-08-02 22:10:19 +12:00
establishes the HttpOnly admin cookie, but browser API requests require both that cookie
and the current 90-day Emby-verified admin session. The old admin cookie therefore cannot
2026-08-02 22:10:19 +12:00
bypass the gate after the browser session expires. Scripts may continue to use
`Authorization: Bearer <MEMBY_ADMIN_TOKEN>` without a browser session.
Those 90 days are idle time, not a hard limit: opening an admin page, making any change,
2026-08-06 22:33:56 +12:00
or reading one while interacting with it slides the expiry forward once it is inside the
last 45 days. What deliberately does **not** extend it is the page's own status
2026-08-06 22:33:56 +12:00
poll — a console left open on a second monitor still times out, which is the whole point
of the TTL. The page marks its own requests with `X-Memby-Admin-Active` when there has
been interaction in the last five minutes, and on a 401 it reloads, so an expiry lands as
the sign-in form with `next` pointing back at the page rather than as an error banner.
| Method | Path | Purpose |
| --- | --- | --- |
| GET | `/admin/` | The page |
2026-08-10 08:37:08 +12:00
| POST | `/admin/logout` | End the browser admin session |
| GET | `/admin/api/status` | Library counts, sync history, maintenance state |
2026-08-02 22:10:19 +12:00
| GET | `/admin/api/runtime` | Protected Go heap, memory-limit and goroutine metrics |
| POST | `/admin/api/sync` | `{"kind":"full"}` or `{"kind":"incremental"}` |
2026-07-29 15:26:27 +12:00
| POST | `/admin/api/for-you` | `{"action":"incremental-import"}`, `{"action":"full-import"}`, or `{"action":"rebuild-all"}` |
| POST | `/admin/api/maintenance` | `{"enabled":true,"message":"…"}` |
| POST | `/admin/api/update-policy` | `{"enabled":true,"latestVersion":"0.1.54","downloadUrl":"…","required":false}` |
2026-08-02 22:10:19 +12:00
| POST | `/admin/api/features` | Publish feature overrides, enter safe mode, reset defaults, or roll back one revision |
2026-08-03 08:52:55 +12:00
| POST | `/admin/api/mdblist-settings` | Enable MDBList, replace/clear its API key, and select visible rating sources |
| GET | `/admin/api/analytics?days=7` | Row engagement |
2026-08-10 20:39:26 +12:00
| GET | `/admin/api/journeys?days=30&userId=…` | Journey health, feature use, paths and event history |
2026-08-02 22:10:19 +12:00
The **Features** admin page is the recovery surface for optional TV behaviour. Flags are
registered in the server catalogue and persist only explicit overrides; clearing an
override returns it to its code-owned safe default. Every write is revision-checked and
retains the prior revision. Safe mode disables all optional flags while leaving sign-in,
browsing, playback, maintenance and the admin console operational.
TVs report `X-Memby-Version`, `X-Memby-Protocol`, and a bounded
`X-Memby-Capabilities` list on every request. Protocol and capability checks fail closed,
so a server never advertises an optional contract to a client that did not declare it.
The first migrated controls are the Sonarr preroll, automatic My Shows, and its automatic
follow notification; all three can be stopped without publishing an APK.
2026-08-03 08:52:55 +12:00
### Optional MDBList movie ratings
The **Movie ratings** admin page enables MDBList, stores its API key only in the gateway's
`app_settings` table, and selects which supported sources may be returned. It is disabled
by default. Movie detail pages request `/v1/items/{id}/ratings` independently of essential
Emby metadata; missing provider IDs, MDBList errors, invalid or unavailable values all
produce an empty successful response. Successful MDBList responses are cached in Redis for
2026-08-06 22:33:56 +12:00
24 hours and persisted in Postgres for seven days before the configured source filter is
applied, so restarts, multiple TVs, and changing visible sources do not consume more
MDBList quota. If MDBList is temporarily unavailable, the most recently persisted response
is served stale. The API key is never returned by either the TV or admin status APIs.
2026-08-03 08:52:55 +12:00
## Library import
`internal/library` copies Emby's catalogue into Postgres so the gateway answers from its
own data instead of asking Emby per request.
- **Full** — pages through everything (500 items per request), then deletes any row it did
not touch, which is how removals propagate. Run once to seed; re-run after reorganising
the library.
- **Incremental** — asks Emby only for items changed since the last successful run
(`MinDateLastSaved`, with a minute of overlap so nothing falls between runs). This is
the hourly job: new episodes appear within the hour, and the weekly film drop rides
along with no extra configuration.
An incremental run with no previous success upgrades itself to a full one, so a fresh
deployment self-seeds on its first tick. Only one import runs at a time; the scheduler
skips its tick if one is still going, and interrupted runs are marked failed at boot
rather than sitting on "running" forever.
**Credentials.** Imports use `MEMBY_SYNC_USER_ID` + `MEMBY_SYNC_API_KEY` when set, and
otherwise borrow the most recently active TV session. The fallback means a new deployment
imports as soon as somebody signs in, but it stops working if that user is deleted — set a
service account for anything long-lived.
**What is *not* imported:** every query runs with `EnableUserData=false`. Watched flags,
favourites and resume positions are per-user and cannot be shared across a household, so
they still come from Emby live. The imported copy powers search and the recommendation
candidate pool.
2026-08-18 14:59:29 +12:00
### Event-driven ingest
Sonarr and Radarr are the things that put files on disk, so they are what the catalogue
learns from. Both post to the gateway, each event is recorded in `library_ingest_queue`,
and a single worker reads the named title out of Emby a minute later — rather than the
whole catalogue waiting on the next sweep. An episode imported at 19:05 is searchable at
19:06 instead of as late as 20:00.
| | |
|---|---|
| Sonarr | `POST /hooks/sonarr?token=$MEMBY_SONARR_WEBHOOK_TOKEN` |
| Radarr | `POST /hooks/radarr?token=$MEMBY_RADARR_WEBHOOK_TOKEN` |
In each *arr: **Settings → Connect → + → Webhook**, method POST, with **On Import, On
Upgrade, On Rename, On File Delete** and **On Series/Movie Delete** ticked. The token may
also be sent as `X-Memby-Token`, a bearer token or basic-auth password; an unset token
makes the hook 404, so a deployment that never configured one cannot be posted to. Press
**Test** to check reachability — it answers 200 and records nothing.
Both hooks sit outside the maintenance gate *and* outside the quiet-time gate, which is
the point of the queue being durable: the gate answers 503 and neither *arr re-delivers,
so a quiet hour would otherwise discard every import that happened during it. The hook
records at any hour; the worker is where quiet time is honoured.
**What each event does.** An import or an upgrade re-reads the item — an upgrade is silent
as *news*, because the film was already there, but the file genuinely changed. A rename is
a refresh and never an invalidation: the Emby item id survives a move, and so does the
credits marker measured against it. A delete removes the row and its credits marker, and
only counts when the media went with it — a series unfollowed in Sonarr with its files
left on disk is still in the library.
**Nothing is done twice.** The queue key is derived from the *file* rather than the
delivery, so a repeated webhook collapses onto one row; a file deleted and re-imported is
a different file and its own work. Emby not having scanned a new file yet is the expected
first answer rather than a fault: one rescan nudge is sent and the row retries on a
widening backoff (1m, 5m, 20m, 1h, then four-hourly) before being given up on.
**The sweep is reconciliation now.** `MEMBY_SYNC_INTERVAL` still runs the incremental
import, and with both webhooks wired up it exists for what the *arrs do not manage — a
file dropped in by hand, a title edited in Emby, a notification that never arrived because
the container was down. Six hours is a sensible value then; the console can set it at
**Settings → Catalogue sweep** without a redeployment, and it takes effect on the next
cycle rather than at the next restart.
**Where to look.** Admin console → **Imports** shows whether each hook is configured, what
is waiting, and the last fifty events with why each was queued and what happened to it —
which is the page to read when somebody says a new episode is not showing up. In the log
it is `event=arr_ingest` with an `outcome` of `queued`, `imported`, `removed`, `absent`,
`not_found` or `deferred`.
## Maintenance mode
Takes Memby down independently of Emby: all `/v1` routes answer `503` with
`{"maintenance": true, "message": "…"}`, and the TV shows the operator's message instead of
2026-07-27 21:06:51 +12:00
a network error. The exact `/v1/status` control route stays up too: open clients poll it
every ten seconds, stop active playback, and move immediately to the maintenance screen.
`/healthz`, `/readyz` and `/admin` also stay up — they are what you need while the app is
deliberately off.
The switch lives in Postgres, not memory, so a restart cannot quietly bring the app back
up mid-repair. Each instance caches it and re-reads every 30 seconds, so toggling it
directly in the database works too.
## App update policy
The gateway decides whether a TV may keep running its current build. Clients send
2026-07-27 21:06:51 +12:00
`X-Memby-Version` on every request and ask `GET /v1/update` on each launch and hourly
2026-08-02 22:10:19 +12:00
while left open; the verdict is `none`, `optional` or `mandatory`. This endpoint is
deliberately public and remains available during maintenance: update policy is checked
before login and never reads, validates, or mutates a viewer session.
2026-08-10 20:54:00 +12:00
The App updates page separates a required update from a destructive one. Required covers
the home screen until the APK is installed but keeps the viewer's session. “Set the
destructive floor to this update” records the release as the destructive floor: the next request
from an older build deletes its session, and signing in again is refused until that build
has updated. “Sign out builds below” lets the operator set that floor to an exact version
instead; leaving it blank disables destructive retirement.
2026-08-02 22:10:19 +12:00
### First-time TV installation
The gateway hosts a public bootstrap page at:
```text
https://mserver.sublogue.com/install
```
The site root displays the same page. Before any release details or download link are
shown, it asks for an Emby household username and password and verifies them directly
against Emby. Credentials are never stored. The temporary Emby access token is logged
out immediately, and the browser receives a secure, HTTP-only, same-site installer cookie
valid for 30 minutes. This browser login does not create a Memby TV session and therefore
2026-08-12 13:08:53 +12:00
does not appear in the user's signed-in device list. Emby records the password check
under the gateway's own client name (`MEMBY_GATEWAY_CLIENT_NAME`), not a television's, so
its temporary device record is never mistaken for a set in the house.
2026-08-02 22:10:19 +12:00
After authentication, `/updates/latest.apk` redirects to the current immutable, versioned
APK in the persistent `memby-releases` volume. Direct APK requests require either that
short-lived browser session or the per-release signed URL returned to an authenticated
Memby app during its update check; guessed filenames return 404. This is the exact signed
artefact produced and published by `deploy-server.ps1`, not a separate bootstrap build.
The APK is compiled with `memby.gatewayUrl=https://mserver.sublogue.com`, so a fresh
installation opens directly at Memby's login screen.
On a new TV, enter the short `/install` address in a browser or the Downloader app,
download the APK, allow that browser/downloader to install unknown apps if Android asks,
then install and launch Memby. No Files/Storage permission is needed by Memby itself.
The installer is also intentionally absent from search: `/robots.txt` disallows the complete
host, the page and APK responses send `X-Robots-Tag` no-index/no-follow/no-archive
directives, and the HTML repeats those controls for major crawlers. A scanner can discover
that the host exists, but it cannot see a release or download an APK without valid access.
2026-07-27 21:06:51 +12:00
### Automated tagged releases
`.gitea/workflows/release.yml` turns a pushed semantic-version tag into an update:
1. Gitea Actions derives the Android version from a tag such as `v0.1.54`.
2. It runs the tests and builds the APK with the established release signing key.
3. It uploads the signed APK to `POST /admin/api/release`.
4. The gateway stores the APK in its `memby-releases` volume and makes it the latest
optional update. Older TVs prompt on their next check.
The repository needs an Actions runner with the `ubuntu-latest` label and these Actions
secrets:
| Secret | Purpose |
| --- | --- |
| `ANDROID_KEYSTORE_BASE64` | Existing release keystore, base64 encoded as one line |
| `ANDROID_KEYSTORE_PASSWORD` | Keystore password |
| `ANDROID_KEY_ALIAS` | Signing alias |
| `ANDROID_KEY_PASSWORD` | Key password |
| `MEMBY_RELEASE_PUBLISH_TOKEN` | Same dedicated value configured on the gateway |
Enable repository Actions and give the job permission to read contents. Then releasing
from any clone is:
```bash
git tag -a v0.1.54 -m "Personalised favourites"
git push origin main v0.1.54
```
Tags are the release boundary; an ordinary branch push never publishes an APK. Android
will only accept updates signed by the same keystore as the installed app.
2026-08-02 22:10:19 +12:00
### Publishing an update with `deploy-server.ps1`
The normal NAS deployment can build and publish the TV app in the same operation:
```powershell
.\deploy-server.ps1 -AppVersion 0.1.70 -ReleaseNotes 'Reliable in-app updates'
```
Append `--m` to make the published update mandatory. Older clients receive a blocking
update screen with no skip or dismiss action:
```powershell
.\deploy-server.ps1 -ReleaseNotes 'Required security update' --m
```
It loads the same user-level `MEMBY_KEYSTORE*` variables as `deploy-tv.ps1`, runs the
Android tests, builds the requested release without editing `build.gradle.kts`, verifies
the APK signature, and sends the signed artefact in the SSH deployment archive. After
PostgreSQL, Redis and the replacement gateway are healthy, the remote script publishes
the APK through `POST /admin/api/release`. The gateway records its SHA-256 digest and
byte size in PostgreSQL and serves it from the persistent `memby-releases` volume.
`MEMBY_RELEASE_PUBLISH_TOKEN` must contain a stable, private value in `.env.example`.
Every update intended for installed TVs needs a version higher than the one they already
run; Android rejects equal or lower `versionCode` values. Reusing a version for different
APK bytes is also rejected because release URLs are immutable.
For a server-only emergency deployment, explicitly opt out:
```powershell
.\deploy-server.ps1 -SkipAppRelease
```
Set it on the admin page: **latest version**, **APK URL** (normally the same file the
2026-08-10 20:54:00 +12:00
landing page serves), release notes, the required-update toggle, and an optional
destructive compatibility floor.
- *Optional* — a dismissable prompt. Dismissal lasts for that session only.
- *Required* — a full-screen panel over the home screen with no way past it. Back is
swallowed and there is one button. Use it when a build is genuinely unusable, not for
ordinary releases.
"Required" sets `minimumVersion = latestVersion`; anything below that floor is forced.
`minimumVersion` can also be set directly for a staged rollout where the forced floor is
older than the newest build.
2026-08-10 20:54:00 +12:00
When **Sign out builds below** is set, a build below that version is retired on its next
authenticated request: the gateway deletes the session and returns 401, which makes the TV
remove the rejected local profile. The public update check continues to return the
mandatory update screen, and the gateway refuses a new login from the retired build, so
signing in again cannot bypass the update. The floor remains dormant when the policy has no
download URL or its latest release is older than the selected floor.
Two deliberate safeguards, both tested in `internal/appupdate`:
- A client that cannot report a version is **never** forced. It would otherwise be stuck
behind a prompt it may have no way to satisfy.
- The client only shows a verdict carrying a download URL, so a misconfigured policy
cannot produce an unblockable screen with a dead button. An unreachable gateway shows
nothing at all.
The verdict is deliberately *not* part of `/v1/home`: that payload is cached per user,
while this answer depends on the requesting client's version, so sharing a cache entry
would hand one TV's answer to another.
## Row analytics
The TV reports three signals per row — `impression` (drawn), `focus` (the remote landed
there, with dwell), `select` (something was opened) — batched and uploaded every 20
seconds to `POST /v1/analytics/rows`. Dwell below 400 ms is dropped client-side as D-pad
travel rather than attention, and the server clamps anything over 30 minutes.
Read it at `/admin/`, sorted by dwell. Dwell is the number worth watching: impressions
only say a row was on screen, while dwell says someone stopped there. It is the fastest
way to tell whether "Recommended from your watching history" is earning its slot.
Raw events are pruned after `MEMBY_ANALYTICS_RETENTION` (90 days) and aggregates are
computed at read time, so nothing survives the prune. This is tuning telemetry, not a
record of what anyone watched.
2026-08-10 20:39:26 +12:00
Journey events are ingested separately at `POST /v1/analytics/events` and have their own
**Journeys** admin page. The gateway derives completion, abandonment, active visits,
feature use, significant actions and common paths from the retained events. An unfinished
visit remains active for 30 minutes before it counts as abandoned. Selecting one profile
reveals its ordered event history; search text, content titles and setting values are never
accepted into this stream.
## Caching
Redis holds everything user-scoped under `u:<embyUserId>:*`, plus session lookups under
`sess:<tokenHash>` and recommendation rows under `r:<embyUserId>:rows`. Any mutation —
favourite, watched, playback stopped — drops the `u:` keys, so the next home request
re-reads Emby rather than serving a row it just contradicted. Partial home payloads are
served but never cached. The `r:` namespace is deliberately excluded from that wipe (see
Recommendations above).
2026-07-29 15:26:27 +12:00
Postgres is the durable half: it holds gateway sessions, the imported Emby catalogue,
Tracearr's compact recommendation history, derived profiles and prepared candidate pools.
Losing Redis costs a cold cache; losing Postgres signs everyone out and requires catalogue
and recommendation backfills. A gateway session is unique per Emby user and stable device
2026-08-02 22:10:19 +12:00
ID; signing the same TV in again rotates its token. Accounts have no device cap; viewers
can review and revoke signed-in TVs from the app's Settings screen.
## Configuration
| Variable | Default | Notes |
| --- | --- | --- |
| `MEMBY_EMBY_URL` | *required* | How the gateway reaches Emby |
| `MEMBY_EMBY_PUBLIC_URL` | = `MEMBY_EMBY_URL` | What TVs stream from |
| `MEMBY_DATABASE_URL` | *required* | Postgres DSN |
| `MEMBY_REDIS_URL` | `redis://localhost:6379/0` | |
2026-07-29 15:26:27 +12:00
| `MEMBY_LISTEN_ADDR` | `:8080` outside Compose; `:32768` in the NAS stack | |
2026-08-14 11:47:32 +12:00
| `MEMBY_LOG_LEVEL` | `INFO` | `DEBUG` adds diagnostic request/playback detail; `TRACE` adds deep negotiation tracing |
2026-08-12 09:57:56 +12:00
| `MEMBY_LOG_BUFFER_CAPACITY` | `5000` | Bounded persistent admin event history; `0` disables capture |
| `MEMBY_LOG_HISTORY_PATH` | `/data/logs/events.jsonl` | JSONL history restored after container replacement |
2026-08-02 22:10:19 +12:00
| `MEMBY_GOMEMLIMIT` | `384MiB` | Compose value passed to Go as `GOMEMLIMIT` |
| `MEMBY_SERVER_MEMORY_LIMIT` | `512m` | Compose hard memory ceiling for the server container |
2026-07-27 21:06:51 +12:00
| `MEMBY_TIMEZONE` | `Pacific/Auckland` | Local day and time labels for schedule rows |
2026-08-06 22:33:56 +12:00
| `MEMBY_CLIENT_NAME` | `MbyATV` | Client name sent to Emby; must match the app's direct path |
2026-08-12 13:08:53 +12:00
| `MEMBY_GATEWAY_CLIENT_NAME` | `MbyGateway` | Client name Emby records for the gateway's own requests, including an admin or installer sign-in |
| `MEMBY_HOME_TTL` | `60s` | Also `MEMBY_ITEM_TTL`, `MEMBY_SEARCH_TTL`, `MEMBY_SCREENSAVER_TTL` |
2026-08-02 22:10:19 +12:00
| `MEMBY_RECOMMEND_TTL` | `24h` | How long computed recommendation rows stay warm |
| `MEMBY_RECOMMEND_TIMEOUT` | `60s` | Bounds a background rebuild |
2026-07-29 15:26:27 +12:00
| `MEMBY_TRACEARR_URL` / `MEMBY_TRACEARR_API_KEY` | *empty* | Read-only Tracearr public API; both values are required |
| `MEMBY_TRACEARR_SERVER_ID` | *empty* | Optional server scope when Tracearr monitors several servers |
2026-08-06 22:33:56 +12:00
| `MEMBY_TRACEARR_SYNC_INTERVAL` | `5m` | Minimum age of the last incremental import before another is owed; `0` disables incremental passes |
| `MEMBY_TRACEARR_FULL_INTERVAL` | `24h` | Minimum age of the last full reconciliation before another is owed; `0` disables full passes |
2026-08-02 22:10:19 +12:00
| `MEMBY_FOR_YOU_MIN_REBUILD_AGE` | `24h` | Safety bound for non-forced pool rebuilds |
| `MEMBY_FOR_YOU_REFRESH_INTERVAL` | `24h` | Acceptable prepared-pool staleness |
| `MEMBY_FOR_YOU_REBUILD_HOUR` | `4` | Local hour (023) for the daily prepared rebuild |
| `MEMBY_ADMIN_TOKEN` | *empty* | Enables `/admin`. Empty = admin disabled |
2026-07-27 21:06:51 +12:00
| `MEMBY_PUBLIC_URL` | *empty* | Public gateway origin used for APK download links |
| `MEMBY_RELEASE_DIR` | `/data/releases` | Persistent signed APK directory |
| `MEMBY_RELEASE_PUBLISH_TOKEN` | *empty* | Enables the CI-only release upload endpoint |
| `MEMBY_SYNC_INTERVAL` | `1h` | Incremental import cadence; `0` disables |
| `MEMBY_SYNC_TIMEOUT` | `30m` | Bounds one import |
| `MEMBY_SYNC_ON_START` | `false` | Import at boot |
2026-07-27 21:06:51 +12:00
| `MEMBY_SONARR_URL` | *empty* | Sonarr address reachable by the gateway; empty disables integration |
| `MEMBY_SONARR_API_KEY` | *empty* | Sonarr Settings → General → Security API key |
| `MEMBY_SONARR_TTL` | `5m` | Shared Redis lifetime for today's calendar |
2026-07-29 15:26:27 +12:00
| `MEMBY_SONARR_ALERT_WINDOW` | `3h` | How long after air time an "aired" banner stays current; `0` disables banners |
2026-08-02 22:10:19 +12:00
| `MEMBY_RADARR_URL` | *empty* | Radarr address reachable by the gateway; empty disables integration |
| `MEMBY_RADARR_API_KEY` | *empty* | Radarr Settings → General → Security API key |
2026-08-17 19:09:17 +12:00
| `MEMBY_RADARR_TTL` | `5m` | Shared Redis lifetime for the month-long digital-release calendar |
| `MEMBY_SYNC_USER_ID` / `MEMBY_SYNC_API_KEY` | *empty* | Emby service account for imports |
| `MEMBY_ANALYTICS_RETENTION` | `2160h` (90d) | Raw row events are pruned past this |
| `MEMBY_SESSION_CACHE_TTL` | `5m` | How long a token lookup stays in Redis |
| `MEMBY_SESSION_IDLE_EXPIRY` | `2160h` (90d) | Unused tokens are swept every 6h |
| `MEMBY_UPSTREAM_TIMEOUT` | `20s` | |
## Security notes
- The `sessions` table stores **live Emby access tokens** in plaintext. Gateway tokens are
stored only as SHA-256 hashes, so a database dump does not yield working gateway
credentials — but it does yield working *Emby* ones. Treat the Postgres volume as a
secret store, and encrypting `emby_token` at rest is the obvious next hardening step.
- Image URLs carry the gateway token in a query string, so it will appear in any access
log in front of this service. That token is revocable and grants nothing outside Memby,
which is why the artwork proxy exists at all.
- Nothing here terminates TLS. Put it behind your existing reverse proxy before exposing
it beyond the LAN.
## Not built yet
- **Live change feed.** Imports are polled hourly rather than driven by Emby's WebSocket,
so a brand-new episode can be up to an hour late. Good enough for a household; the
WebSocket would make it instant.
- **Cache warming.** Rows go cold after `MEMBY_HOME_TTL`; the first TV to ask pays for the
refresh. A background refresher per active session would hide that.
- **Rate limiting** on `/v1/auth/login`.