0.2.76 - Icon Packs
This commit is contained in:
ponzischeme89
@@ -429,6 +429,56 @@ favourites and resume positions are per-user and cannot be shared across a house
|
||||
they still come from Emby live. The imported copy powers search and the recommendation
|
||||
candidate pool.
|
||||
|
||||
### Event-driven ingest
|
||||
|
||||
Sonarr and Radarr are the things that put files on disk, so they are what the catalogue
|
||||
learns from. Both post to the gateway, each event is recorded in `library_ingest_queue`,
|
||||
and a single worker reads the named title out of Emby a minute later — rather than the
|
||||
whole catalogue waiting on the next sweep. An episode imported at 19:05 is searchable at
|
||||
19:06 instead of as late as 20:00.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Sonarr | `POST /hooks/sonarr?token=$MEMBY_SONARR_WEBHOOK_TOKEN` |
|
||||
| Radarr | `POST /hooks/radarr?token=$MEMBY_RADARR_WEBHOOK_TOKEN` |
|
||||
|
||||
In each *arr: **Settings → Connect → + → Webhook**, method POST, with **On Import, On
|
||||
Upgrade, On Rename, On File Delete** and **On Series/Movie Delete** ticked. The token may
|
||||
also be sent as `X-Memby-Token`, a bearer token or basic-auth password; an unset token
|
||||
makes the hook 404, so a deployment that never configured one cannot be posted to. Press
|
||||
**Test** to check reachability — it answers 200 and records nothing.
|
||||
|
||||
Both hooks sit outside the maintenance gate *and* outside the quiet-time gate, which is
|
||||
the point of the queue being durable: the gate answers 503 and neither *arr re-delivers,
|
||||
so a quiet hour would otherwise discard every import that happened during it. The hook
|
||||
records at any hour; the worker is where quiet time is honoured.
|
||||
|
||||
**What each event does.** An import or an upgrade re-reads the item — an upgrade is silent
|
||||
as *news*, because the film was already there, but the file genuinely changed. A rename is
|
||||
a refresh and never an invalidation: the Emby item id survives a move, and so does the
|
||||
credits marker measured against it. A delete removes the row and its credits marker, and
|
||||
only counts when the media went with it — a series unfollowed in Sonarr with its files
|
||||
left on disk is still in the library.
|
||||
|
||||
**Nothing is done twice.** The queue key is derived from the *file* rather than the
|
||||
delivery, so a repeated webhook collapses onto one row; a file deleted and re-imported is
|
||||
a different file and its own work. Emby not having scanned a new file yet is the expected
|
||||
first answer rather than a fault: one rescan nudge is sent and the row retries on a
|
||||
widening backoff (1m, 5m, 20m, 1h, then four-hourly) before being given up on.
|
||||
|
||||
**The sweep is reconciliation now.** `MEMBY_SYNC_INTERVAL` still runs the incremental
|
||||
import, and with both webhooks wired up it exists for what the *arrs do not manage — a
|
||||
file dropped in by hand, a title edited in Emby, a notification that never arrived because
|
||||
the container was down. Six hours is a sensible value then; the console can set it at
|
||||
**Settings → Catalogue sweep** without a redeployment, and it takes effect on the next
|
||||
cycle rather than at the next restart.
|
||||
|
||||
**Where to look.** Admin console → **Imports** shows whether each hook is configured, what
|
||||
is waiting, and the last fifty events with why each was queued and what happened to it —
|
||||
which is the page to read when somebody says a new episode is not showing up. In the log
|
||||
it is `event=arr_ingest` with an `outcome` of `queued`, `imported`, `removed`, `absent`,
|
||||
`not_found` or `deferred`.
|
||||
|
||||
## Maintenance mode
|
||||
|
||||
Takes Memby down independently of Emby: all `/v1` routes answer `503` with
|
||||
|
||||
@@ -167,6 +167,20 @@ func run(log *slog.Logger, events *logging.Buffer, logLevel *slog.LevelVar) erro
|
||||
DeviceID: "memby-gateway-sync",
|
||||
Gateway: true,
|
||||
}, log.With("component", "library"))
|
||||
|
||||
// Sonarr and Radarr are what put files on disk, so they are what the catalogue learns
|
||||
// from. The worker is built whenever either webhook is configured; with neither token
|
||||
// set both hooks 404 and nothing here ever has anything to do, so it is not started.
|
||||
var ingester *library.Ingester
|
||||
if cfg.SonarrWebhookToken != "" || cfg.RadarrWebhookToken != "" {
|
||||
ingester = &library.Ingester{
|
||||
Store: st,
|
||||
Emby: embyClient,
|
||||
Credentials: syncer.EmbyCredentials,
|
||||
Log: log.With("component", "library-ingest"),
|
||||
Settle: cfg.IngestSettleDelay,
|
||||
}
|
||||
}
|
||||
var forYouService *foryou.Service
|
||||
if tracearrClient != nil {
|
||||
forYouService = foryou.New(
|
||||
@@ -265,6 +279,7 @@ func run(log *slog.Logger, events *logging.Buffer, logLevel *slog.LevelVar) erro
|
||||
Credits: creditsService,
|
||||
CreditsLoad: creditsLoad,
|
||||
Syncer: syncer,
|
||||
Ingester: ingester,
|
||||
Log: log,
|
||||
Events: events,
|
||||
|
||||
@@ -283,6 +298,16 @@ func run(log *slog.Logger, events *logging.Buffer, logLevel *slog.LevelVar) erro
|
||||
creditsService.SetPaused(server.ActivityPaused)
|
||||
go creditsService.Run(ctx)
|
||||
}
|
||||
if ingester != nil {
|
||||
// Quiet time is honoured here rather than at the hook: the webhook is recorded
|
||||
// whatever the hour, and this is what waits.
|
||||
ingester.Paused = server.ActivityPaused
|
||||
// The news follows the scan rather than the webhook, so the banner can say a title
|
||||
// is there rather than that it is coming. Installed here for the same reason
|
||||
// SetAfterSync is: library stays ignorant of what an alert is.
|
||||
ingester.Announce = server.AnnounceLibraryIngest
|
||||
go ingester.Run(ctx)
|
||||
}
|
||||
|
||||
// Registration is separate from construction so the task list reads as a declaration
|
||||
// of what the gateway does in the background rather than as more wiring in here.
|
||||
@@ -326,7 +351,7 @@ func run(log *slog.Logger, events *logging.Buffer, logLevel *slog.LevelVar) erro
|
||||
}
|
||||
go server.WatchUpdatePolicy(ctx, 60*time.Second)
|
||||
|
||||
go syncer.Schedule(ctx, cfg.SyncInterval, server.ActivityPaused)
|
||||
go syncer.Schedule(ctx, server.LibrarySyncInterval, server.ActivityPaused)
|
||||
if cfg.SyncOnStart {
|
||||
go func() {
|
||||
if server.ActivityPaused() {
|
||||
|
||||
@@ -55,6 +55,7 @@ func (s *Server) adminRoutes() http.Handler {
|
||||
mux.Handle("GET /admin/api/events", s.adminAuth(s.handleAdminEvents))
|
||||
mux.Handle("GET /admin/api/runtime", s.adminAuth(s.handleAdminRuntime))
|
||||
mux.Handle("POST /admin/api/sync", s.adminAuth(s.handleAdminSync))
|
||||
mux.Handle("GET /admin/api/ingest", s.adminAuth(s.handleAdminIngest))
|
||||
mux.Handle("POST /admin/api/for-you", s.adminAuth(s.handleAdminForYou))
|
||||
mux.Handle("GET /admin/api/gateway-settings", s.adminAuth(s.handleAdminGatewaySettings))
|
||||
mux.Handle("POST /admin/api/gateway-settings", s.adminAuth(s.handleAdminGatewaySettings))
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
"github.com/ponzischeme89/memby/server/internal/store"
|
||||
)
|
||||
|
||||
// What the *arr webhooks have been doing, for the Imports page.
|
||||
//
|
||||
// It exists because a webhook is the one part of this gateway that fails *silently*: a
|
||||
// token typed wrongly into Sonarr, a URL the container cannot be reached on, or a
|
||||
// notification never enabled all look exactly like a household in which nothing has been
|
||||
// imported lately. Without this an operator's only recourse is reading container logs.
|
||||
|
||||
// ingestEventLimit is how much of the log the page carries. Enough to cover an evening's
|
||||
// imports and a season pack, which is what somebody is looking at when they open it.
|
||||
const ingestEventLimit = 50
|
||||
|
||||
type adminIngestResponse struct {
|
||||
// Configured says whether each hook would answer at all. Both unset is the honest
|
||||
// explanation of an empty table, and the page says so rather than leaving an operator
|
||||
// to conclude the feature is broken.
|
||||
SonarrConfigured bool `json:"sonarrConfigured"`
|
||||
RadarrConfigured bool `json:"radarrConfigured"`
|
||||
SettleSeconds int `json:"settleSeconds"`
|
||||
SyncMinutes int `json:"syncMinutes"`
|
||||
Counts store.IngestCounts `json:"counts"`
|
||||
Recent []store.IngestJob `json:"recent"`
|
||||
}
|
||||
|
||||
func (s *Server) handleAdminIngest(w http.ResponseWriter, r *http.Request) {
|
||||
ctx := r.Context()
|
||||
response := adminIngestResponse{
|
||||
SonarrConfigured: s.cfg.SonarrWebhookToken != "",
|
||||
RadarrConfigured: s.cfg.RadarrWebhookToken != "",
|
||||
SettleSeconds: int(s.ingestSettle().Seconds()),
|
||||
SyncMinutes: int(s.LibrarySyncInterval().Minutes()),
|
||||
Recent: []store.IngestJob{},
|
||||
}
|
||||
|
||||
counts, err := s.store.IngestStateCounts(ctx)
|
||||
if err != nil {
|
||||
s.loggerFor(ctx).Error("ingest counts failed", "error", err)
|
||||
writeError(w, http.StatusInternalServerError, "could not read the import queue")
|
||||
return
|
||||
}
|
||||
response.Counts = counts
|
||||
|
||||
recent, err := s.store.RecentIngests(ctx, ingestEventLimit)
|
||||
if err != nil {
|
||||
s.loggerFor(ctx).Error("ingest history failed", "error", err)
|
||||
writeError(w, http.StatusInternalServerError, "could not read the import queue")
|
||||
return
|
||||
}
|
||||
response.Recent = recent
|
||||
writeJSON(w, http.StatusOK, response)
|
||||
}
|
||||
@@ -13,6 +13,10 @@ import (
|
||||
const (
|
||||
alertKindSonarrAired = "sonarr-aired"
|
||||
alertKindRadarrImport = "radarr-import"
|
||||
// A new episode that has finished scanning in. Distinct from sonarr-aired, which is
|
||||
// about an episode that has been broadcast and is *not* here yet — the two are opposite
|
||||
// halves of the same wait and a viewer reads them differently.
|
||||
alertKindSonarrImport = "sonarr-import"
|
||||
alertKindLibrarySync = "library-updated"
|
||||
alertKindServerDown = "server-unreachable"
|
||||
alertKindServerUp = "server-restored"
|
||||
|
||||
@@ -59,8 +59,11 @@ type Server struct {
|
||||
credits *credits.Service
|
||||
creditsLoad *credits.PlaybackLoad
|
||||
syncer syncerHandle
|
||||
log *slog.Logger
|
||||
events *serverlogging.Buffer
|
||||
// ingester records what Sonarr and Radarr say changed. Nil where no webhook token is
|
||||
// configured, which is also what makes both hooks 404.
|
||||
ingester ingesterHandle
|
||||
log *slog.Logger
|
||||
events *serverlogging.Buffer
|
||||
// adminEvents is the administrative feed: the console's notification bell and every
|
||||
// outgoing integration read from it. Distinct from `events` above, which is the
|
||||
// structured log ring — a log line is what the gateway did, an admin event is
|
||||
@@ -96,6 +99,9 @@ type Server struct {
|
||||
// logged by name.
|
||||
playbackTitles playbackTitles
|
||||
|
||||
// ingestRuns collapses a season pack's worth of finished scans into one banner.
|
||||
ingestRuns ingestRuns
|
||||
|
||||
recommendationBuilds recommendationBuilds
|
||||
maintenance maintenanceState
|
||||
quietTime quietTimeState
|
||||
@@ -130,6 +136,7 @@ type Deps struct {
|
||||
Credits *credits.Service
|
||||
CreditsLoad *credits.PlaybackLoad
|
||||
Syncer syncerHandle
|
||||
Ingester ingesterHandle
|
||||
Log *slog.Logger
|
||||
Events *serverlogging.Buffer
|
||||
|
||||
@@ -157,6 +164,7 @@ func New(cfg config.Config, deps Deps) *Server {
|
||||
credits: deps.Credits,
|
||||
creditsLoad: deps.CreditsLoad,
|
||||
syncer: deps.Syncer,
|
||||
ingester: deps.Ingester,
|
||||
log: deps.Log,
|
||||
events: deps.Events,
|
||||
|
||||
@@ -314,9 +322,17 @@ func (s *Server) Routes() http.Handler {
|
||||
// status even while every normal /v1 operation is deliberately unavailable.
|
||||
mux.Handle("GET /v1/status", s.authed(s.handleServiceStatus))
|
||||
mux.Handle("/v1/", s.maintenanceGate(v1))
|
||||
// Radarr pushes here when an import finishes. Outside the gate on purpose: an event
|
||||
// arriving during maintenance would otherwise be lost rather than delayed.
|
||||
mux.Handle("POST /hooks/radarr", s.quietTimeGate(http.HandlerFunc(s.handleRadarrWebhook)))
|
||||
// Sonarr and Radarr push here when something lands, is upgraded, is renamed or is
|
||||
// deleted. Outside the maintenance gate on purpose: an event arriving during
|
||||
// maintenance would otherwise be lost rather than delayed.
|
||||
//
|
||||
// Outside the *quiet-time* gate too, which the Radarr hook was previously inside. That
|
||||
// gate answers 503, and neither *arr re-delivers — so a quiet hour used to silently
|
||||
// discard every import that happened during it. The durable queue is what makes the
|
||||
// distinction possible: the hook records the news whatever the hour, and the worker is
|
||||
// where quiet time is honoured.
|
||||
mux.HandleFunc("POST /hooks/radarr", s.handleRadarrWebhook)
|
||||
mux.HandleFunc("POST /hooks/sonarr", s.handleSonarrWebhook)
|
||||
// State the canonical console URL explicitly. The console and its assets live below
|
||||
// /admin/, while a bare /admin is routinely typed and some reverse proxies do not
|
||||
// preserve ServeMux's implicit trailing-slash redirect for a mounted subtree.
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/subtle"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
|
||||
"github.com/ponzischeme89/memby/server/internal/library"
|
||||
)
|
||||
|
||||
// The two things that push into the gateway.
|
||||
//
|
||||
// Radarr's hook was already here, announcing a film as news. Both hooks now also *record*
|
||||
// what changed, which is the half that replaces asking Emby every hour whether anything
|
||||
// had happened: Sonarr and Radarr are the things that put files on disk, so they are the
|
||||
// things that know.
|
||||
//
|
||||
// Recording is all a hook does. The lookup, the import and the retry all belong to the
|
||||
// worker in internal/library, which is what lets these answer in a millisecond and, more
|
||||
// importantly, what lets them answer *at all* during quiet hours — see below.
|
||||
|
||||
// ingesterHandle is the slice of the ingest worker the API needs, so api does not depend
|
||||
// on the concrete type for testing. Same arrangement as syncerHandle.
|
||||
type ingesterHandle interface {
|
||||
Enqueue(ctx context.Context, source string, requests []library.IngestRequest) (int, error)
|
||||
}
|
||||
|
||||
// handleSonarrWebhook accepts Sonarr's import, upgrade, rename and delete notifications.
|
||||
//
|
||||
// Unconfigured means absent — the stance /admin and the Radarr hook already take: a
|
||||
// deployment that never set a token must not expose an endpoint anything can post to.
|
||||
func (s *Server) handleSonarrWebhook(w http.ResponseWriter, r *http.Request) {
|
||||
if s.cfg.SonarrWebhookToken == "" {
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
if subtle.ConstantTimeCompare(
|
||||
[]byte(webhookToken(r)), []byte(s.cfg.SonarrWebhookToken),
|
||||
) != 1 {
|
||||
writeError(w, http.StatusUnauthorized, "invalid webhook token")
|
||||
return
|
||||
}
|
||||
|
||||
var payload library.SonarrWebhook
|
||||
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<20)).Decode(&payload); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid webhook payload")
|
||||
return
|
||||
}
|
||||
// Sonarr's Test button posts a stub. Answering 200 without recording work about a
|
||||
// series that does not exist is what makes that button mean "reachable".
|
||||
if library.IsTestEvent(payload.EventType) {
|
||||
s.loggerFor(r.Context()).Info("sonarr webhook test received")
|
||||
writeJSON(w, http.StatusOK, map[string]any{"ok": true, "test": true})
|
||||
return
|
||||
}
|
||||
|
||||
queued := s.queueIngest(r, "sonarr", payload.EventType, library.SonarrRequests(payload))
|
||||
writeJSON(w, http.StatusOK, map[string]any{"ok": true, "queued": queued})
|
||||
}
|
||||
|
||||
// queueIngest records the work a notification implies and reports how much of it was news.
|
||||
//
|
||||
// The context is deliberately detached from the request. A webhook is answered in a
|
||||
// millisecond and Sonarr closes the connection; hanging the insert off the request would
|
||||
// abandon exactly the deliveries that arrive in bursts, which is what a season pack is.
|
||||
func (s *Server) queueIngest(
|
||||
r *http.Request, source, eventType string, requests []library.IngestRequest,
|
||||
) int {
|
||||
log := s.loggerFor(r.Context())
|
||||
if s.ingester == nil {
|
||||
return 0
|
||||
}
|
||||
if len(requests) == 0 {
|
||||
// A grab, a health check, an unfollowed series whose files stayed on disk: all
|
||||
// real events, none of them a reason to re-read anything.
|
||||
log.Debug("webhook implies no catalogue work", "source", source, "event", eventType)
|
||||
return 0
|
||||
}
|
||||
ctx := context.WithoutCancel(r.Context())
|
||||
queued, err := s.ingester.Enqueue(ctx, source, requests)
|
||||
if err != nil {
|
||||
// The event is lost, which is the one failure worth an error line here: the *arrs
|
||||
// do not re-deliver, so nothing will bring this news again. The reconciliation
|
||||
// sweep is what eventually covers it.
|
||||
log.Error("could not record webhook work",
|
||||
"source", source, "event", eventType, "error", err)
|
||||
return queued
|
||||
}
|
||||
if queued > 0 {
|
||||
log.Info("arr ingest queued",
|
||||
"event", "arr_ingest", "source", source, "webhook_event", eventType,
|
||||
"outcome", "queued", "items", queued, "reason", requests[0].Reason)
|
||||
}
|
||||
return queued
|
||||
}
|
||||
@@ -0,0 +1,168 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/ponzischeme89/memby/server/internal/config"
|
||||
"github.com/ponzischeme89/memby/server/internal/library"
|
||||
)
|
||||
|
||||
// recordingIngester stands in for the worker. The hook's whole job is to record, so what
|
||||
// it recorded is the only thing worth asserting on here.
|
||||
type recordingIngester struct {
|
||||
sources []string
|
||||
requests []library.IngestRequest
|
||||
}
|
||||
|
||||
func (r *recordingIngester) Enqueue(
|
||||
_ context.Context, source string, requests []library.IngestRequest,
|
||||
) (int, error) {
|
||||
r.sources = append(r.sources, source)
|
||||
r.requests = append(r.requests, requests...)
|
||||
return len(requests), nil
|
||||
}
|
||||
|
||||
func sonarrHookRequest(body string) *http.Request {
|
||||
return httptest.NewRequest(
|
||||
http.MethodPost, "/hooks/sonarr?token=hook-secret", strings.NewReader(body))
|
||||
}
|
||||
|
||||
func TestSonarrWebhookIsHiddenUntilATokenIsConfigured(t *testing.T) {
|
||||
s := &Server{cfg: config.Config{}, log: discardLogger()}
|
||||
rec := httptest.NewRecorder()
|
||||
|
||||
s.handleSonarrWebhook(rec, sonarrHookRequest("{}"))
|
||||
|
||||
if rec.Code != http.StatusNotFound {
|
||||
t.Fatalf("got %d, want 404 for an unconfigured hook", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSonarrWebhookRejectsAWrongToken(t *testing.T) {
|
||||
ingester := &recordingIngester{}
|
||||
s := &Server{
|
||||
cfg: config.Config{SonarrWebhookToken: "hook-secret"},
|
||||
log: discardLogger(),
|
||||
ingester: ingester,
|
||||
}
|
||||
rec := httptest.NewRecorder()
|
||||
|
||||
s.handleSonarrWebhook(rec, httptest.NewRequest(
|
||||
http.MethodPost, "/hooks/sonarr?token=guess", strings.NewReader("{}")))
|
||||
|
||||
if rec.Code != http.StatusUnauthorized {
|
||||
t.Fatalf("got %d, want 401", rec.Code)
|
||||
}
|
||||
if len(ingester.requests) != 0 {
|
||||
t.Fatal("an unauthorised delivery recorded work")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSonarrWebhookRecordsAnImport(t *testing.T) {
|
||||
ingester := &recordingIngester{}
|
||||
s := &Server{
|
||||
cfg: config.Config{SonarrWebhookToken: "hook-secret"},
|
||||
log: discardLogger(),
|
||||
ingester: ingester,
|
||||
}
|
||||
rec := httptest.NewRecorder()
|
||||
|
||||
s.handleSonarrWebhook(rec, sonarrHookRequest(`{
|
||||
"eventType":"Download",
|
||||
"series":{"id":12,"title":"Blue Bloods","year":2010},
|
||||
"episodes":[{"id":551,"seasonNumber":6,"episodeNumber":7}],
|
||||
"episodeFile":{"id":8123}
|
||||
}`))
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("got %d, want 200", rec.Code)
|
||||
}
|
||||
if len(ingester.requests) != 1 {
|
||||
t.Fatalf("expected one recorded request, got %d", len(ingester.requests))
|
||||
}
|
||||
request := ingester.requests[0]
|
||||
if request.Kind != library.KindEpisode || request.Episode != 7 || request.Season != 6 {
|
||||
t.Fatalf("unexpected request: %+v", request)
|
||||
}
|
||||
if ingester.sources[0] != "sonarr" {
|
||||
t.Fatalf("unexpected source: %q", ingester.sources[0])
|
||||
}
|
||||
}
|
||||
|
||||
// The Test button must answer without recording work about a series that does not exist.
|
||||
func TestSonarrWebhookTestEventRecordsNothing(t *testing.T) {
|
||||
ingester := &recordingIngester{}
|
||||
s := &Server{
|
||||
cfg: config.Config{SonarrWebhookToken: "hook-secret"},
|
||||
log: discardLogger(),
|
||||
ingester: ingester,
|
||||
}
|
||||
rec := httptest.NewRecorder()
|
||||
|
||||
s.handleSonarrWebhook(rec, sonarrHookRequest(
|
||||
`{"eventType":"Test","series":{"id":1,"title":"Test Title"}}`))
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("got %d, want 200", rec.Code)
|
||||
}
|
||||
if len(ingester.requests) != 0 {
|
||||
t.Fatalf("the test event recorded work: %+v", ingester.requests)
|
||||
}
|
||||
}
|
||||
|
||||
// A Radarr upgrade is silent as news and still a reason to re-read the row. The two
|
||||
// judgements are made in different places and this is what pins them apart.
|
||||
func TestRadarrUpgradeIsRecordedThoughItIsNotAnnounced(t *testing.T) {
|
||||
ingester := &recordingIngester{}
|
||||
s := &Server{
|
||||
cfg: config.Config{
|
||||
RadarrWebhookToken: "hook-secret",
|
||||
RadarrAlertWindow: 0, // no announcement is possible
|
||||
},
|
||||
log: discardLogger(),
|
||||
ingester: ingester,
|
||||
}
|
||||
rec := httptest.NewRecorder()
|
||||
|
||||
s.handleRadarrWebhook(rec, httptest.NewRequest(
|
||||
http.MethodPost, "/hooks/radarr?token=hook-secret", strings.NewReader(`{
|
||||
"eventType":"Download","isUpgrade":true,
|
||||
"movie":{"id":44,"title":"Arrival","year":2016},
|
||||
"movieFile":{"id":441,"quality":"Bluray-1080p"}
|
||||
}`)))
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("got %d, want 200", rec.Code)
|
||||
}
|
||||
if len(ingester.requests) != 1 {
|
||||
t.Fatalf("the upgrade was not recorded: %+v", ingester.requests)
|
||||
}
|
||||
if ingester.requests[0].Reason != library.ReasonUpgrade {
|
||||
t.Fatalf("unexpected reason: %q", ingester.requests[0].Reason)
|
||||
}
|
||||
if ingester.requests[0].Action != library.ActionRefresh {
|
||||
t.Fatalf("an upgrade must refresh, got %q", ingester.requests[0].Action)
|
||||
}
|
||||
}
|
||||
|
||||
// A grab is a real event and not a reason to re-read anything.
|
||||
func TestSonarrGrabRecordsNothing(t *testing.T) {
|
||||
ingester := &recordingIngester{}
|
||||
s := &Server{
|
||||
cfg: config.Config{SonarrWebhookToken: "hook-secret"},
|
||||
log: discardLogger(),
|
||||
ingester: ingester,
|
||||
}
|
||||
rec := httptest.NewRecorder()
|
||||
|
||||
s.handleSonarrWebhook(rec, sonarrHookRequest(
|
||||
`{"eventType":"Grab","series":{"id":12,"title":"Blue Bloods"}}`))
|
||||
|
||||
if rec.Code != http.StatusOK || len(ingester.requests) != 0 {
|
||||
t.Fatalf("got %d with %d requests", rec.Code, len(ingester.requests))
|
||||
}
|
||||
}
|
||||
@@ -129,6 +129,15 @@ func (s *Server) embyHealthInterval() time.Duration {
|
||||
s.cfg.EmbyHealthInterval)
|
||||
}
|
||||
|
||||
// LibrarySyncInterval is how often the catalogue sweep runs. Exported because the syncer's
|
||||
// schedule reads it every tick rather than closing over it at start-up — a setting read
|
||||
// once at start-up is not a setting, and an operator lengthening the sweep after wiring up
|
||||
// the webhooks must not have to restart the container to see it take effect.
|
||||
func (s *Server) LibrarySyncInterval() time.Duration {
|
||||
return overrideWindow(s.gatewaySettings.get().LibrarySyncMinutes, time.Minute,
|
||||
s.cfg.SyncInterval)
|
||||
}
|
||||
|
||||
// overrideWindow reads one of the three settings that can be switched off: a negative
|
||||
// value is off, zero is "whatever was deployed", anything else is the override in the
|
||||
// given unit.
|
||||
@@ -153,6 +162,7 @@ type deployedGatewaySettings struct {
|
||||
SonarrAlertMinutes int `json:"sonarrAlertMinutes"`
|
||||
RadarrAlertMinutes int `json:"radarrAlertMinutes"`
|
||||
EmbyHealthSeconds int `json:"embyHealthSeconds"`
|
||||
LibrarySyncMinutes int `json:"librarySyncMinutes"`
|
||||
}
|
||||
|
||||
func (s *Server) deployedSettings() deployedGatewaySettings {
|
||||
@@ -167,6 +177,7 @@ func (s *Server) deployedSettings() deployedGatewaySettings {
|
||||
SonarrAlertMinutes: int(s.cfg.SonarrAlertWindow / time.Minute),
|
||||
RadarrAlertMinutes: int(s.cfg.RadarrAlertWindow / time.Minute),
|
||||
EmbyHealthSeconds: int(s.cfg.EmbyHealthInterval / time.Second),
|
||||
LibrarySyncMinutes: int(s.cfg.SyncInterval / time.Minute),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -184,3 +195,13 @@ func levelName(level slog.Level) string {
|
||||
return "error"
|
||||
}
|
||||
}
|
||||
|
||||
// ingestSettle is what the console prints beside the webhook activity, and it is a helper
|
||||
// for the same reason the others here are: the delay is configuration, and the page must
|
||||
// report the value actually in force rather than the constant it defaults to.
|
||||
func (s *Server) ingestSettle() time.Duration {
|
||||
if s.cfg.IngestSettleDelay > 0 {
|
||||
return s.cfg.IngestSettleDelay
|
||||
}
|
||||
return time.Minute
|
||||
}
|
||||
|
||||
@@ -77,6 +77,20 @@ func (s *Server) RegisterHousekeeping(sched *scheduler.Scheduler) {
|
||||
},
|
||||
})
|
||||
|
||||
sched.Register(scheduler.Task{
|
||||
ID: "ingest-cleanup",
|
||||
Name: "Import queue cleanup",
|
||||
Group: "Housekeeping",
|
||||
Description: fmt.Sprintf(
|
||||
"Removes settled Sonarr and Radarr import records older than %d days. Work still waiting is never removed.",
|
||||
int(store.IngestRetention/(24*time.Hour))),
|
||||
Interval: 24 * time.Hour,
|
||||
Run: func(ctx context.Context) (string, error) {
|
||||
removed, err := s.store.PruneIngests(ctx, store.IngestRetention)
|
||||
return countDetail(removed, "import record"), err
|
||||
},
|
||||
})
|
||||
|
||||
sched.Register(scheduler.Task{
|
||||
ID: "task-history-cleanup",
|
||||
Name: "Task history cleanup",
|
||||
|
||||
@@ -0,0 +1,210 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/ponzischeme89/memby/server/internal/library"
|
||||
)
|
||||
|
||||
// News about a finished Sonarr or Radarr scan.
|
||||
//
|
||||
// The webhook is not the news. Both *arrs fire the moment they have moved a file, and Emby
|
||||
// has not scanned it in yet — which is why the banner this replaces had to say a film would
|
||||
// be available "shortly", and why an episode could not be announced at all: there was
|
||||
// nothing truthful to say about one until it was actually there. The gateway now knows when
|
||||
// that moment arrives, because the ingest worker is what makes it arrive, so the
|
||||
// announcement is made from the far end of the scan and says the title is ready.
|
||||
//
|
||||
// The cost is that the news is a minute or two later than the webhook, and that a title Emby
|
||||
// never manages to scan is never announced. Both are the right way round: a notice about
|
||||
// something a viewer can press Play on is worth more than an earlier one about something
|
||||
// they cannot.
|
||||
|
||||
const (
|
||||
// ingestRunWindow is how long two imports count as one piece of news. A season pack
|
||||
// arrives as a dozen webhooks over a couple of minutes, and a household does not want a
|
||||
// dozen banners about it — it wants to be told the show has new episodes.
|
||||
ingestRunWindow = 15 * time.Minute
|
||||
|
||||
// trackedIngestRuns bounds the tally. A household imports a handful of things at once;
|
||||
// this is generous enough that a season pack always collapses and small enough that it
|
||||
// can never grow into a leak.
|
||||
trackedIngestRuns = 64
|
||||
)
|
||||
|
||||
// AnnounceLibraryIngest turns a completed scan into the banner every open television shows.
|
||||
//
|
||||
// Only a genuine import is announced. An upgrade is deliberately silent — the title was
|
||||
// already there, and "new episode" would be a lie about a file that was replaced with a
|
||||
// better copy — and so are a rename and a delete, which are housekeeping rather than news.
|
||||
// That judgement lives here rather than in the worker: the worker's business is that the
|
||||
// row changed, this is the separate question of whether anybody should be told.
|
||||
func (s *Server) AnnounceLibraryIngest(ctx context.Context, result library.IngestResult) {
|
||||
if result.Reason != library.ReasonImport {
|
||||
return
|
||||
}
|
||||
switch result.Kind {
|
||||
case library.KindMovie:
|
||||
s.announceImportedMovie(ctx, result)
|
||||
case library.KindEpisode:
|
||||
s.announceImportedEpisode(ctx, result)
|
||||
}
|
||||
// A series-level result is a rename settling or a show written ahead of its first
|
||||
// episode. Neither is a title somebody can watch, and the episode that follows is.
|
||||
}
|
||||
|
||||
func (s *Server) announceImportedMovie(ctx context.Context, result library.IngestResult) {
|
||||
window := s.radarrAlertWindow()
|
||||
title := strings.TrimSpace(result.Name)
|
||||
if window <= 0 || title == "" || result.ItemID == "" {
|
||||
return
|
||||
}
|
||||
now := time.Now().UTC()
|
||||
name := title
|
||||
if result.Year > 0 {
|
||||
name = fmt.Sprintf("%s (%d)", title, result.Year)
|
||||
}
|
||||
s.publishAlert(ctx, clientAlert{
|
||||
// Keyed on the Emby item, so a repeated delivery of one import is one banner while
|
||||
// a film deleted and re-imported is news again. Clients dedupe on this id forever.
|
||||
ID: "ingest:movie:" + result.ItemID,
|
||||
Kind: alertKindRadarrImport,
|
||||
Label: "NEW MOVIE ADDED",
|
||||
Title: name,
|
||||
Message: fmt.Sprintf("%s is ready to watch.", title),
|
||||
ItemID: result.ItemID,
|
||||
ImageTag: result.ImageTag,
|
||||
AiredAt: now.Format(time.RFC3339),
|
||||
}, window)
|
||||
s.loggerFor(ctx).Info("library ingest announced",
|
||||
"event", "arr_ingest_alert", "source", result.Source, "kind", result.Kind,
|
||||
"title", name, "item", result.ItemID)
|
||||
}
|
||||
|
||||
func (s *Server) announceImportedEpisode(ctx context.Context, result library.IngestResult) {
|
||||
// The Sonarr window, so MEMBY_SONARR_ALERT_WINDOW=0 switches episode news off exactly
|
||||
// as it does the "aired, coming soon" one, without touching films.
|
||||
window := s.sonarrAlertWindow()
|
||||
series := strings.TrimSpace(result.SeriesName)
|
||||
if window <= 0 || series == "" || result.ItemID == "" {
|
||||
return
|
||||
}
|
||||
now := time.Now().UTC()
|
||||
run := s.ingestRuns.record(
|
||||
seasonRunKey(series, result.Season), result.ItemID,
|
||||
episodeSummary(result), now, ingestRunWindow,
|
||||
)
|
||||
s.publishAlert(ctx, clientAlert{
|
||||
// The run's *first* episode anchors the id, so every later arrival in the same
|
||||
// season pack replaces one banner rather than stacking another — and next week's
|
||||
// episode, arriving after the window has closed, starts a run of its own and is
|
||||
// therefore its own news rather than one the fleet has already dismissed as seen.
|
||||
ID: "ingest:episode:" + run.Anchor,
|
||||
Kind: alertKindSonarrImport,
|
||||
Label: "NEW EPISODE ADDED",
|
||||
Title: series,
|
||||
Message: episodeRunMessage(run),
|
||||
ItemID: result.ItemID,
|
||||
ImageTag: result.ImageTag,
|
||||
AiredAt: now.Format(time.RFC3339),
|
||||
}, window)
|
||||
s.loggerFor(ctx).Info("library ingest announced",
|
||||
"event", "arr_ingest_alert", "source", result.Source, "kind", result.Kind,
|
||||
"series", series, "episodes", run.Count, "item", result.ItemID)
|
||||
}
|
||||
|
||||
// episodeSummary is how one episode is named in a banner: "S03E05 — The Bear". The code
|
||||
// alone is what a viewer scanning a shelf recognises, and the title is what tells them it
|
||||
// is the one they were waiting for, so both are kept where both exist.
|
||||
func episodeSummary(result library.IngestResult) string {
|
||||
code := ""
|
||||
if result.Season > 0 || result.Episode > 0 {
|
||||
code = fmt.Sprintf("S%02dE%02d", result.Season, result.Episode)
|
||||
}
|
||||
title := strings.TrimSpace(result.Name)
|
||||
switch {
|
||||
case code == "":
|
||||
return title
|
||||
case title == "" || strings.EqualFold(title, result.SeriesName):
|
||||
return code
|
||||
default:
|
||||
return fmt.Sprintf("%s — %s", code, title)
|
||||
}
|
||||
}
|
||||
|
||||
// episodeRunMessage words one arrival by name and several by count. Naming the last of six
|
||||
// would be arbitrary — nothing makes it the one worth mentioning — where the count is the
|
||||
// thing the viewer actually wants to know.
|
||||
func episodeRunMessage(run ingestRun) string {
|
||||
if run.Count > 1 {
|
||||
return fmt.Sprintf("%d new episodes are ready to watch.", run.Count)
|
||||
}
|
||||
if run.Latest == "" {
|
||||
return "A new episode is ready to watch."
|
||||
}
|
||||
return fmt.Sprintf("%s is ready to watch.", run.Latest)
|
||||
}
|
||||
|
||||
func seasonRunKey(series string, season int) string {
|
||||
return fmt.Sprintf("%s|%d", library.NormalizedTitle(series), season)
|
||||
}
|
||||
|
||||
// ingestRun is what a season's imports have amounted to so far.
|
||||
type ingestRun struct {
|
||||
// Anchor is the first item id seen in this run, and is what keeps a burst of banners
|
||||
// collapsed onto one.
|
||||
Anchor string
|
||||
Count int
|
||||
Latest string
|
||||
}
|
||||
|
||||
// ingestRuns collapses a burst of imports of one season into a single piece of news.
|
||||
//
|
||||
// Deliberately in memory and deliberately lossy, the playbackTitles arrangement: a gateway
|
||||
// restarted half way through a season pack announces the rest as a second run, which is a
|
||||
// far better trade than a table recording what a banner said.
|
||||
type ingestRuns struct {
|
||||
mu sync.Mutex
|
||||
runs map[string]*runState
|
||||
order []string
|
||||
}
|
||||
|
||||
type runState struct {
|
||||
anchor string
|
||||
count int
|
||||
latest string
|
||||
until time.Time
|
||||
}
|
||||
|
||||
// record folds one import into its season's run and reports where that run now stands. A
|
||||
// run whose window has closed is replaced rather than extended, so a show importing an
|
||||
// episode a week is a separate notice every week.
|
||||
func (r *ingestRuns) record(
|
||||
key, itemID, summary string, now time.Time, window time.Duration,
|
||||
) ingestRun {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if r.runs == nil {
|
||||
r.runs = make(map[string]*runState, trackedIngestRuns)
|
||||
}
|
||||
state, live := r.runs[key]
|
||||
if !live || !state.until.After(now) {
|
||||
if !live {
|
||||
r.order = append(r.order, key)
|
||||
if len(r.order) > trackedIngestRuns {
|
||||
delete(r.runs, r.order[0])
|
||||
r.order = r.order[1:]
|
||||
}
|
||||
}
|
||||
state = &runState{anchor: itemID}
|
||||
r.runs[key] = state
|
||||
}
|
||||
state.count++
|
||||
state.latest = summary
|
||||
state.until = now.Add(window)
|
||||
return ingestRun{Anchor: state.anchor, Count: state.count, Latest: state.latest}
|
||||
}
|
||||
@@ -0,0 +1,155 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/ponzischeme89/memby/server/internal/library"
|
||||
)
|
||||
|
||||
func episodeResult(season, episode int, title string) library.IngestResult {
|
||||
return library.IngestResult{
|
||||
Source: "sonarr",
|
||||
Kind: library.KindEpisode,
|
||||
Reason: library.ReasonImport,
|
||||
ItemID: "emby-" + title,
|
||||
Name: title,
|
||||
SeriesName: "The Bear",
|
||||
Season: season,
|
||||
Episode: episode,
|
||||
}
|
||||
}
|
||||
|
||||
func TestEpisodeSummaryNamesTheEpisodeBothWays(t *testing.T) {
|
||||
if got := episodeSummary(episodeResult(3, 5, "Children")); got != "S03E05 — Children" {
|
||||
t.Errorf("summary = %q, want the code and the title", got)
|
||||
}
|
||||
|
||||
// Emby records plenty of episodes under the show's own name, and "S03E05 — The Bear"
|
||||
// reads as a mistake where the code alone reads as an episode.
|
||||
same := episodeResult(3, 5, "The Bear")
|
||||
if got := episodeSummary(same); got != "S03E05" {
|
||||
t.Errorf("summary = %q, want the code alone when the title repeats the series", got)
|
||||
}
|
||||
|
||||
untitled := episodeResult(3, 5, "")
|
||||
if got := episodeSummary(untitled); got != "S03E05" {
|
||||
t.Errorf("summary = %q, want the code alone", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A season pack is one piece of news. Every arrival replaces the same banner, which is
|
||||
// what the shared anchor is for, and the wording moves from the episode to the count.
|
||||
func TestIngestRunsCollapseASeasonPack(t *testing.T) {
|
||||
var runs ingestRuns
|
||||
now := time.Date(2026, 8, 18, 19, 4, 0, 0, time.UTC)
|
||||
|
||||
first := runs.record("bear|3", "emby-1", "S03E01", now, ingestRunWindow)
|
||||
if first.Count != 1 || first.Anchor != "emby-1" {
|
||||
t.Fatalf("first = %+v, want a run of one anchored on it", first)
|
||||
}
|
||||
if got := episodeRunMessage(first); got != "S03E01 is ready to watch." {
|
||||
t.Errorf("message = %q, want the episode named", got)
|
||||
}
|
||||
|
||||
second := runs.record("bear|3", "emby-2", "S03E02", now.Add(30*time.Second), ingestRunWindow)
|
||||
if second.Anchor != "emby-1" {
|
||||
t.Errorf("anchor = %q, want the run's first episode so the banner is replaced", second.Anchor)
|
||||
}
|
||||
if second.Count != 2 {
|
||||
t.Errorf("count = %d, want 2", second.Count)
|
||||
}
|
||||
if got := episodeRunMessage(second); got != "2 new episodes are ready to watch." {
|
||||
t.Errorf("message = %q, want the count once there is more than one", got)
|
||||
}
|
||||
}
|
||||
|
||||
// Next week's episode is its own news. Televisions dedupe on the alert id forever, so a
|
||||
// run that reused last week's anchor would be silently swallowed on every set in the house.
|
||||
func TestIngestRunsStartAfreshOnceTheWindowHasClosed(t *testing.T) {
|
||||
var runs ingestRuns
|
||||
now := time.Date(2026, 8, 18, 19, 4, 0, 0, time.UTC)
|
||||
|
||||
runs.record("bear|3", "emby-1", "S03E01", now, ingestRunWindow)
|
||||
later := runs.record("bear|3", "emby-2", "S03E02", now.Add(ingestRunWindow+time.Minute), ingestRunWindow)
|
||||
|
||||
if later.Anchor != "emby-2" {
|
||||
t.Errorf("anchor = %q, want a new run", later.Anchor)
|
||||
}
|
||||
if later.Count != 1 {
|
||||
t.Errorf("count = %d, want a run of one", later.Count)
|
||||
}
|
||||
}
|
||||
|
||||
// Two shows importing at once are two pieces of news, not one run of four episodes.
|
||||
func TestIngestRunsAreKeptPerSeason(t *testing.T) {
|
||||
var runs ingestRuns
|
||||
now := time.Date(2026, 8, 18, 19, 4, 0, 0, time.UTC)
|
||||
|
||||
runs.record(seasonRunKey("The Bear", 3), "bear-1", "S03E01", now, ingestRunWindow)
|
||||
other := runs.record(seasonRunKey("Slow Horses", 4), "horses-1", "S04E01", now, ingestRunWindow)
|
||||
|
||||
if other.Count != 1 || other.Anchor != "horses-1" {
|
||||
t.Fatalf("other show = %+v, want a run of its own", other)
|
||||
}
|
||||
// And the same show under a different spelling is still the same show, the rule the
|
||||
// schedule row and the ingest worker already match titles by.
|
||||
same := runs.record(seasonRunKey("the bear!", 3), "bear-2", "S03E02", now, ingestRunWindow)
|
||||
if same.Anchor != "bear-1" || same.Count != 2 {
|
||||
t.Fatalf("same season = %+v, want it folded into the first run", same)
|
||||
}
|
||||
}
|
||||
|
||||
// The tally is memory the gateway can afford to lose, so it must also be memory it cannot
|
||||
// grow without bound.
|
||||
func TestIngestRunsAreBounded(t *testing.T) {
|
||||
var runs ingestRuns
|
||||
now := time.Date(2026, 8, 18, 19, 4, 0, 0, time.UTC)
|
||||
for i := 0; i < trackedIngestRuns*2; i++ {
|
||||
runs.record(time.Duration(i).String(), "item", "S01E01", now, ingestRunWindow)
|
||||
}
|
||||
if len(runs.runs) > trackedIngestRuns {
|
||||
t.Fatalf("tracking %d runs, want a cap of %d", len(runs.runs), trackedIngestRuns)
|
||||
}
|
||||
}
|
||||
|
||||
// Only an import is news. An upgrade replaced a file that was already watchable, and a
|
||||
// rename or a delete is housekeeping — announcing any of them trains viewers to look away.
|
||||
func TestOnlyAnImportIsAnnounced(t *testing.T) {
|
||||
s := &Server{log: discardLogger()}
|
||||
for _, reason := range []string{
|
||||
library.ReasonUpgrade, library.ReasonRename, library.ReasonDelete,
|
||||
} {
|
||||
result := episodeResult(3, 5, "Children")
|
||||
result.Reason = reason
|
||||
// A nil cache would be reached by publishAlert if this announced anything; it
|
||||
// returns early on one, so the assertion is that nothing is recorded either.
|
||||
s.AnnounceLibraryIngest(t.Context(), result)
|
||||
if len(s.ingestRuns.runs) != 0 {
|
||||
t.Fatalf("%s was treated as news", reason)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A series-level result is a rename settling or a show written ahead of its first episode.
|
||||
// Neither is something anybody can press Play on.
|
||||
func TestASeriesRefreshIsNotAnnounced(t *testing.T) {
|
||||
s := &Server{log: discardLogger()}
|
||||
s.AnnounceLibraryIngest(t.Context(), library.IngestResult{
|
||||
Source: "sonarr", Kind: library.KindSeries, Reason: library.ReasonImport,
|
||||
ItemID: "series-1", Name: "The Bear", SeriesName: "The Bear",
|
||||
})
|
||||
if len(s.ingestRuns.runs) != 0 {
|
||||
t.Fatal("a series refresh was announced")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMovieRunMessageSaysTheFilmIsThere(t *testing.T) {
|
||||
// The wording is the whole point of moving the announcement behind the scan: the
|
||||
// banner published from the webhook could only ever promise the film was coming.
|
||||
run := ingestRun{Anchor: "a", Count: 1, Latest: "S01E01"}
|
||||
if strings.Contains(episodeRunMessage(run), "shortly") {
|
||||
t.Error("the message still promises rather than states")
|
||||
}
|
||||
}
|
||||
@@ -61,6 +61,7 @@ func TestComponentNamesThePartOfTheAppARouteBelongsTo(t *testing.T) {
|
||||
"/v1/my-shows": "my-shows",
|
||||
"/admin/api/status": "admin",
|
||||
"/hooks/radarr": "webhooks",
|
||||
"/hooks/sonarr": "webhooks",
|
||||
"/install": "installer",
|
||||
"/updates/latest.apk": "updates",
|
||||
"/something-nobody-has-written": "api",
|
||||
|
||||
@@ -136,6 +136,19 @@ var preferenceCatalogue = []preferenceDefinition{
|
||||
Kind: preferenceChoice, Default: defaultThemeID,
|
||||
Options: themeOptions(),
|
||||
},
|
||||
{
|
||||
// The marks, kept apart from the palette because they are a different decision
|
||||
// about legibility rather than about taste — a household watching from a sofa may
|
||||
// well want the solid pack on every scheme they own.
|
||||
//
|
||||
// The options come from iconPackOptions() in themes.go for the reason themeId's
|
||||
// come from the catalogue: a pack added there cannot become a value this rejects,
|
||||
// and a pack removed cannot stay selectable here.
|
||||
Key: "iconSet", Name: "Icon set", Area: "Presentation",
|
||||
Description: "Which set of marks this viewer's televisions draw.",
|
||||
Kind: preferenceChoice, Default: defaultIconPackID,
|
||||
Options: iconPackOptions(),
|
||||
},
|
||||
{
|
||||
Key: "welcomeQuoteStyle", Name: "Welcome tone", Area: "Presentation",
|
||||
Description: "Tone of the short line shown after signing in.",
|
||||
|
||||
@@ -3,10 +3,10 @@ package api
|
||||
import (
|
||||
"crypto/subtle"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/ponzischeme89/memby/server/internal/library"
|
||||
)
|
||||
|
||||
// Radarr's import notification arrives as a webhook, which is why this is the one part
|
||||
@@ -31,6 +31,30 @@ type radarrWebhookPayload struct {
|
||||
ID int `json:"id"`
|
||||
Quality string `json:"quality"`
|
||||
} `json:"movieFile"`
|
||||
// The delete events carry the file id at the top level rather than under movieFile,
|
||||
// and say whether the media went with the entry. Both are read by the catalogue half
|
||||
// only; the banner has nothing to say about a deletion.
|
||||
MovieFileID int `json:"movieFileId"`
|
||||
DeletedFiles bool `json:"deletedFiles"`
|
||||
}
|
||||
|
||||
// ingestPayload hands the same notification to the catalogue rules.
|
||||
//
|
||||
// Written out rather than shared as one struct because the two halves genuinely read
|
||||
// different fields for different reasons — the banner wants the quality string, the
|
||||
// catalogue wants the deletion flags — and a single type would grow whichever field
|
||||
// either of them needed next.
|
||||
func ingestPayload(payload radarrWebhookPayload) library.RadarrWebhook {
|
||||
var out library.RadarrWebhook
|
||||
out.EventType = payload.EventType
|
||||
out.IsUpgrade = payload.IsUpgrade
|
||||
out.Movie.ID = payload.Movie.ID
|
||||
out.Movie.Title = payload.Movie.Title
|
||||
out.Movie.Year = payload.Movie.Year
|
||||
out.MovieFile.ID = payload.MovieFile.ID
|
||||
out.MovieFileID = payload.MovieFileID
|
||||
out.DeletedFiles = payload.DeletedFiles
|
||||
return out
|
||||
}
|
||||
|
||||
// handleRadarrWebhook accepts Radarr's "On Import" notification.
|
||||
@@ -66,23 +90,20 @@ func (s *Server) handleRadarrWebhook(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
alert, ok := radarrImportAlert(payload, time.Now().UTC())
|
||||
if !ok {
|
||||
// A grab, a rename, a health check or an upgrade of something already in the
|
||||
// library: all real events, none of them "a new film is here".
|
||||
writeJSON(w, http.StatusOK, map[string]any{"ok": true, "announced": false})
|
||||
return
|
||||
// Recording is now the whole of what this hook does. The banner used to be published
|
||||
// from right here, which meant it was published before Emby had scanned the film in —
|
||||
// hence its wording, that the film would be available "shortly". It is announced from
|
||||
// the far end of the scan instead (AnnounceLibraryIngest), where it can say the film is
|
||||
// actually there and where an episode can be announced on the same terms.
|
||||
//
|
||||
// An upgrade is still recorded and still silent as news: the file genuinely changed, so
|
||||
// the row must be re-read, but the film was already there.
|
||||
queued := s.queueIngest(r, "radarr", payload.EventType, library.RadarrRequests(ingestPayload(payload)))
|
||||
if queued > 0 {
|
||||
s.loggerFor(r.Context()).Debug("radarr import recorded",
|
||||
"movie", payload.Movie.Title, "quality", payload.MovieFile.Quality)
|
||||
}
|
||||
window := s.radarrAlertWindow()
|
||||
if window <= 0 {
|
||||
writeJSON(w, http.StatusOK, map[string]any{"ok": true, "announced": false})
|
||||
return
|
||||
}
|
||||
|
||||
s.publishAlert(r.Context(), alert, window)
|
||||
s.loggerFor(r.Context()).Info("radarr import announced",
|
||||
"movie", alert.Title, "alert_id", alert.ID, "quality", payload.MovieFile.Quality)
|
||||
writeJSON(w, http.StatusOK, map[string]any{"ok": true, "announced": true})
|
||||
writeJSON(w, http.StatusOK, map[string]any{"ok": true, "queued": queued})
|
||||
}
|
||||
|
||||
// webhookToken accepts the shared secret three ways because Radarr's webhook settings
|
||||
@@ -100,53 +121,3 @@ func webhookToken(r *http.Request) string {
|
||||
}
|
||||
return strings.TrimSpace(r.URL.Query().Get("token"))
|
||||
}
|
||||
|
||||
// radarrImportAlert turns an import notification into the banner a TV shows, or reports
|
||||
// that this event is not worth announcing.
|
||||
//
|
||||
// An upgrade is deliberately silent: the film was already there, and "new movie added"
|
||||
// would be a lie about a file that was replaced with a better copy.
|
||||
func radarrImportAlert(payload radarrWebhookPayload, now time.Time) (clientAlert, bool) {
|
||||
if !isRadarrImportEvent(payload.EventType) || payload.IsUpgrade {
|
||||
return clientAlert{}, false
|
||||
}
|
||||
title := strings.TrimSpace(payload.Movie.Title)
|
||||
if title == "" || payload.Movie.ID <= 0 {
|
||||
return clientAlert{}, false
|
||||
}
|
||||
|
||||
// Keyed on the file, so a title deleted and re-imported is news again while a
|
||||
// repeated delivery of the same import is not. Clients dedupe on this id forever.
|
||||
id := fmt.Sprintf("radarr:%d:file:%d", payload.Movie.ID, payload.MovieFile.ID)
|
||||
if payload.MovieFile.ID <= 0 {
|
||||
id = fmt.Sprintf("radarr:%d:imported:%d", payload.Movie.ID, now.Unix())
|
||||
}
|
||||
|
||||
name := title
|
||||
if payload.Movie.Year > 0 {
|
||||
name = fmt.Sprintf("%s (%d)", title, payload.Movie.Year)
|
||||
}
|
||||
return clientAlert{
|
||||
ID: id,
|
||||
Kind: alertKindRadarrImport,
|
||||
Label: "NEW MOVIE ADDED",
|
||||
Title: name,
|
||||
Message: fmt.Sprintf("%s will be available in Emby shortly.", title),
|
||||
// The image proxy already serves Radarr covers under this id and tag, so the
|
||||
// banner shows the poster before Emby has finished scanning the film in.
|
||||
ItemID: fmt.Sprintf("radarr:%d", payload.Movie.ID),
|
||||
ImageTag: "radarr",
|
||||
AiredAt: now.UTC().Format(time.RFC3339),
|
||||
}, true
|
||||
}
|
||||
|
||||
// isRadarrImportEvent matches the event Radarr fires once a downloaded file has been
|
||||
// imported into the library. The name has moved between versions, so both are accepted.
|
||||
func isRadarrImportEvent(eventType string) bool {
|
||||
switch strings.ToLower(strings.TrimSpace(eventType)) {
|
||||
case "download", "moviefileimported":
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
@@ -12,96 +12,6 @@ import (
|
||||
"github.com/ponzischeme89/memby/server/internal/config"
|
||||
)
|
||||
|
||||
func importPayload(movieID, fileID int, title string, year int) radarrWebhookPayload {
|
||||
var payload radarrWebhookPayload
|
||||
payload.EventType = "Download"
|
||||
payload.Movie.ID = movieID
|
||||
payload.Movie.Title = title
|
||||
payload.Movie.Year = year
|
||||
payload.MovieFile.ID = fileID
|
||||
return payload
|
||||
}
|
||||
|
||||
func TestRadarrImportAlertAnnouncesANewFilm(t *testing.T) {
|
||||
now := time.Date(2026, 7, 31, 19, 4, 0, 0, time.UTC)
|
||||
|
||||
alert, ok := radarrImportAlert(importPayload(412, 9001, "Mr. Smith Goes to Washington", 1939), now)
|
||||
if !ok {
|
||||
t.Fatal("expected an import to be announced")
|
||||
}
|
||||
if alert.ID != "radarr:412:file:9001" {
|
||||
t.Errorf("alert id = %q, want it keyed on the imported file", alert.ID)
|
||||
}
|
||||
if alert.Kind != alertKindRadarrImport {
|
||||
t.Errorf("kind = %q, want %q", alert.Kind, alertKindRadarrImport)
|
||||
}
|
||||
if alert.Label == "" {
|
||||
t.Error("want a label: the app cannot know the wording for a kind it predates")
|
||||
}
|
||||
if alert.Title != "Mr. Smith Goes to Washington (1939)" {
|
||||
t.Errorf("title = %q, want the year alongside it", alert.Title)
|
||||
}
|
||||
if !strings.Contains(alert.Message, "available in Emby shortly") {
|
||||
t.Errorf("message = %q, want it to say the film is on its way", alert.Message)
|
||||
}
|
||||
// The image proxy serves Radarr covers under this pair, so the banner has a poster
|
||||
// before Emby has scanned the film in.
|
||||
if alert.ItemID != "radarr:412" || alert.ImageTag != "radarr" {
|
||||
t.Errorf("artwork = %q/%q, want the radarr media cover", alert.ItemID, alert.ImageTag)
|
||||
}
|
||||
if alert.AiredAt != now.Format(time.RFC3339) {
|
||||
t.Errorf("airedAt = %q, want the import time so it sorts with the rest", alert.AiredAt)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRadarrImportAlertIgnoresEventsThatAreNotANewFilm(t *testing.T) {
|
||||
now := time.Date(2026, 7, 31, 19, 4, 0, 0, time.UTC)
|
||||
|
||||
upgrade := importPayload(412, 9002, "Mr. Smith Goes to Washington", 1939)
|
||||
upgrade.IsUpgrade = true
|
||||
|
||||
grab := importPayload(413, 0, "Some Film", 2024)
|
||||
grab.EventType = "Grab"
|
||||
|
||||
untitled := importPayload(414, 9003, " ", 2024)
|
||||
|
||||
unknownMovie := importPayload(0, 9004, "No Id", 2024)
|
||||
|
||||
for name, payload := range map[string]radarrWebhookPayload{
|
||||
"quality upgrade of a film already there": upgrade,
|
||||
"grabbed but not imported": grab,
|
||||
"no title": untitled,
|
||||
"no movie id": unknownMovie,
|
||||
} {
|
||||
if _, ok := radarrImportAlert(payload, now); ok {
|
||||
t.Errorf("%s: expected no alert", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRadarrImportAlertAcceptsTheNewerEventName(t *testing.T) {
|
||||
now := time.Date(2026, 7, 31, 19, 4, 0, 0, time.UTC)
|
||||
payload := importPayload(415, 9005, "Rear Window", 1954)
|
||||
payload.EventType = "MovieFileImported"
|
||||
|
||||
if _, ok := radarrImportAlert(payload, now); !ok {
|
||||
t.Error("expected the alternate import event name to be announced")
|
||||
}
|
||||
}
|
||||
|
||||
// A file id is what makes a repeated notification the same news; without one the id
|
||||
// falls back to the clock so a re-import is not silently swallowed.
|
||||
func TestRadarrImportAlertWithoutAFileIDIsStillAnnounced(t *testing.T) {
|
||||
now := time.Date(2026, 7, 31, 19, 4, 0, 0, time.UTC)
|
||||
alert, ok := radarrImportAlert(importPayload(416, 0, "Sabotage", 1936), now)
|
||||
if !ok {
|
||||
t.Fatal("expected an alert")
|
||||
}
|
||||
if !strings.HasPrefix(alert.ID, "radarr:416:imported:") {
|
||||
t.Errorf("alert id = %q, want a time-keyed fallback", alert.ID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAppendAlertPrunesExpiredAndDeduplicates(t *testing.T) {
|
||||
now := time.Date(2026, 7, 31, 19, 4, 0, 0, time.UTC)
|
||||
existing := []storedAlert{
|
||||
|
||||
@@ -68,6 +68,39 @@ const (
|
||||
decorationBlossom = "blossom"
|
||||
)
|
||||
|
||||
// The icon packs a theme may draw its marks from. Slugs, for the reason the decorations
|
||||
// above are slugs: the shapes live on the television, in ui/theme/MembyIconPacks.kt, and
|
||||
// the gateway has no business describing geometry to it. A client that does not recognise
|
||||
// one draws the marks it shipped with — so this list may gain a pack before the fleet has
|
||||
// the build that knows it, the MembyHeroLabel precedent again.
|
||||
//
|
||||
// The reason to want any of this is that Material's marks are the marks every Android app
|
||||
// on the television already wears. Moving a household off them is a decision an operator
|
||||
// should be able to make in the gateway, not one that waits on an APK reaching every set.
|
||||
const (
|
||||
iconPackMaterial = "material"
|
||||
iconPackLucide = "lucide"
|
||||
iconPackFontAwesome = "fontawesome"
|
||||
)
|
||||
|
||||
// defaultIconPackID is the marks the app shipped with, so nothing changes appearance on the
|
||||
// day this lands.
|
||||
const defaultIconPackID = iconPackMaterial
|
||||
|
||||
// iconPackOptions is the vocabulary of the `iconSet` preference, and the only place a pack
|
||||
// is declared. A pack this list does not name is one no viewer and no operator can select.
|
||||
//
|
||||
// The wording is about how the marks read from a sofa, because that is the whole of the
|
||||
// choice: stroke sets are drawn for 16-24px on a monitor, and on a rail chip at three
|
||||
// metres they go thin where a solid mark keeps its shape.
|
||||
func iconPackOptions() []preferenceOption {
|
||||
return []preferenceOption{
|
||||
option(iconPackMaterial, "Material — Android's own marks"),
|
||||
option(iconPackLucide, "Lucide — lighter, drawn as outlines"),
|
||||
option(iconPackFontAwesome, "Font Awesome — solid, clearest at a distance"),
|
||||
}
|
||||
}
|
||||
|
||||
type themeDefinition struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
@@ -83,6 +116,14 @@ type themeDefinition struct {
|
||||
// it. It is empty on every selectable theme by construction rather than by a check at
|
||||
// the point of use.
|
||||
Decoration string `json:"decoration,omitempty"`
|
||||
// IconSet lets a theme bring its own marks, and like Decoration only a seasonal theme
|
||||
// carries one. Empty means the viewer's own choice stands.
|
||||
//
|
||||
// The asymmetry with the palette is deliberate. A season *is* a look, so it may say
|
||||
// what the marks are; a scheme somebody picked to live with all year must not silently
|
||||
// take their marks away, because there would be no way to tell which of the two
|
||||
// choices had done it.
|
||||
IconSet string `json:"iconSet,omitempty"`
|
||||
}
|
||||
|
||||
const (
|
||||
@@ -164,6 +205,7 @@ var themeCatalogue = []themeDefinition{
|
||||
ID: themeHalloween, Name: "Halloween", Seasonal: true,
|
||||
Description: "Pumpkin orange on black, for the last week of October.",
|
||||
Decoration: decorationBats,
|
||||
IconSet: iconPackFontAwesome,
|
||||
Palette: themePalette{
|
||||
Surface: "#FF0A0704", SurfaceRaised: "#FF17100A", Accent: "#FFFF8A1F",
|
||||
OnSurface: "#FFF2E7DA", MutedText: "#FFE2D2BE", QuietText: "#FFBBA48C",
|
||||
@@ -341,6 +383,12 @@ type resolvedTheme struct {
|
||||
// theme by the television — a set holding a cached Christmas palette must not keep
|
||||
// snowing after the switch has been thrown.
|
||||
Decoration string `json:"decoration,omitempty"`
|
||||
// IconSet is the pack the television draws its marks from: "material", "lucide",
|
||||
// "fontawesome". Resolved here rather than derived on the set from the theme id,
|
||||
// exactly as Decoration is — a television holding a cached seasonal answer must stop
|
||||
// using that season's marks when the switch is thrown, and it has no way to know that
|
||||
// on its own.
|
||||
IconSet string `json:"iconSet,omitempty"`
|
||||
// Reason is the sentence the picker prints while it is locked. The gateway's wording,
|
||||
// the MembyHeroLabel precedent, so a season invented later reads correctly on today's
|
||||
// build rather than as a blank space where an explanation should be.
|
||||
@@ -364,6 +412,7 @@ type resolvedTheme struct {
|
||||
// a season. The only switch is seasonalEnabled, and that is the operator's feature flag.
|
||||
func resolveTheme(
|
||||
chosen string,
|
||||
chosenIconPack string,
|
||||
allowed []string,
|
||||
seasonalEnabled bool,
|
||||
decorationsEnabled bool,
|
||||
@@ -394,15 +443,38 @@ func resolveTheme(
|
||||
decoration = applied.Decoration
|
||||
}
|
||||
|
||||
// The viewer's marks, unless the season brought its own. Note that this is *not* gated
|
||||
// on decorationsEnabled: that switch is about the cost of a continuous animation on a
|
||||
// weak box, and a set of icons costs nothing to draw.
|
||||
iconPack := chosenIconPack
|
||||
if !knownIconPack(iconPack) {
|
||||
iconPack = defaultIconPackID
|
||||
}
|
||||
if seasonal && applied.IconSet != "" {
|
||||
iconPack = applied.IconSet
|
||||
}
|
||||
|
||||
resolved := resolvedTheme{
|
||||
ID: applied.ID, Name: applied.Name, Palette: applied.Palette,
|
||||
Seasonal: seasonal, Locked: seasonal, Chosen: pick.ID, Reason: reason,
|
||||
Decoration: decoration,
|
||||
Decoration: decoration, IconSet: iconPack,
|
||||
}
|
||||
resolved.Revision = themeRevision(resolved)
|
||||
return resolved
|
||||
}
|
||||
|
||||
// knownIconPack keeps an unreadable or retired slug out of the answer. The television
|
||||
// would fall back on its own — membyIconPackFor answers with Material for anything it does
|
||||
// not know — but a gateway that echoed a pack nobody can draw would make every set in the
|
||||
// house look broken in the same way while reporting that it had done what it was asked.
|
||||
func knownIconPack(id string) bool {
|
||||
switch id {
|
||||
case iconPackMaterial, iconPackLucide, iconPackFontAwesome:
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// themeAllowed applies the operator's per-user list. An empty list is *permissive*: no row
|
||||
// has ever been written for the great majority of households, and reading that as "this
|
||||
// person may have no themes" would empty every picker in the house the day this ships.
|
||||
@@ -422,7 +494,7 @@ func themeRevision(resolved resolvedTheme) string {
|
||||
for _, part := range []string{
|
||||
strconv.Itoa(themeSchemaVersion), resolved.ID, resolved.Chosen,
|
||||
strconv.FormatBool(resolved.Seasonal), strconv.FormatBool(resolved.Locked), resolved.Reason,
|
||||
resolved.Decoration,
|
||||
resolved.Decoration, resolved.IconSet,
|
||||
palette.Surface, palette.SurfaceRaised, palette.Accent, palette.OnSurface,
|
||||
palette.MutedText, palette.QuietText, palette.Hairline, palette.RatingsSurface,
|
||||
} {
|
||||
@@ -442,12 +514,17 @@ func themeRevision(resolved resolvedTheme) string {
|
||||
// palette it falls back to is the one the app shipped with.
|
||||
func (s *Server) themeFor(ctx context.Context, sess store.Session) resolvedTheme {
|
||||
chosen, _ := preferenceDefault("themeId").(string)
|
||||
iconPack, _ := preferenceDefault("iconSet").(string)
|
||||
allowed := []string(nil)
|
||||
if s.store != nil && sess.EmbyUserID != "" {
|
||||
if stored, err := s.store.UserPreferences(ctx, sess.EmbyUserID); err == nil {
|
||||
if value, ok := decodePreferences(stored.Preferences)["themeId"].(string); ok {
|
||||
decoded := decodePreferences(stored.Preferences)
|
||||
if value, ok := decoded["themeId"].(string); ok {
|
||||
chosen = value
|
||||
}
|
||||
if value, ok := decoded["iconSet"].(string); ok {
|
||||
iconPack = value
|
||||
}
|
||||
} else {
|
||||
s.loggerFor(ctx).Warn("theme preference unavailable", "error", err)
|
||||
}
|
||||
@@ -458,7 +535,7 @@ func (s *Server) themeFor(ctx context.Context, sess store.Session) resolvedTheme
|
||||
}
|
||||
}
|
||||
return resolveTheme(
|
||||
chosen, allowed,
|
||||
chosen, iconPack, allowed,
|
||||
s.featureEnabled(ctx, featureSeasonalThemes),
|
||||
s.featureEnabled(ctx, featureSeasonalDecorations),
|
||||
s.now(),
|
||||
|
||||
@@ -77,7 +77,7 @@ func TestEasterWindowIsTheLongWeekend(t *testing.T) {
|
||||
// A season is the one thing on this feature nobody on a television can decline, so the
|
||||
// tests that matter most are the ones asserting that no argument suppresses it.
|
||||
func TestSeasonOutranksTheViewer(t *testing.T) {
|
||||
resolved := resolveTheme(themePlum, nil, true, true, date(2026, time.December, 20))
|
||||
resolved := resolveTheme(themePlum, "", nil, true, true, date(2026, time.December, 20))
|
||||
if resolved.ID != themeChristmas {
|
||||
t.Fatalf("applied theme = %q, want %q", resolved.ID, themeChristmas)
|
||||
}
|
||||
@@ -98,7 +98,7 @@ func TestSeasonOutranksTheViewer(t *testing.T) {
|
||||
// grantable per person, and an operator restricting a viewer to one palette must not be a
|
||||
// way of exempting them from Christmas.
|
||||
func TestAllowlistDoesNotApplyToSeasons(t *testing.T) {
|
||||
resolved := resolveTheme(themePlum, []string{themeEmber}, true, true, date(2026, time.October, 31))
|
||||
resolved := resolveTheme(themePlum, "", []string{themeEmber}, true, true, date(2026, time.October, 31))
|
||||
if resolved.ID != themeHalloween {
|
||||
t.Fatalf("applied theme = %q, want %q", resolved.ID, themeHalloween)
|
||||
}
|
||||
@@ -110,7 +110,7 @@ func TestAllowlistDoesNotApplyToSeasons(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestSeasonsOffLeavesTheViewersChoice(t *testing.T) {
|
||||
resolved := resolveTheme(themeEmber, nil, false, true, date(2026, time.December, 20))
|
||||
resolved := resolveTheme(themeEmber, "", nil, false, true, date(2026, time.December, 20))
|
||||
if resolved.ID != themeEmber {
|
||||
t.Fatalf("applied theme = %q, want %q", resolved.ID, themeEmber)
|
||||
}
|
||||
@@ -138,7 +138,7 @@ func TestResolveThemeFallbacks(t *testing.T) {
|
||||
}
|
||||
for _, testCase := range cases {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
got := resolveTheme(testCase.chosen, testCase.allowed, true, true, ordinary)
|
||||
got := resolveTheme(testCase.chosen, "", testCase.allowed, true, true, ordinary)
|
||||
if got.ID != testCase.want {
|
||||
t.Fatalf("resolveTheme(%q, %v) = %q, want %q",
|
||||
testCase.chosen, testCase.allowed, got.ID, testCase.want)
|
||||
@@ -150,14 +150,14 @@ func TestResolveThemeFallbacks(t *testing.T) {
|
||||
// The revision is the whole delivery mechanism: a television refetches the palette only when
|
||||
// this moves. If it did not move when a season began, no set in the house would repaint.
|
||||
func TestThemeRevisionTracksTheAnswer(t *testing.T) {
|
||||
ordinary := resolveTheme(themePlum, nil, true, true, date(2026, time.June, 14))
|
||||
christmas := resolveTheme(themePlum, nil, true, true, date(2026, time.December, 20))
|
||||
ordinary := resolveTheme(themePlum, "", nil, true, true, date(2026, time.June, 14))
|
||||
christmas := resolveTheme(themePlum, "", nil, true, true, date(2026, time.December, 20))
|
||||
if ordinary.Revision == christmas.Revision {
|
||||
t.Fatal("the revision must change when the season does, or nothing refetches")
|
||||
}
|
||||
// And it must be stable, or every poll would look like a change and every set would
|
||||
// fetch the palette six times a minute.
|
||||
again := resolveTheme(themePlum, nil, true, true, date(2026, time.June, 15))
|
||||
again := resolveTheme(themePlum, "", nil, true, true, date(2026, time.June, 15))
|
||||
if ordinary.Revision != again.Revision {
|
||||
t.Fatalf("the revision moved on an ordinary day: %s then %s", ordinary.Revision, again.Revision)
|
||||
}
|
||||
@@ -168,10 +168,10 @@ func TestThemeRevisionTracksTheAnswer(t *testing.T) {
|
||||
// every day of the year.
|
||||
func TestDecorationsAreSeasonalAndSeparatelySwitchable(t *testing.T) {
|
||||
christmas := date(2026, time.December, 20)
|
||||
if got := resolveTheme(themePlum, nil, true, true, christmas); got.Decoration != decorationSnow {
|
||||
if got := resolveTheme(themePlum, "", nil, true, true, christmas); got.Decoration != decorationSnow {
|
||||
t.Fatalf("decoration = %q, want %q", got.Decoration, decorationSnow)
|
||||
}
|
||||
off := resolveTheme(themePlum, nil, true, false, christmas)
|
||||
off := resolveTheme(themePlum, "", nil, true, false, christmas)
|
||||
if off.Decoration != "" {
|
||||
t.Fatalf("decorations off still returned %q", off.Decoration)
|
||||
}
|
||||
@@ -179,10 +179,10 @@ func TestDecorationsAreSeasonalAndSeparatelySwitchable(t *testing.T) {
|
||||
t.Fatal("turning decorations off must keep the seasonal palette")
|
||||
}
|
||||
// The revision has to move, or a set already snowing is never told to stop.
|
||||
if off.Revision == resolveTheme(themePlum, nil, true, true, christmas).Revision {
|
||||
if off.Revision == resolveTheme(themePlum, "", nil, true, true, christmas).Revision {
|
||||
t.Fatal("the revision must change when the decoration does")
|
||||
}
|
||||
ordinary := resolveTheme(themePlum, nil, true, true, date(2026, time.June, 14))
|
||||
ordinary := resolveTheme(themePlum, "", nil, true, true, date(2026, time.June, 14))
|
||||
if ordinary.Decoration != "" {
|
||||
t.Fatalf("a chosen theme carries a decoration: %q", ordinary.Decoration)
|
||||
}
|
||||
@@ -277,3 +277,88 @@ func TestNormalizeThemeAllowlist(t *testing.T) {
|
||||
t.Fatalf("allowlist is not in catalogue order: %v", ordered)
|
||||
}
|
||||
}
|
||||
|
||||
// --- Icon packs ---------------------------------------------------------------------------
|
||||
|
||||
// The gateway's half of the wire contract with ui/theme/MembyIconPacks.kt. Its own
|
||||
// MembyIconPackTest pins the same three slugs from the television's end; change one
|
||||
// without the other and a household is sent marks nothing can draw.
|
||||
func TestIconPackOptionsAreTheKnownPacks(t *testing.T) {
|
||||
for _, option := range iconPackOptions() {
|
||||
if !knownIconPack(option.Value) {
|
||||
t.Fatalf("offered icon pack %q is not one the resolver will accept", option.Value)
|
||||
}
|
||||
}
|
||||
if !knownIconPack(defaultIconPackID) {
|
||||
t.Fatalf("the default icon pack %q is not selectable", defaultIconPackID)
|
||||
}
|
||||
}
|
||||
|
||||
// A slug the resolver does not recognise must never reach a television. The set would fall
|
||||
// back on its own, but a gateway echoing a retired pack would make every screen in the
|
||||
// house look wrong in the same way while reporting it had done what it was asked.
|
||||
func TestUnknownIconPackFallsBackToTheShippedMarks(t *testing.T) {
|
||||
ordinary := time.Date(2026, time.May, 3, 12, 0, 0, 0, time.UTC)
|
||||
for _, chosen := range []string{"", "tabler", "material "} {
|
||||
got := resolveTheme(themeMidnight, chosen, nil, true, true, ordinary)
|
||||
if got.IconSet != defaultIconPackID {
|
||||
t.Fatalf("resolveTheme(icon %q) = %q, want %q", chosen, got.IconSet, defaultIconPackID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The viewer's marks stand on every scheme they can choose, and only a season may replace
|
||||
// them — the same asymmetry decorations have, and for the same reason: a season is a look,
|
||||
// where a scheme somebody picked to live with all year must not silently take their marks
|
||||
// away with no way to tell which choice did it.
|
||||
func TestOnlyASeasonMayReplaceTheViewersMarks(t *testing.T) {
|
||||
ordinary := time.Date(2026, time.May, 3, 12, 0, 0, 0, time.UTC)
|
||||
if got := resolveTheme(themePlum, iconPackLucide, nil, true, true, ordinary); got.IconSet != iconPackLucide {
|
||||
t.Fatalf("an ordinary day = %q, want the viewer's %q", got.IconSet, iconPackLucide)
|
||||
}
|
||||
|
||||
halloween := time.Date(2026, time.October, 30, 20, 0, 0, 0, time.UTC)
|
||||
got := resolveTheme(themePlum, iconPackLucide, nil, true, true, halloween)
|
||||
if !got.Seasonal {
|
||||
t.Fatalf("expected the Halloween window to be seasonal")
|
||||
}
|
||||
if got.IconSet != iconPackFontAwesome {
|
||||
t.Fatalf("Halloween = %q, want its own %q", got.IconSet, iconPackFontAwesome)
|
||||
}
|
||||
if got.Chosen != themePlum {
|
||||
t.Fatalf("the viewer's own choice should survive underneath a season, got %q", got.Chosen)
|
||||
}
|
||||
|
||||
// Seasons off: the viewer keeps both halves.
|
||||
if off := resolveTheme(themePlum, iconPackLucide, nil, false, true, halloween); off.IconSet != iconPackLucide {
|
||||
t.Fatalf("with seasons off = %q, want the viewer's %q", off.IconSet, iconPackLucide)
|
||||
}
|
||||
}
|
||||
|
||||
// Only seasonal themes may declare marks of their own, the rule that holds the asymmetry
|
||||
// above in place by construction rather than by a check at the point of use.
|
||||
func TestOnlySeasonalThemesDeclareAnIconSet(t *testing.T) {
|
||||
for _, theme := range themeCatalogue {
|
||||
if theme.Seasonal {
|
||||
if theme.IconSet != "" && !knownIconPack(theme.IconSet) {
|
||||
t.Fatalf("%s names an icon pack %q nothing can draw", theme.ID, theme.IconSet)
|
||||
}
|
||||
continue
|
||||
}
|
||||
if theme.IconSet != "" {
|
||||
t.Fatalf("selectable theme %s takes the viewer's marks away", theme.ID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The revision is the entire delivery mechanism: televisions compare it and refetch only
|
||||
// when it moves. A pack change the hash did not notice would reach nobody until something
|
||||
// else about the theme happened to change.
|
||||
func TestThemeRevisionTracksTheIconSet(t *testing.T) {
|
||||
ordinary := time.Date(2026, time.May, 3, 12, 0, 0, 0, time.UTC)
|
||||
material := resolveTheme(themeMidnight, iconPackMaterial, nil, true, true, ordinary)
|
||||
lucide := resolveTheme(themeMidnight, iconPackLucide, nil, true, true, ordinary)
|
||||
if material.Revision == lucide.Revision {
|
||||
t.Fatalf("the same revision %q for two different icon packs", material.Revision)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1 +1 @@
|
||||
0.1.53
|
||||
0.1.55
|
||||
|
||||
@@ -131,6 +131,15 @@ type Config struct {
|
||||
// touching the five-day schedule row.
|
||||
SonarrAlertWindow time.Duration
|
||||
|
||||
// SonarrWebhookToken guards the Sonarr import/upgrade/rename/delete webhook. Empty
|
||||
// means the hook 404s, the stance the Radarr one takes.
|
||||
SonarrWebhookToken string
|
||||
|
||||
// IngestSettleDelay is how long after a webhook the gateway first looks for the file
|
||||
// in Emby. Sonarr fires the moment it has moved the file into place and Emby has not
|
||||
// scanned it yet, so asking immediately spends a request to learn nothing.
|
||||
IngestSettleDelay time.Duration
|
||||
|
||||
// Radarr is optional. Its calendar supplies the five-day digital movie release row.
|
||||
RadarrURL string
|
||||
RadarrAPIKey string
|
||||
@@ -247,6 +256,8 @@ func Load() (Config, error) {
|
||||
SonarrAPIKey: strings.TrimSpace(os.Getenv("MEMBY_SONARR_API_KEY")),
|
||||
SonarrTTL: duration("MEMBY_SONARR_TTL", 5*time.Minute),
|
||||
SonarrAlertWindow: duration("MEMBY_SONARR_ALERT_WINDOW", 3*time.Hour),
|
||||
SonarrWebhookToken: strings.TrimSpace(os.Getenv("MEMBY_SONARR_WEBHOOK_TOKEN")),
|
||||
IngestSettleDelay: duration("MEMBY_ARR_INGEST_SETTLE", time.Minute),
|
||||
RadarrURL: strings.TrimRight(strings.TrimSpace(os.Getenv("MEMBY_RADARR_URL")), "/"),
|
||||
RadarrAPIKey: strings.TrimSpace(os.Getenv("MEMBY_RADARR_API_KEY")),
|
||||
RadarrTTL: duration("MEMBY_RADARR_TTL", 5*time.Minute),
|
||||
|
||||
@@ -0,0 +1,355 @@
|
||||
package library
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Turning a Sonarr or Radarr notification into a piece of work, and all of it pure.
|
||||
//
|
||||
// The gateway used to learn that a file had appeared by asking Emby every hour whether
|
||||
// anything had been saved since the last time it asked. Sonarr and Radarr already know —
|
||||
// they are the things that put the file there — so this is the translation from what they
|
||||
// say into the one question the ingest worker answers: which item should be re-read, or
|
||||
// removed, and how do two deliveries of the same news collapse into one.
|
||||
//
|
||||
// Nothing here does I/O, which is what lets every rule below be a table test.
|
||||
|
||||
// Ingest actions. A rename is deliberately a refresh like any other: the Emby item id
|
||||
// survives a file being moved, and so does the credits fingerprint measured against it —
|
||||
// the only stale thing is the row's payload.
|
||||
const (
|
||||
ActionRefresh = "refresh"
|
||||
ActionRemove = "remove"
|
||||
)
|
||||
|
||||
// Ingest kinds.
|
||||
const (
|
||||
KindEpisode = "episode"
|
||||
KindMovie = "movie"
|
||||
KindSeries = "series"
|
||||
)
|
||||
|
||||
// Why a request exists. It is carried through to the log and the console, because "this
|
||||
// episode was re-read because Sonarr upgraded the file" is the sentence an operator needs
|
||||
// and "an item changed" is not.
|
||||
const (
|
||||
ReasonImport = "import"
|
||||
ReasonUpgrade = "upgrade"
|
||||
ReasonRename = "rename"
|
||||
ReasonDelete = "delete"
|
||||
)
|
||||
|
||||
// IngestRequest is one piece of work. It carries what the *arr knew rather than an Emby
|
||||
// id, because at the moment a webhook arrives Emby has very often not scanned the file in
|
||||
// yet and there is no id to carry.
|
||||
type IngestRequest struct {
|
||||
// Key is the dedupe identity, and it names the *file* rather than the event. Two
|
||||
// deliveries of one import collapse onto one row; a file deleted and re-imported is a
|
||||
// different file and therefore its own work. Same reasoning as the alert id in
|
||||
// radarrImportAlert.
|
||||
Key string `json:"-"`
|
||||
Action string `json:"-"`
|
||||
Kind string `json:"-"`
|
||||
Reason string `json:"-"`
|
||||
|
||||
// Series identity, for an episode or a series-level event.
|
||||
Series string `json:"series,omitempty"`
|
||||
SeriesYear int `json:"seriesYear,omitempty"`
|
||||
Season int `json:"season,omitempty"`
|
||||
Episode int `json:"episode,omitempty"`
|
||||
|
||||
// Film identity.
|
||||
Title string `json:"title,omitempty"`
|
||||
Year int `json:"year,omitempty"`
|
||||
|
||||
// EmbyItemID is filled in only where the caller already knows it — a delete of
|
||||
// something the catalogue holds. Empty is the ordinary case.
|
||||
EmbyItemID string `json:"embyItemId,omitempty"`
|
||||
}
|
||||
|
||||
// IngestResult is a finished piece of ingest work, handed to whoever wants to announce it.
|
||||
//
|
||||
// It is what makes "a scan has completed" a thing the gateway can say: a webhook only means
|
||||
// the *arr has moved a file, and the several minutes between that and Emby having scanned
|
||||
// it in are exactly the minutes in which a banner saying the title is there would be wrong.
|
||||
// This is emitted from the other end, once the row is in the catalogue.
|
||||
//
|
||||
// It carries both what the *arr said and what Emby turned out to call the thing, because
|
||||
// the news is about the title and the item id is what can put artwork behind it.
|
||||
type IngestResult struct {
|
||||
Source string // sonarr | radarr
|
||||
Kind string // KindEpisode | KindMovie | KindSeries
|
||||
Reason string // ReasonImport | ReasonUpgrade | ReasonRename | ReasonDelete
|
||||
|
||||
// ItemID and Name are Emby's, filled in from the row that was just written. ItemID is
|
||||
// empty for a series-wide refresh, which is about a show rather than about one file.
|
||||
ItemID string
|
||||
Name string
|
||||
ImageTag string
|
||||
|
||||
// SeriesName is Emby's name for the show an episode belongs to, which is what a banner
|
||||
// leads with — the episode's own Name is its title.
|
||||
SeriesName string
|
||||
Season int
|
||||
Episode int
|
||||
Year int
|
||||
}
|
||||
|
||||
// SonarrWebhook is the subset of Sonarr's body this reads. Sonarr sends considerably
|
||||
// more; anything not named here is ignored on purpose, so a Sonarr upgrade that adds
|
||||
// fields cannot break the hook.
|
||||
type SonarrWebhook struct {
|
||||
EventType string `json:"eventType"`
|
||||
Series struct {
|
||||
ID int `json:"id"`
|
||||
Title string `json:"title"`
|
||||
Year int `json:"year"`
|
||||
} `json:"series"`
|
||||
Episodes []struct {
|
||||
ID int `json:"id"`
|
||||
SeasonNumber int `json:"seasonNumber"`
|
||||
EpisodeNumber int `json:"episodeNumber"`
|
||||
} `json:"episodes"`
|
||||
EpisodeFile struct {
|
||||
ID int `json:"id"`
|
||||
SeasonNumber int `json:"seasonNumber"`
|
||||
} `json:"episodeFile"`
|
||||
// RenamedEpisodeFiles is what On Rename carries: the files that moved, each with the
|
||||
// id the library already knows them by.
|
||||
RenamedEpisodeFiles []struct {
|
||||
ID int `json:"id"`
|
||||
SeasonNumber int `json:"seasonNumber"`
|
||||
} `json:"renamedEpisodeFiles"`
|
||||
IsUpgrade bool `json:"isUpgrade"`
|
||||
// DeletedFiles marks a series delete that took the media with it. A series removed
|
||||
// from Sonarr's list while its files stay on disk is not a reason to forget it.
|
||||
DeletedFiles bool `json:"deletedFiles"`
|
||||
}
|
||||
|
||||
// RadarrWebhook is the same narrow reading of Radarr's body.
|
||||
type RadarrWebhook struct {
|
||||
EventType string `json:"eventType"`
|
||||
IsUpgrade bool `json:"isUpgrade"`
|
||||
Movie struct {
|
||||
ID int `json:"id"`
|
||||
Title string `json:"title"`
|
||||
Year int `json:"year"`
|
||||
} `json:"movie"`
|
||||
MovieFile struct {
|
||||
ID int `json:"id"`
|
||||
} `json:"movieFile"`
|
||||
MovieFileID int `json:"movieFileId"`
|
||||
DeletedFiles bool `json:"deletedFiles"`
|
||||
}
|
||||
|
||||
// IsTestEvent reports the payload a webhook's Test button sends. It is answered 200 and
|
||||
// enqueues nothing, which is what makes that button mean "reachable" rather than
|
||||
// "reachable, and here is a row about a series that does not exist".
|
||||
func IsTestEvent(eventType string) bool {
|
||||
return strings.EqualFold(strings.TrimSpace(eventType), "Test")
|
||||
}
|
||||
|
||||
// SonarrRequests turns one Sonarr notification into the work it implies.
|
||||
//
|
||||
// A notification can name several episodes — a multi-episode file, or a rename that moved
|
||||
// a season — so this answers a slice. Each carries its own key, because each is its own
|
||||
// file and the two may well arrive again separately.
|
||||
func SonarrRequests(payload SonarrWebhook) []IngestRequest {
|
||||
event := strings.ToLower(strings.TrimSpace(payload.EventType))
|
||||
title := strings.TrimSpace(payload.Series.Title)
|
||||
|
||||
switch event {
|
||||
case "download", "episodefileimported":
|
||||
if title == "" || len(payload.Episodes) == 0 {
|
||||
return nil
|
||||
}
|
||||
reason := ReasonImport
|
||||
if payload.IsUpgrade {
|
||||
// The file genuinely changed, so the row must be re-read. That it is not
|
||||
// *news* is a separate judgement, made by the alert half.
|
||||
reason = ReasonUpgrade
|
||||
}
|
||||
out := make([]IngestRequest, 0, len(payload.Episodes))
|
||||
for _, episode := range payload.Episodes {
|
||||
out = append(out, IngestRequest{
|
||||
Key: sonarrEpisodeKey(payload.EpisodeFile.ID, episode.ID),
|
||||
Action: ActionRefresh,
|
||||
Kind: KindEpisode,
|
||||
Reason: reason,
|
||||
Series: title,
|
||||
SeriesYear: payload.Series.Year,
|
||||
Season: episode.SeasonNumber,
|
||||
Episode: episode.EpisodeNumber,
|
||||
})
|
||||
}
|
||||
return out
|
||||
|
||||
case "rename":
|
||||
if title == "" {
|
||||
return nil
|
||||
}
|
||||
// A rename names files rather than episodes, and Sonarr does not say which episode
|
||||
// each file held. The series is the unit of work: one re-read of the show's
|
||||
// episodes settles every file that moved, and a season rename would otherwise be
|
||||
// one request per episode for the same answer.
|
||||
return []IngestRequest{{
|
||||
Key: fmt.Sprintf("sonarr:series:%d:rename", payload.Series.ID),
|
||||
Action: ActionRefresh,
|
||||
Kind: KindSeries,
|
||||
Reason: ReasonRename,
|
||||
Series: title,
|
||||
SeriesYear: payload.Series.Year,
|
||||
}}
|
||||
|
||||
case "episodefiledelete", "episodefiledeleted":
|
||||
if title == "" {
|
||||
return nil
|
||||
}
|
||||
season, episode := deletedEpisodeNumbers(payload)
|
||||
if episode <= 0 {
|
||||
return nil
|
||||
}
|
||||
return []IngestRequest{{
|
||||
Key: fmt.Sprintf("sonarr:episodefile:%d:delete", payload.EpisodeFile.ID),
|
||||
Action: ActionRemove,
|
||||
Kind: KindEpisode,
|
||||
Reason: ReasonDelete,
|
||||
Series: title,
|
||||
SeriesYear: payload.Series.Year,
|
||||
Season: season,
|
||||
Episode: episode,
|
||||
}}
|
||||
|
||||
case "seriesdelete", "seriesdeleted":
|
||||
// Only a delete that took the files. A series unfollowed in Sonarr while its
|
||||
// episodes stay on disk is still in the library and must stay in the catalogue.
|
||||
if title == "" || !payload.DeletedFiles {
|
||||
return nil
|
||||
}
|
||||
return []IngestRequest{{
|
||||
Key: fmt.Sprintf("sonarr:series:%d:delete", payload.Series.ID),
|
||||
Action: ActionRemove,
|
||||
Kind: KindSeries,
|
||||
Reason: ReasonDelete,
|
||||
Series: title,
|
||||
SeriesYear: payload.Series.Year,
|
||||
}}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// sonarrEpisodeKey prefers the file id, which is the thing that actually changed. Sonarr
|
||||
// omits it on some versions of the import event, and the episode id is then the only
|
||||
// stable identity available — coarser, since it does not change when the file is
|
||||
// replaced, but a repeated delivery still collapses, which is what the key is for.
|
||||
func sonarrEpisodeKey(fileID, episodeID int) string {
|
||||
if fileID > 0 {
|
||||
return fmt.Sprintf("sonarr:episodefile:%d:%d", fileID, episodeID)
|
||||
}
|
||||
return fmt.Sprintf("sonarr:episode:%d", episodeID)
|
||||
}
|
||||
|
||||
// deletedEpisodeNumbers reads the position of a deleted file. The episode list is
|
||||
// preferred because it carries the episode number; the file's own season number stands in
|
||||
// where the list is absent.
|
||||
func deletedEpisodeNumbers(payload SonarrWebhook) (int, int) {
|
||||
for _, episode := range payload.Episodes {
|
||||
if episode.EpisodeNumber > 0 {
|
||||
season := episode.SeasonNumber
|
||||
if season == 0 && payload.EpisodeFile.SeasonNumber > 0 {
|
||||
season = payload.EpisodeFile.SeasonNumber
|
||||
}
|
||||
return season, episode.EpisodeNumber
|
||||
}
|
||||
}
|
||||
return payload.EpisodeFile.SeasonNumber, 0
|
||||
}
|
||||
|
||||
// RadarrRequests turns one Radarr notification into the work it implies.
|
||||
func RadarrRequests(payload RadarrWebhook) []IngestRequest {
|
||||
event := strings.ToLower(strings.TrimSpace(payload.EventType))
|
||||
title := strings.TrimSpace(payload.Movie.Title)
|
||||
if title == "" || payload.Movie.ID <= 0 {
|
||||
return nil
|
||||
}
|
||||
fileID := payload.MovieFile.ID
|
||||
if fileID <= 0 {
|
||||
fileID = payload.MovieFileID
|
||||
}
|
||||
|
||||
switch event {
|
||||
case "download", "moviefileimported":
|
||||
reason := ReasonImport
|
||||
if payload.IsUpgrade {
|
||||
reason = ReasonUpgrade
|
||||
}
|
||||
return []IngestRequest{{
|
||||
Key: radarrFileKey(payload.Movie.ID, fileID, reason),
|
||||
Action: ActionRefresh,
|
||||
Kind: KindMovie,
|
||||
Reason: reason,
|
||||
Title: title,
|
||||
Year: payload.Movie.Year,
|
||||
}}
|
||||
|
||||
case "rename":
|
||||
return []IngestRequest{{
|
||||
Key: fmt.Sprintf("radarr:movie:%d:rename", payload.Movie.ID),
|
||||
Action: ActionRefresh,
|
||||
Kind: KindMovie,
|
||||
Reason: ReasonRename,
|
||||
Title: title,
|
||||
Year: payload.Movie.Year,
|
||||
}}
|
||||
|
||||
case "moviefiledelete", "moviefiledeleted":
|
||||
return []IngestRequest{{
|
||||
Key: radarrFileKey(payload.Movie.ID, fileID, ReasonDelete),
|
||||
Action: ActionRemove,
|
||||
Kind: KindMovie,
|
||||
Reason: ReasonDelete,
|
||||
Title: title,
|
||||
Year: payload.Movie.Year,
|
||||
}}
|
||||
|
||||
case "moviedelete", "moviedeleted":
|
||||
if !payload.DeletedFiles {
|
||||
return nil
|
||||
}
|
||||
return []IngestRequest{{
|
||||
Key: fmt.Sprintf("radarr:movie:%d:delete", payload.Movie.ID),
|
||||
Action: ActionRemove,
|
||||
Kind: KindMovie,
|
||||
Reason: ReasonDelete,
|
||||
Title: title,
|
||||
Year: payload.Movie.Year,
|
||||
}}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func radarrFileKey(movieID, fileID int, reason string) string {
|
||||
if fileID > 0 {
|
||||
return fmt.Sprintf("radarr:moviefile:%d:%s", fileID, reason)
|
||||
}
|
||||
return fmt.Sprintf("radarr:movie:%d:%s", movieID, reason)
|
||||
}
|
||||
|
||||
// NormalizedTitle strips a title down to the letters and digits it shares with whatever
|
||||
// the other system calls it, so "Marvel's Daredevil" and "Marvels Daredevil" are one show.
|
||||
//
|
||||
// It lives here because both halves of the gateway need it and there must be exactly one
|
||||
// answer to "which show is this": the schedule row matches Sonarr titles against the Emby
|
||||
// catalogue with it, and the ingest worker matches the same titles against Emby itself.
|
||||
func NormalizedTitle(value string) string {
|
||||
return strings.Map(func(r rune) rune {
|
||||
if r >= 'A' && r <= 'Z' {
|
||||
return r + ('a' - 'A')
|
||||
}
|
||||
if (r >= 'a' && r <= 'z') || (r >= '0' && r <= '9') {
|
||||
return r
|
||||
}
|
||||
return -1
|
||||
}, value)
|
||||
}
|
||||
@@ -0,0 +1,229 @@
|
||||
package library
|
||||
|
||||
import "testing"
|
||||
|
||||
func sonarrDownload(upgrade bool) SonarrWebhook {
|
||||
var payload SonarrWebhook
|
||||
payload.EventType = "Download"
|
||||
payload.IsUpgrade = upgrade
|
||||
payload.Series.ID = 12
|
||||
payload.Series.Title = "Blue Bloods"
|
||||
payload.Series.Year = 2010
|
||||
payload.EpisodeFile.ID = 8123
|
||||
payload.Episodes = append(payload.Episodes, struct {
|
||||
ID int `json:"id"`
|
||||
SeasonNumber int `json:"seasonNumber"`
|
||||
EpisodeNumber int `json:"episodeNumber"`
|
||||
}{ID: 551, SeasonNumber: 6, EpisodeNumber: 7})
|
||||
return payload
|
||||
}
|
||||
|
||||
func TestSonarrImportBecomesOneEpisodeRefresh(t *testing.T) {
|
||||
requests := SonarrRequests(sonarrDownload(false))
|
||||
if len(requests) != 1 {
|
||||
t.Fatalf("expected one request, got %d", len(requests))
|
||||
}
|
||||
request := requests[0]
|
||||
if request.Action != ActionRefresh || request.Kind != KindEpisode {
|
||||
t.Fatalf("unexpected shape: %+v", request)
|
||||
}
|
||||
if request.Reason != ReasonImport {
|
||||
t.Fatalf("expected an import, got %q", request.Reason)
|
||||
}
|
||||
if request.Series != "Blue Bloods" || request.Season != 6 || request.Episode != 7 {
|
||||
t.Fatalf("unexpected identity: %+v", request)
|
||||
}
|
||||
}
|
||||
|
||||
// An upgrade is silent as *news* and is still a reason to re-read the row: the file
|
||||
// genuinely changed. Conflating those two judgements is how a replaced file would keep a
|
||||
// catalogue entry describing the copy it replaced.
|
||||
func TestSonarrUpgradeStillRefreshes(t *testing.T) {
|
||||
requests := SonarrRequests(sonarrDownload(true))
|
||||
if len(requests) != 1 || requests[0].Reason != ReasonUpgrade {
|
||||
t.Fatalf("expected one upgrade refresh, got %+v", requests)
|
||||
}
|
||||
if requests[0].Action != ActionRefresh {
|
||||
t.Fatalf("an upgrade must refresh, got %q", requests[0].Action)
|
||||
}
|
||||
}
|
||||
|
||||
// The key names the file, so two deliveries of one import are one piece of work. Both
|
||||
// *arrs re-notify on retry and neither promises exactly-once.
|
||||
func TestRepeatedDeliveryKeepsOneKey(t *testing.T) {
|
||||
first := SonarrRequests(sonarrDownload(false))
|
||||
second := SonarrRequests(sonarrDownload(false))
|
||||
if first[0].Key != second[0].Key {
|
||||
t.Fatalf("the same import produced two keys: %q and %q", first[0].Key, second[0].Key)
|
||||
}
|
||||
// A different file for the same episode is different work, or a replacement would be
|
||||
// swallowed by the row its predecessor left behind.
|
||||
replaced := sonarrDownload(true)
|
||||
replaced.EpisodeFile.ID = 9001
|
||||
if SonarrRequests(replaced)[0].Key == first[0].Key {
|
||||
t.Fatal("a replacement file must not reuse the previous file's key")
|
||||
}
|
||||
}
|
||||
|
||||
// A multi-episode file names several episodes and each is its own row, because each may
|
||||
// well be delivered again on its own.
|
||||
func TestSonarrMultiEpisodeFileProducesOneRequestEach(t *testing.T) {
|
||||
payload := sonarrDownload(false)
|
||||
payload.Episodes = append(payload.Episodes, struct {
|
||||
ID int `json:"id"`
|
||||
SeasonNumber int `json:"seasonNumber"`
|
||||
EpisodeNumber int `json:"episodeNumber"`
|
||||
}{ID: 552, SeasonNumber: 6, EpisodeNumber: 8})
|
||||
requests := SonarrRequests(payload)
|
||||
if len(requests) != 2 {
|
||||
t.Fatalf("expected two requests, got %d", len(requests))
|
||||
}
|
||||
if requests[0].Key == requests[1].Key {
|
||||
t.Fatal("two episodes of one file collapsed onto one key")
|
||||
}
|
||||
}
|
||||
|
||||
// A rename is series-wide because Sonarr does not say which episode each moved file held,
|
||||
// and it is a refresh rather than an invalidation: the Emby id survives a move.
|
||||
func TestSonarrRenameRefreshesTheSeries(t *testing.T) {
|
||||
var payload SonarrWebhook
|
||||
payload.EventType = "Rename"
|
||||
payload.Series.ID = 12
|
||||
payload.Series.Title = "Blue Bloods"
|
||||
requests := SonarrRequests(payload)
|
||||
if len(requests) != 1 {
|
||||
t.Fatalf("expected one request, got %d", len(requests))
|
||||
}
|
||||
if requests[0].Kind != KindSeries || requests[0].Action != ActionRefresh {
|
||||
t.Fatalf("unexpected shape: %+v", requests[0])
|
||||
}
|
||||
if requests[0].Reason != ReasonRename {
|
||||
t.Fatalf("expected a rename, got %q", requests[0].Reason)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSonarrEpisodeDeleteRemovesThatEpisode(t *testing.T) {
|
||||
payload := sonarrDownload(false)
|
||||
payload.EventType = "EpisodeFileDelete"
|
||||
requests := SonarrRequests(payload)
|
||||
if len(requests) != 1 || requests[0].Action != ActionRemove {
|
||||
t.Fatalf("expected one removal, got %+v", requests)
|
||||
}
|
||||
if requests[0].Season != 6 || requests[0].Episode != 7 {
|
||||
t.Fatalf("unexpected position: %+v", requests[0])
|
||||
}
|
||||
}
|
||||
|
||||
// A series removed from Sonarr's list while its files stay on disk is still in the
|
||||
// library. Only a delete that took the media with it removes anything.
|
||||
func TestSonarrSeriesDeleteOnlyCountsWhenFilesWent(t *testing.T) {
|
||||
var payload SonarrWebhook
|
||||
payload.EventType = "SeriesDelete"
|
||||
payload.Series.ID = 12
|
||||
payload.Series.Title = "Blue Bloods"
|
||||
|
||||
if requests := SonarrRequests(payload); len(requests) != 0 {
|
||||
t.Fatalf("an unfollowed series must not be removed: %+v", requests)
|
||||
}
|
||||
payload.DeletedFiles = true
|
||||
requests := SonarrRequests(payload)
|
||||
if len(requests) != 1 || requests[0].Action != ActionRemove || requests[0].Kind != KindSeries {
|
||||
t.Fatalf("expected a series removal, got %+v", requests)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSonarrIgnoresEventsThatChangeNothing(t *testing.T) {
|
||||
for _, event := range []string{"Grab", "Health", "ApplicationUpdate", "", "ManualInteractionRequired"} {
|
||||
var payload SonarrWebhook
|
||||
payload.EventType = event
|
||||
payload.Series.Title = "Blue Bloods"
|
||||
if requests := SonarrRequests(payload); len(requests) != 0 {
|
||||
t.Fatalf("%q produced work: %+v", event, requests)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func radarrDownload(upgrade bool) RadarrWebhook {
|
||||
var payload RadarrWebhook
|
||||
payload.EventType = "Download"
|
||||
payload.IsUpgrade = upgrade
|
||||
payload.Movie.ID = 44
|
||||
payload.Movie.Title = "Arrival"
|
||||
payload.Movie.Year = 2016
|
||||
payload.MovieFile.ID = 441
|
||||
return payload
|
||||
}
|
||||
|
||||
func TestRadarrImportAndUpgradeBothRefresh(t *testing.T) {
|
||||
imported := RadarrRequests(radarrDownload(false))
|
||||
if len(imported) != 1 || imported[0].Reason != ReasonImport || imported[0].Kind != KindMovie {
|
||||
t.Fatalf("unexpected import: %+v", imported)
|
||||
}
|
||||
upgraded := RadarrRequests(radarrDownload(true))
|
||||
if len(upgraded) != 1 || upgraded[0].Reason != ReasonUpgrade {
|
||||
t.Fatalf("unexpected upgrade: %+v", upgraded)
|
||||
}
|
||||
// The two are separate work: an upgrade of a file already imported must not be
|
||||
// swallowed by the settled row its import left behind.
|
||||
if imported[0].Key == upgraded[0].Key {
|
||||
t.Fatal("an upgrade reused the import's key")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRadarrDeleteReadsTheTopLevelFileID(t *testing.T) {
|
||||
var payload RadarrWebhook
|
||||
payload.EventType = "MovieFileDelete"
|
||||
payload.Movie.ID = 44
|
||||
payload.Movie.Title = "Arrival"
|
||||
payload.MovieFileID = 441
|
||||
requests := RadarrRequests(payload)
|
||||
if len(requests) != 1 || requests[0].Action != ActionRemove {
|
||||
t.Fatalf("expected one removal, got %+v", requests)
|
||||
}
|
||||
if requests[0].Key != "radarr:moviefile:441:delete" {
|
||||
t.Fatalf("unexpected key: %q", requests[0].Key)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRadarrMovieDeleteOnlyCountsWhenFilesWent(t *testing.T) {
|
||||
var payload RadarrWebhook
|
||||
payload.EventType = "MovieDelete"
|
||||
payload.Movie.ID = 44
|
||||
payload.Movie.Title = "Arrival"
|
||||
if requests := RadarrRequests(payload); len(requests) != 0 {
|
||||
t.Fatalf("an unmonitored film must not be removed: %+v", requests)
|
||||
}
|
||||
payload.DeletedFiles = true
|
||||
if requests := RadarrRequests(payload); len(requests) != 1 {
|
||||
t.Fatalf("expected a removal, got %+v", requests)
|
||||
}
|
||||
}
|
||||
|
||||
// The Test button must be answered without recording work about something that does not
|
||||
// exist, which is what makes it mean "reachable".
|
||||
func TestTestEventIsRecognisedFromEitherArr(t *testing.T) {
|
||||
if !IsTestEvent("Test") || !IsTestEvent(" test ") {
|
||||
t.Fatal("a test event was not recognised")
|
||||
}
|
||||
if IsTestEvent("Download") {
|
||||
t.Fatal("an import was read as a test")
|
||||
}
|
||||
var sonarrTest SonarrWebhook
|
||||
sonarrTest.EventType = "Test"
|
||||
sonarrTest.Series.Title = "Test Title"
|
||||
if requests := SonarrRequests(sonarrTest); len(requests) != 0 {
|
||||
t.Fatalf("the test event produced work: %+v", requests)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizedTitleIgnoresPunctuationAndCase(t *testing.T) {
|
||||
if NormalizedTitle("Marvel's Daredevil") != NormalizedTitle("Marvels Daredevil") {
|
||||
t.Fatal("punctuation changed the answer")
|
||||
}
|
||||
if NormalizedTitle("The Pitt") != "thepitt" {
|
||||
t.Fatalf("unexpected normalisation: %q", NormalizedTitle("The Pitt"))
|
||||
}
|
||||
if NormalizedTitle(" ") != "" {
|
||||
t.Fatal("a blank title must normalise to nothing")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,740 @@
|
||||
package library
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/ponzischeme89/memby/server/internal/emby"
|
||||
"github.com/ponzischeme89/memby/server/internal/store"
|
||||
)
|
||||
|
||||
// The worker that drains what Sonarr and Radarr told us.
|
||||
//
|
||||
// The scheduled import asks Emby "what has changed since an hour ago" and pages through
|
||||
// the answer. This asks Emby "where is this one episode", which is a request whose size
|
||||
// does not grow with the library, and it asks only because something that actually puts
|
||||
// files on disk said there was a reason to.
|
||||
//
|
||||
// Emby is the *lookup* here and never the discovery mechanism. Nothing in this file
|
||||
// enumerates a library, and the one thing that still does — Syncer.Schedule — is demoted
|
||||
// to reconciliation for media the *arrs do not manage.
|
||||
|
||||
const (
|
||||
// defaultSettleDelay is how long after a webhook the first attempt is made. Sonarr
|
||||
// fires On Import the moment it has moved the file; Emby has not scanned it yet, and
|
||||
// asking immediately would spend a request to learn that.
|
||||
defaultSettleDelay = 60 * time.Second
|
||||
|
||||
// idlePoll is how often the worker looks for due work. Coarse on purpose: everything
|
||||
// here is already late by a settle delay, and a tight loop against Postgres on an idle
|
||||
// NAS is exactly the background cost this replaces.
|
||||
idlePoll = 20 * time.Second
|
||||
|
||||
// claimBatch bounds one pass. A season pack arrives as a dozen notifications at once
|
||||
// and there is no hurry: draining a few per pass keeps Emby's request rate flat.
|
||||
claimBatch = 4
|
||||
|
||||
// maxAttempts is where a piece of work is given up on. With the backoff below that is
|
||||
// most of a day, after which the item is the reconciliation sweep's problem — which is
|
||||
// the honest answer, since something other than timing is wrong by then.
|
||||
maxAttempts = 7
|
||||
|
||||
// jobBudget bounds one piece of work end to end.
|
||||
jobBudget = 60 * time.Second
|
||||
)
|
||||
|
||||
// IngestStore is the slice of the store this needs. Narrow so the whole worker can be
|
||||
// exercised against maps in a test, and so it is visible at a glance that the only things
|
||||
// it writes are catalogue rows and the queue's own state.
|
||||
type IngestStore interface {
|
||||
EnqueueIngest(ctx context.Context, job store.IngestJob) (bool, error)
|
||||
ClaimIngest(ctx context.Context, now time.Time, limit int) ([]store.IngestJob, error)
|
||||
FinishIngest(ctx context.Context, key, state, outcome, itemID, errorText string, retryAt time.Time) error
|
||||
UpsertLibraryItems(ctx context.Context, items []store.LibraryItem, syncedAt time.Time) (int64, error)
|
||||
DeleteLibraryItem(ctx context.Context, itemID string) (int64, error)
|
||||
SeriesRefs(ctx context.Context) ([]store.SeriesRef, error)
|
||||
CreditsSeriesEpisodes(ctx context.Context, seriesIDs []string) ([]store.CreditsEpisodeRow, error)
|
||||
LibraryItemsByName(ctx context.Context, itemType, name string) ([]store.NamedItem, error)
|
||||
}
|
||||
|
||||
// EmbySource is the slice of Emby this needs: two reads and one nudge.
|
||||
type EmbySource interface {
|
||||
Items(ctx context.Context, cred emby.Credentials, params url.Values) (*emby.ItemsResult, error)
|
||||
Episodes(ctx context.Context, cred emby.Credentials, seriesID string, params url.Values) (*emby.ItemsResult, error)
|
||||
RefreshItem(ctx context.Context, cred emby.Credentials, itemID string) error
|
||||
}
|
||||
|
||||
// Ingester drains the durable queue.
|
||||
type Ingester struct {
|
||||
Store IngestStore
|
||||
Emby EmbySource
|
||||
Credentials func(ctx context.Context) (emby.Credentials, error)
|
||||
Log *slog.Logger
|
||||
// Paused is the server-wide quiet-time gate. The queue is durable precisely so this can
|
||||
// say no: a webhook that arrives during quiet hours is recorded and read afterwards,
|
||||
// where the old arrangement answered it 503 and lost the event outright.
|
||||
Paused func() bool
|
||||
// Settle is the delay applied when work is enqueued. Held here so the hook and the
|
||||
// worker cannot disagree about it.
|
||||
Settle time.Duration
|
||||
// Announce is told about a finished import, so the news reaches the televisions from
|
||||
// the moment the title is actually there rather than from the moment the *arr said it
|
||||
// would be. Installed from main.go, like syncer.SetAfterSync and for the same reason:
|
||||
// library has no business knowing what an alert is. Nil is ordinary — a gateway with
|
||||
// nothing to announce to, and every test in this package.
|
||||
Announce func(ctx context.Context, result IngestResult)
|
||||
}
|
||||
|
||||
func (i *Ingester) log() *slog.Logger {
|
||||
if i == nil || i.Log == nil {
|
||||
return slog.Default()
|
||||
}
|
||||
return i.Log
|
||||
}
|
||||
|
||||
// SettleDelay is what the hook stamps onto a new row.
|
||||
func (i *Ingester) SettleDelay() time.Duration {
|
||||
if i == nil || i.Settle <= 0 {
|
||||
return defaultSettleDelay
|
||||
}
|
||||
return i.Settle
|
||||
}
|
||||
|
||||
// Run is the worker. One goroutine for the whole gateway.
|
||||
func (i *Ingester) Run(ctx context.Context) {
|
||||
if i == nil || i.Store == nil || i.Emby == nil || i.Credentials == nil {
|
||||
return
|
||||
}
|
||||
i.log().Info("library ingest worker started", "settle", i.SettleDelay().String())
|
||||
for {
|
||||
if ctx.Err() != nil {
|
||||
return
|
||||
}
|
||||
worked := false
|
||||
if i.Paused == nil || !i.Paused() {
|
||||
worked = i.drain(ctx)
|
||||
}
|
||||
if worked {
|
||||
continue
|
||||
}
|
||||
if !sleep(ctx, idlePoll) {
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// drain works everything currently due and reports whether it did anything, so a busy
|
||||
// queue is emptied without waiting a poll interval between rows.
|
||||
func (i *Ingester) drain(ctx context.Context) bool {
|
||||
jobs, err := i.Store.ClaimIngest(ctx, time.Now().UTC(), claimBatch)
|
||||
if err != nil {
|
||||
if ctx.Err() == nil {
|
||||
i.log().Warn("could not read the ingest queue", "error", err)
|
||||
}
|
||||
return false
|
||||
}
|
||||
if len(jobs) == 0 {
|
||||
return false
|
||||
}
|
||||
for _, job := range jobs {
|
||||
if ctx.Err() != nil {
|
||||
return false
|
||||
}
|
||||
jobCtx, cancel := context.WithTimeout(ctx, jobBudget)
|
||||
i.work(jobCtx, job)
|
||||
cancel()
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// work is one row, start to finish. Every exit records an outcome, because the row *is*
|
||||
// the operator's answer to "why was this item re-read, and did it work".
|
||||
func (i *Ingester) work(ctx context.Context, job store.IngestJob) {
|
||||
var request IngestRequest
|
||||
if err := json.Unmarshal(job.Payload, &request); err != nil {
|
||||
i.settle(ctx, job, store.IngestFailed, "invalid", "", err)
|
||||
return
|
||||
}
|
||||
request.Key, request.Action = job.Key, job.Action
|
||||
request.Kind, request.Reason = job.Kind, job.Reason
|
||||
|
||||
cred, err := i.Credentials(ctx)
|
||||
if err != nil {
|
||||
// Nobody has signed in yet, so there is no way to ask Emby anything. That is a
|
||||
// deferral rather than a failure: the work is still valid, it simply cannot be
|
||||
// done until a television signs in.
|
||||
i.defer_(ctx, job, "no_credentials", err)
|
||||
return
|
||||
}
|
||||
|
||||
if job.Action == ActionRemove {
|
||||
i.remove(ctx, job, request)
|
||||
return
|
||||
}
|
||||
i.refresh(ctx, job, request, cred)
|
||||
}
|
||||
|
||||
// refresh is the ordinary path: find the item in Emby and write it into the catalogue.
|
||||
func (i *Ingester) refresh(
|
||||
ctx context.Context, job store.IngestJob, request IngestRequest, cred emby.Credentials,
|
||||
) {
|
||||
items, itemID, err := i.resolve(ctx, request, cred)
|
||||
if err != nil {
|
||||
i.defer_(ctx, job, "lookup_failed", err)
|
||||
return
|
||||
}
|
||||
if len(items) == 0 {
|
||||
// Emby has not scanned the file in yet, which on a fresh import is the expected
|
||||
// first answer rather than a fault. One nudge, then wait: the backoff is what turns
|
||||
// "not yet" into "not ever" without a request per minute in between.
|
||||
i.nudge(ctx, request, cred)
|
||||
i.defer_(ctx, job, "not_found", nil)
|
||||
return
|
||||
}
|
||||
|
||||
written := make([]store.LibraryItem, 0, len(items))
|
||||
for _, raw := range items {
|
||||
if item, ok := toLibraryItem(raw); ok {
|
||||
written = append(written, item)
|
||||
}
|
||||
}
|
||||
if len(written) == 0 {
|
||||
i.defer_(ctx, job, "not_found", nil)
|
||||
return
|
||||
}
|
||||
// Stamped now, like any other import, so a title written here is never the victim of a
|
||||
// full pass that happens to be running.
|
||||
if _, err := i.Store.UpsertLibraryItems(ctx, written, time.Now().UTC()); err != nil {
|
||||
i.defer_(ctx, job, "write_failed", err)
|
||||
return
|
||||
}
|
||||
i.settle(ctx, job, store.IngestDone, "imported", itemID, nil)
|
||||
i.log().Info("arr ingest",
|
||||
"event", "arr_ingest", "key", job.Key, "source", job.Source, "reason", job.Reason,
|
||||
"kind", job.Kind, "outcome", "imported", "items", len(written),
|
||||
"item", itemID, "attempts", job.Attempts+1)
|
||||
// After the row is recorded, never before: the announcement is a claim that the title
|
||||
// is in the catalogue, and it must not be made by a pass that then failed to record it.
|
||||
i.announce(ctx, job, request, written, itemID)
|
||||
}
|
||||
|
||||
// announce reports a finished scan, if anybody is listening.
|
||||
//
|
||||
// Whether a given import is worth a banner is deliberately not decided here — that is a
|
||||
// question about what viewers should be told, which belongs with the rest of the alert
|
||||
// wording. This says what happened; the API package decides what to say about it.
|
||||
func (i *Ingester) announce(
|
||||
ctx context.Context, job store.IngestJob, request IngestRequest,
|
||||
written []store.LibraryItem, itemID string,
|
||||
) {
|
||||
if i.Announce == nil {
|
||||
return
|
||||
}
|
||||
result := IngestResult{
|
||||
Source: job.Source,
|
||||
Kind: job.Kind,
|
||||
Reason: job.Reason,
|
||||
ItemID: itemID,
|
||||
SeriesName: request.Series,
|
||||
Season: request.Season,
|
||||
Episode: request.Episode,
|
||||
Name: request.Title,
|
||||
Year: request.Year,
|
||||
}
|
||||
// Emby's own record of the item outranks what the *arr called it: they disagree about
|
||||
// punctuation and about years often enough that the banner and the card underneath it
|
||||
// would otherwise name the same thing two ways.
|
||||
if item, found := findWritten(written, itemID); found {
|
||||
result.Name = item.Name
|
||||
result.ImageTag = primaryImageTag(item.Payload)
|
||||
if item.SeriesName != "" {
|
||||
result.SeriesName = item.SeriesName
|
||||
}
|
||||
if item.ProductionYear != nil {
|
||||
result.Year = *item.ProductionYear
|
||||
}
|
||||
}
|
||||
i.Announce(ctx, result)
|
||||
}
|
||||
|
||||
func findWritten(written []store.LibraryItem, itemID string) (store.LibraryItem, bool) {
|
||||
if itemID == "" {
|
||||
return store.LibraryItem{}, false
|
||||
}
|
||||
for _, item := range written {
|
||||
if item.ID == itemID {
|
||||
return item, true
|
||||
}
|
||||
}
|
||||
return store.LibraryItem{}, false
|
||||
}
|
||||
|
||||
// primaryImageTag digs the poster tag out of the payload that was just stored, so a banner
|
||||
// can carry artwork without a second lookup. An absent tag is ordinary and costs nothing:
|
||||
// the alert simply travels without one.
|
||||
func primaryImageTag(payload json.RawMessage) string {
|
||||
var parsed struct {
|
||||
ImageTags map[string]string `json:"ImageTags"`
|
||||
}
|
||||
if json.Unmarshal(payload, &parsed) != nil {
|
||||
return ""
|
||||
}
|
||||
return parsed.ImageTags["Primary"]
|
||||
}
|
||||
|
||||
// remove takes a deleted title out of the catalogue.
|
||||
//
|
||||
// It resolves against the *local* catalogue rather than against Emby, which is the one
|
||||
// place in this file that is deliberately the other way round: the thing being removed is
|
||||
// a row in Memby's copy, and Emby — having had the file deleted underneath it — is the
|
||||
// least likely place to still be able to name it.
|
||||
func (i *Ingester) remove(ctx context.Context, job store.IngestJob, request IngestRequest) {
|
||||
itemID, err := i.localItemID(ctx, request)
|
||||
if err != nil {
|
||||
i.defer_(ctx, job, "lookup_failed", err)
|
||||
return
|
||||
}
|
||||
if itemID == "" {
|
||||
// Nothing to remove. Ordinary rather than a failure: the catalogue may never have
|
||||
// held it, or a previous delivery of this event already did the work.
|
||||
i.settle(ctx, job, store.IngestDone, "absent", "", nil)
|
||||
i.log().Info("arr ingest",
|
||||
"event", "arr_ingest", "key", job.Key, "source", job.Source, "reason", job.Reason,
|
||||
"kind", job.Kind, "outcome", "absent")
|
||||
return
|
||||
}
|
||||
removed, err := i.Store.DeleteLibraryItem(ctx, itemID)
|
||||
if err != nil {
|
||||
i.defer_(ctx, job, "delete_failed", err)
|
||||
return
|
||||
}
|
||||
i.settle(ctx, job, store.IngestDone, "removed", itemID, nil)
|
||||
i.log().Info("arr ingest",
|
||||
"event", "arr_ingest", "key", job.Key, "source", job.Source, "reason", job.Reason,
|
||||
"kind", job.Kind, "outcome", "removed", "rows", removed, "item", itemID)
|
||||
}
|
||||
|
||||
// resolve turns what the *arr said into Emby items, narrowly.
|
||||
//
|
||||
// The second return is the item the work was about, for the log and the console. It is
|
||||
// empty for a series-wide refresh, which is about a show rather than about one file.
|
||||
func (i *Ingester) resolve(
|
||||
ctx context.Context, request IngestRequest, cred emby.Credentials,
|
||||
) ([]json.RawMessage, string, error) {
|
||||
switch request.Kind {
|
||||
case KindMovie:
|
||||
return i.resolveMovie(ctx, request, cred)
|
||||
case KindEpisode, KindSeries:
|
||||
return i.resolveFromSeries(ctx, request, cred)
|
||||
}
|
||||
return nil, "", fmt.Errorf("library: unknown ingest kind %q", request.Kind)
|
||||
}
|
||||
|
||||
func (i *Ingester) resolveMovie(
|
||||
ctx context.Context, request IngestRequest, cred emby.Credentials,
|
||||
) ([]json.RawMessage, string, error) {
|
||||
page, err := i.Emby.Items(ctx, cred, itemQuery(url.Values{
|
||||
"SearchTerm": {request.Title},
|
||||
"IncludeItemTypes": {"Movie"},
|
||||
"Recursive": {"true"},
|
||||
"Limit": {"20"},
|
||||
}))
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
if page == nil {
|
||||
return nil, "", nil
|
||||
}
|
||||
match, id := pickByTitle(page.Items, request.Title, request.Year)
|
||||
if match == nil {
|
||||
return nil, "", nil
|
||||
}
|
||||
return []json.RawMessage{match}, id, nil
|
||||
}
|
||||
|
||||
// resolveFromSeries handles both an episode and a whole-series refresh, because they share
|
||||
// the expensive half: working out which Emby show this is.
|
||||
func (i *Ingester) resolveFromSeries(
|
||||
ctx context.Context, request IngestRequest, cred emby.Credentials,
|
||||
) ([]json.RawMessage, string, error) {
|
||||
seriesID, seriesPayload, err := i.seriesItem(ctx, request, cred)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
if seriesID == "" {
|
||||
return nil, "", nil
|
||||
}
|
||||
|
||||
params := itemQuery(url.Values{})
|
||||
if request.Kind == KindEpisode && request.Season > 0 {
|
||||
// One season rather than a show. A long-running series is a thousand records and
|
||||
// this runs per imported file.
|
||||
params.Set("Season", strconv.Itoa(request.Season))
|
||||
}
|
||||
page, err := i.Emby.Episodes(ctx, cred, seriesID, params)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
|
||||
out := make([]json.RawMessage, 0, 8)
|
||||
if seriesPayload != nil {
|
||||
// A show Emby has only just created has no row here yet, and its episodes would be
|
||||
// imported as children of a series the catalogue has never heard of.
|
||||
out = append(out, seriesPayload)
|
||||
}
|
||||
if page == nil {
|
||||
return out, "", nil
|
||||
}
|
||||
if request.Kind == KindSeries {
|
||||
// A rename moved files; which files is not something Sonarr says, so the show is
|
||||
// the unit of work and one re-read settles all of them.
|
||||
return append(out, page.Items...), seriesID, nil
|
||||
}
|
||||
|
||||
for _, raw := range page.Items {
|
||||
var parsed struct {
|
||||
ID string `json:"Id"`
|
||||
IndexNumber *int `json:"IndexNumber"`
|
||||
ParentIndexNumber *int `json:"ParentIndexNumber"`
|
||||
}
|
||||
if json.Unmarshal(raw, &parsed) != nil || parsed.IndexNumber == nil {
|
||||
continue
|
||||
}
|
||||
if *parsed.IndexNumber != request.Episode {
|
||||
continue
|
||||
}
|
||||
if parsed.ParentIndexNumber != nil && *parsed.ParentIndexNumber != request.Season {
|
||||
continue
|
||||
}
|
||||
return append(out, raw), parsed.ID, nil
|
||||
}
|
||||
// The series is there and the episode is not: Emby has the show but has not scanned the
|
||||
// new file. Reporting nothing found keeps that on the deferral path — but the series
|
||||
// payload is still worth writing if it was new.
|
||||
if len(out) > 0 {
|
||||
if _, err := i.Store.UpsertLibraryItems(ctx, seriesItems(out), time.Now().UTC()); err != nil {
|
||||
i.log().Debug("could not write the series row ahead of its episode", "error", err)
|
||||
}
|
||||
}
|
||||
return nil, "", nil
|
||||
}
|
||||
|
||||
// seriesItem answers which Emby series this is, preferring the catalogue.
|
||||
//
|
||||
// The local index is one query the gateway already makes elsewhere and it is right for
|
||||
// every show that has ever been imported. Emby is asked only when it misses, which is
|
||||
// exactly the case this feature exists for — a brand-new show whose first episode has just
|
||||
// landed — and the payload comes back with it so the series row can be written too.
|
||||
func (i *Ingester) seriesItem(
|
||||
ctx context.Context, request IngestRequest, cred emby.Credentials,
|
||||
) (string, json.RawMessage, error) {
|
||||
if id := i.localSeriesID(ctx, request.Series, request.SeriesYear); id != "" {
|
||||
return id, nil, nil
|
||||
}
|
||||
page, err := i.Emby.Items(ctx, cred, itemQuery(url.Values{
|
||||
"SearchTerm": {request.Series},
|
||||
"IncludeItemTypes": {"Series"},
|
||||
"Recursive": {"true"},
|
||||
"Limit": {"20"},
|
||||
}))
|
||||
if err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
if page == nil {
|
||||
return "", nil, nil
|
||||
}
|
||||
match, id := pickByTitle(page.Items, request.Series, request.SeriesYear)
|
||||
return id, match, nil
|
||||
}
|
||||
|
||||
func (i *Ingester) localSeriesID(ctx context.Context, title string, year int) string {
|
||||
refs, err := i.Store.SeriesRefs(ctx)
|
||||
if err != nil {
|
||||
i.log().Debug("series index unavailable for ingest", "error", err)
|
||||
return ""
|
||||
}
|
||||
return matchByTitle(refs, title, year)
|
||||
}
|
||||
|
||||
// localItemID resolves a delete against the catalogue.
|
||||
func (i *Ingester) localItemID(ctx context.Context, request IngestRequest) (string, error) {
|
||||
switch request.Kind {
|
||||
case KindMovie:
|
||||
named, err := i.Store.LibraryItemsByName(ctx, "Movie", request.Title)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return matchNamed(named, request.Title, request.Year), nil
|
||||
|
||||
case KindSeries:
|
||||
return i.localSeriesID(ctx, request.Series, request.SeriesYear), nil
|
||||
|
||||
case KindEpisode:
|
||||
seriesID := i.localSeriesID(ctx, request.Series, request.SeriesYear)
|
||||
if seriesID == "" {
|
||||
return "", nil
|
||||
}
|
||||
episodes, err := i.Store.CreditsSeriesEpisodes(ctx, []string{seriesID})
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
for _, episode := range episodes {
|
||||
if episode.Episode == request.Episode && episode.Season == request.Season {
|
||||
return episode.ItemID, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
return "", nil
|
||||
}
|
||||
|
||||
// nudge asks Emby to look at the folder the file landed in.
|
||||
//
|
||||
// Best-effort and deliberately unreported: it is the same trick the subtitle download uses
|
||||
// after Bazarr writes a sidecar, and a household whose Emby scans on its own does not need
|
||||
// it. Refusing to nudge without a parent is the important half — a refresh of nothing is a
|
||||
// request that cannot help.
|
||||
func (i *Ingester) nudge(ctx context.Context, request IngestRequest, cred emby.Credentials) {
|
||||
if request.Kind == KindMovie {
|
||||
return
|
||||
}
|
||||
seriesID := i.localSeriesID(ctx, request.Series, request.SeriesYear)
|
||||
if seriesID == "" {
|
||||
return
|
||||
}
|
||||
if err := i.Emby.RefreshItem(ctx, cred, seriesID); err != nil {
|
||||
i.log().Debug("could not ask emby to rescan a series", "series", seriesID, "error", err)
|
||||
}
|
||||
}
|
||||
|
||||
// defer_ schedules another attempt, or gives up.
|
||||
func (i *Ingester) defer_(ctx context.Context, job store.IngestJob, outcome string, cause error) {
|
||||
attempts := job.Attempts + 1
|
||||
if attempts >= maxAttempts {
|
||||
i.settle(ctx, job, store.IngestFailed, outcome, "", cause)
|
||||
i.log().Warn("arr ingest gave up",
|
||||
"event", "arr_ingest", "key", job.Key, "source", job.Source, "reason", job.Reason,
|
||||
"kind", job.Kind, "outcome", outcome, "attempts", attempts, "error", errorText(cause))
|
||||
return
|
||||
}
|
||||
retryAt := time.Now().UTC().Add(IngestRetryDelay(attempts))
|
||||
if err := i.Store.FinishIngest(
|
||||
ctx, job.Key, store.IngestPending, outcome, "", errorText(cause), retryAt,
|
||||
); err != nil {
|
||||
i.log().Warn("could not reschedule ingest work", "key", job.Key, "error", err)
|
||||
}
|
||||
i.log().Debug("arr ingest deferred",
|
||||
"event", "arr_ingest", "key", job.Key, "reason", job.Reason, "outcome", outcome,
|
||||
"attempts", attempts, "retry_in", IngestRetryDelay(attempts).String(),
|
||||
"error", errorText(cause))
|
||||
}
|
||||
|
||||
func (i *Ingester) settle(
|
||||
ctx context.Context, job store.IngestJob, state, outcome, itemID string, cause error,
|
||||
) {
|
||||
// Detached from the job's own budget: a row that timed out must still record that it
|
||||
// did, or the next pass claims it again immediately and the backoff never applies.
|
||||
writeCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 5*time.Second)
|
||||
defer cancel()
|
||||
if err := i.Store.FinishIngest(
|
||||
writeCtx, job.Key, state, outcome, itemID, errorText(cause), time.Now().UTC(),
|
||||
); err != nil {
|
||||
i.log().Warn("could not record ingest outcome", "key", job.Key, "error", err)
|
||||
}
|
||||
}
|
||||
|
||||
// IngestRetryDelay is the backoff, and it is a step function rather than an exponent so
|
||||
// the schedule can be read off the page: a minute, five, twenty, an hour, then four-hourly
|
||||
// out to the attempt limit. The early steps are short because the common cause is Emby not
|
||||
// having scanned yet, which resolves in minutes; the late ones are long because by then the
|
||||
// cause is something a faster retry cannot fix.
|
||||
func IngestRetryDelay(attempts int) time.Duration {
|
||||
switch {
|
||||
case attempts <= 1:
|
||||
return time.Minute
|
||||
case attempts == 2:
|
||||
return 5 * time.Minute
|
||||
case attempts == 3:
|
||||
return 20 * time.Minute
|
||||
case attempts == 4:
|
||||
return time.Hour
|
||||
default:
|
||||
return 4 * time.Hour
|
||||
}
|
||||
}
|
||||
|
||||
// itemQuery is the field set every lookup here uses, and it is deliberately the scheduled
|
||||
// import's own.
|
||||
//
|
||||
// Thinning it would leave an event-imported title without People, MediaStreams or
|
||||
// ProviderIds — so no cast on its page, no ratings lookup and no format badges — until Emby
|
||||
// next reported it changed, which for a film nobody edits again is never. Syncer.Find makes
|
||||
// the same promise for the same reason.
|
||||
func itemQuery(params url.Values) url.Values {
|
||||
params.Set("Fields", syncFields)
|
||||
params.Set("ImageTypeLimit", "1")
|
||||
params.Set("EnableImages", "true")
|
||||
params.Set("EnableImageTypes", syncImageTypes)
|
||||
params.Set("EnableTotalRecordCount", "false")
|
||||
params.Set("EnableUserData", "false")
|
||||
return params
|
||||
}
|
||||
|
||||
// pickByTitle chooses the item a title and year names.
|
||||
//
|
||||
// Year-qualified first and title-only as the fallback, the rule the schedule row's series
|
||||
// index already applies: an *arr and Emby disagree about a show's year far more often than
|
||||
// they disagree about its name, but where both know the year it is what separates a remake
|
||||
// from its original.
|
||||
func pickByTitle(items []json.RawMessage, title string, year int) (json.RawMessage, string) {
|
||||
want := NormalizedTitle(title)
|
||||
if want == "" {
|
||||
return nil, ""
|
||||
}
|
||||
var fallback json.RawMessage
|
||||
var fallbackID string
|
||||
for _, raw := range items {
|
||||
var parsed struct {
|
||||
ID string `json:"Id"`
|
||||
Name string `json:"Name"`
|
||||
ProductionYear *int `json:"ProductionYear"`
|
||||
}
|
||||
if json.Unmarshal(raw, &parsed) != nil || parsed.ID == "" {
|
||||
continue
|
||||
}
|
||||
if NormalizedTitle(parsed.Name) != want {
|
||||
continue
|
||||
}
|
||||
if year > 0 && parsed.ProductionYear != nil && *parsed.ProductionYear == year {
|
||||
return raw, parsed.ID
|
||||
}
|
||||
if fallback == nil {
|
||||
fallback, fallbackID = raw, parsed.ID
|
||||
}
|
||||
}
|
||||
return fallback, fallbackID
|
||||
}
|
||||
|
||||
func matchByTitle(refs []store.SeriesRef, title string, year int) string {
|
||||
want := NormalizedTitle(title)
|
||||
if want == "" {
|
||||
return ""
|
||||
}
|
||||
fallback := ""
|
||||
for _, ref := range refs {
|
||||
if NormalizedTitle(ref.Name) != want {
|
||||
continue
|
||||
}
|
||||
if year > 0 && ref.Year == year {
|
||||
return ref.ID
|
||||
}
|
||||
if fallback == "" {
|
||||
fallback = ref.ID
|
||||
}
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
|
||||
func matchNamed(items []store.NamedItem, title string, year int) string {
|
||||
want := NormalizedTitle(title)
|
||||
if want == "" {
|
||||
return ""
|
||||
}
|
||||
fallback := ""
|
||||
for _, item := range items {
|
||||
if NormalizedTitle(item.Name) != want {
|
||||
continue
|
||||
}
|
||||
if year > 0 && item.Year == year {
|
||||
return item.ID
|
||||
}
|
||||
if fallback == "" {
|
||||
fallback = item.ID
|
||||
}
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
|
||||
// seriesItems is the series payload on its own, for the case where the episode has not
|
||||
// appeared yet but the show has.
|
||||
func seriesItems(payloads []json.RawMessage) []store.LibraryItem {
|
||||
out := make([]store.LibraryItem, 0, len(payloads))
|
||||
for _, raw := range payloads {
|
||||
if item, ok := toLibraryItem(raw); ok && item.Type == "Series" {
|
||||
out = append(out, item)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func errorText(err error) string {
|
||||
if err == nil {
|
||||
return ""
|
||||
}
|
||||
return strings.TrimSpace(err.Error())
|
||||
}
|
||||
|
||||
func sleep(ctx context.Context, duration time.Duration) bool {
|
||||
timer := time.NewTimer(duration)
|
||||
defer timer.Stop()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return false
|
||||
case <-timer.C:
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
// Enqueue records work a webhook implied, and answers how much of it was news.
|
||||
//
|
||||
// It is the hook's whole job. Everything expensive happens later, on the worker, which is
|
||||
// what lets the hook answer Sonarr in a millisecond and — more importantly — what lets it
|
||||
// answer at all during quiet hours, when the work itself must wait.
|
||||
func (i *Ingester) Enqueue(
|
||||
ctx context.Context, source string, requests []IngestRequest,
|
||||
) (int, error) {
|
||||
if i == nil || i.Store == nil || len(requests) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
due := time.Now().UTC().Add(i.SettleDelay())
|
||||
fresh := 0
|
||||
var firstErr error
|
||||
for _, request := range requests {
|
||||
payload, err := json.Marshal(request)
|
||||
if err != nil {
|
||||
if firstErr == nil {
|
||||
firstErr = err
|
||||
}
|
||||
continue
|
||||
}
|
||||
inserted, err := i.Store.EnqueueIngest(ctx, store.IngestJob{
|
||||
Key: request.Key,
|
||||
Action: request.Action,
|
||||
Kind: request.Kind,
|
||||
Reason: request.Reason,
|
||||
Source: source,
|
||||
Payload: payload,
|
||||
DueAt: due,
|
||||
})
|
||||
if err != nil {
|
||||
if firstErr == nil {
|
||||
firstErr = err
|
||||
}
|
||||
continue
|
||||
}
|
||||
if inserted {
|
||||
fresh++
|
||||
continue
|
||||
}
|
||||
// A repeat delivery is ordinary — both *arrs re-notify on retry — so it is DEBUG,
|
||||
// the same stance the per-keystroke search line takes.
|
||||
i.log().Debug("arr ingest already queued",
|
||||
"event", "arr_ingest", "key", request.Key, "source", source, "reason", request.Reason)
|
||||
}
|
||||
return fresh, firstErr
|
||||
}
|
||||
@@ -0,0 +1,507 @@
|
||||
package library
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/url"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/ponzischeme89/memby/server/internal/emby"
|
||||
"github.com/ponzischeme89/memby/server/internal/store"
|
||||
)
|
||||
|
||||
// fakeStore is the queue and the catalogue as maps. Everything the worker does is visible
|
||||
// in it, which is the point of the store being an interface here.
|
||||
type fakeStore struct {
|
||||
jobs map[string]*store.IngestJob
|
||||
order []string
|
||||
items map[string]store.LibraryItem
|
||||
deleted []string
|
||||
series []store.SeriesRef
|
||||
episodes []store.CreditsEpisodeRow
|
||||
named []store.NamedItem
|
||||
failNext error
|
||||
}
|
||||
|
||||
func newFakeStore() *fakeStore {
|
||||
return &fakeStore{jobs: map[string]*store.IngestJob{}, items: map[string]store.LibraryItem{}}
|
||||
}
|
||||
|
||||
func (f *fakeStore) EnqueueIngest(_ context.Context, job store.IngestJob) (bool, error) {
|
||||
existing, found := f.jobs[job.Key]
|
||||
if found {
|
||||
// The real table's ON CONFLICT: one row, and a re-delivery never pulls the settle
|
||||
// delay forward.
|
||||
if job.DueAt.After(existing.DueAt) {
|
||||
existing.DueAt = job.DueAt
|
||||
}
|
||||
existing.State = store.IngestPending
|
||||
return false, nil
|
||||
}
|
||||
stored := job
|
||||
stored.State = store.IngestPending
|
||||
f.jobs[job.Key] = &stored
|
||||
f.order = append(f.order, job.Key)
|
||||
return true, nil
|
||||
}
|
||||
|
||||
func (f *fakeStore) ClaimIngest(_ context.Context, now time.Time, limit int) ([]store.IngestJob, error) {
|
||||
out := []store.IngestJob{}
|
||||
for _, key := range f.order {
|
||||
job := f.jobs[key]
|
||||
if job.State != store.IngestPending || job.DueAt.After(now) {
|
||||
continue
|
||||
}
|
||||
out = append(out, *job)
|
||||
if len(out) == limit {
|
||||
break
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (f *fakeStore) FinishIngest(
|
||||
_ context.Context, key, state, outcome, itemID, errorText string, retryAt time.Time,
|
||||
) error {
|
||||
job, found := f.jobs[key]
|
||||
if !found {
|
||||
return errors.New("no such job")
|
||||
}
|
||||
job.State, job.Outcome, job.ItemID = state, outcome, itemID
|
||||
job.LastError, job.DueAt = errorText, retryAt
|
||||
job.Attempts++
|
||||
return nil
|
||||
}
|
||||
|
||||
func (f *fakeStore) UpsertLibraryItems(
|
||||
_ context.Context, items []store.LibraryItem, _ time.Time,
|
||||
) (int64, error) {
|
||||
if f.failNext != nil {
|
||||
err := f.failNext
|
||||
f.failNext = nil
|
||||
return 0, err
|
||||
}
|
||||
for _, item := range items {
|
||||
f.items[item.ID] = item
|
||||
}
|
||||
return int64(len(items)), nil
|
||||
}
|
||||
|
||||
func (f *fakeStore) DeleteLibraryItem(_ context.Context, itemID string) (int64, error) {
|
||||
f.deleted = append(f.deleted, itemID)
|
||||
delete(f.items, itemID)
|
||||
return 1, nil
|
||||
}
|
||||
|
||||
func (f *fakeStore) SeriesRefs(context.Context) ([]store.SeriesRef, error) { return f.series, nil }
|
||||
|
||||
func (f *fakeStore) CreditsSeriesEpisodes(
|
||||
context.Context, []string,
|
||||
) ([]store.CreditsEpisodeRow, error) {
|
||||
return f.episodes, nil
|
||||
}
|
||||
|
||||
func (f *fakeStore) LibraryItemsByName(
|
||||
_ context.Context, _, _ string,
|
||||
) ([]store.NamedItem, error) {
|
||||
return f.named, nil
|
||||
}
|
||||
|
||||
// fakeEmby answers the two lookups and counts the nudges.
|
||||
type fakeEmby struct {
|
||||
items []json.RawMessage
|
||||
episodes []json.RawMessage
|
||||
refreshed []string
|
||||
itemQueries []url.Values
|
||||
seasons []string
|
||||
err error
|
||||
}
|
||||
|
||||
func (f *fakeEmby) Items(
|
||||
_ context.Context, _ emby.Credentials, params url.Values,
|
||||
) (*emby.ItemsResult, error) {
|
||||
f.itemQueries = append(f.itemQueries, params)
|
||||
if f.err != nil {
|
||||
return nil, f.err
|
||||
}
|
||||
return &emby.ItemsResult{Items: f.items}, nil
|
||||
}
|
||||
|
||||
func (f *fakeEmby) Episodes(
|
||||
_ context.Context, _ emby.Credentials, _ string, params url.Values,
|
||||
) (*emby.ItemsResult, error) {
|
||||
f.seasons = append(f.seasons, params.Get("Season"))
|
||||
if f.err != nil {
|
||||
return nil, f.err
|
||||
}
|
||||
return &emby.ItemsResult{Items: f.episodes}, nil
|
||||
}
|
||||
|
||||
func (f *fakeEmby) RefreshItem(_ context.Context, _ emby.Credentials, itemID string) error {
|
||||
f.refreshed = append(f.refreshed, itemID)
|
||||
return nil
|
||||
}
|
||||
|
||||
func testIngester(st *fakeStore, source *fakeEmby) *Ingester {
|
||||
return &Ingester{
|
||||
Store: st,
|
||||
Emby: source,
|
||||
Credentials: func(context.Context) (emby.Credentials, error) {
|
||||
return emby.Credentials{UserID: "u", Token: "t"}, nil
|
||||
},
|
||||
Log: slog.New(slog.NewTextHandler(io.Discard, nil)),
|
||||
Settle: time.Minute,
|
||||
}
|
||||
}
|
||||
|
||||
func episodePayload(id string, season, episode int) json.RawMessage {
|
||||
raw, _ := json.Marshal(map[string]any{
|
||||
"Id": id, "Name": "The Job", "Type": "Episode", "SeriesId": "emby-series",
|
||||
"ParentIndexNumber": season, "IndexNumber": episode,
|
||||
})
|
||||
return raw
|
||||
}
|
||||
|
||||
func enqueueOne(t *testing.T, ingester *Ingester, request IngestRequest) {
|
||||
t.Helper()
|
||||
if _, err := ingester.Enqueue(context.Background(), "sonarr", []IngestRequest{request}); err != nil {
|
||||
t.Fatalf("enqueue: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func episodeRequest() IngestRequest {
|
||||
return IngestRequest{
|
||||
Key: "sonarr:episodefile:8123:551", Action: ActionRefresh, Kind: KindEpisode,
|
||||
Reason: ReasonImport, Series: "Blue Bloods", SeriesYear: 2010, Season: 6, Episode: 7,
|
||||
}
|
||||
}
|
||||
|
||||
// The ordinary path: the series is already in the catalogue, so Emby is asked for one
|
||||
// season and the episode is written.
|
||||
func TestImportWritesTheEpisodeFromOneSeasonLookup(t *testing.T) {
|
||||
st := newFakeStore()
|
||||
st.series = []store.SeriesRef{{ID: "emby-series", Name: "Blue Bloods", Year: 2010}}
|
||||
source := &fakeEmby{episodes: []json.RawMessage{
|
||||
episodePayload("emby-ep-6", 6, 6),
|
||||
episodePayload("emby-ep-7", 6, 7),
|
||||
}}
|
||||
ingester := testIngester(st, source)
|
||||
enqueueOne(t, ingester, episodeRequest())
|
||||
|
||||
ingester.work(context.Background(), *st.jobs["sonarr:episodefile:8123:551"])
|
||||
|
||||
if _, written := st.items["emby-ep-7"]; !written {
|
||||
t.Fatalf("the episode was not written: %v", st.items)
|
||||
}
|
||||
if _, extra := st.items["emby-ep-6"]; extra {
|
||||
t.Fatal("an episode nobody asked about was written")
|
||||
}
|
||||
if len(source.seasons) != 1 || source.seasons[0] != "6" {
|
||||
t.Fatalf("expected one season-scoped lookup, got %v", source.seasons)
|
||||
}
|
||||
// A series already in the catalogue costs no search at all.
|
||||
if len(source.itemQueries) != 0 {
|
||||
t.Fatalf("the catalogue was not used for the series: %v", source.itemQueries)
|
||||
}
|
||||
job := st.jobs["sonarr:episodefile:8123:551"]
|
||||
if job.State != store.IngestDone || job.Outcome != "imported" {
|
||||
t.Fatalf("unexpected outcome: %+v", job)
|
||||
}
|
||||
}
|
||||
|
||||
// The field set must be the scheduled import's own, or an event-imported title arrives
|
||||
// without the cast, streams and provider ids everything downstream reads.
|
||||
func TestLookupsAskForTheFullSyncFields(t *testing.T) {
|
||||
st := newFakeStore()
|
||||
source := &fakeEmby{}
|
||||
ingester := testIngester(st, source)
|
||||
enqueueOne(t, ingester, IngestRequest{
|
||||
Key: "radarr:moviefile:441:import", Action: ActionRefresh, Kind: KindMovie,
|
||||
Reason: ReasonImport, Title: "Arrival", Year: 2016,
|
||||
})
|
||||
ingester.work(context.Background(), *st.jobs["radarr:moviefile:441:import"])
|
||||
|
||||
if len(source.itemQueries) != 1 {
|
||||
t.Fatalf("expected one lookup, got %d", len(source.itemQueries))
|
||||
}
|
||||
query := source.itemQueries[0]
|
||||
if query.Get("Fields") != syncFields {
|
||||
t.Fatalf("a thinner field set was requested: %q", query.Get("Fields"))
|
||||
}
|
||||
if query.Get("EnableUserData") != "false" {
|
||||
t.Fatal("the shared catalogue must never carry one viewer's user data")
|
||||
}
|
||||
}
|
||||
|
||||
// Emby not having scanned the file yet is the expected first answer, not a fault: the row
|
||||
// waits, one nudge is sent, and the backoff widens.
|
||||
func TestNotFoundDefersWithABackoffRatherThanFailing(t *testing.T) {
|
||||
st := newFakeStore()
|
||||
st.series = []store.SeriesRef{{ID: "emby-series", Name: "Blue Bloods", Year: 2010}}
|
||||
source := &fakeEmby{}
|
||||
ingester := testIngester(st, source)
|
||||
enqueueOne(t, ingester, episodeRequest())
|
||||
key := "sonarr:episodefile:8123:551"
|
||||
|
||||
before := time.Now().UTC()
|
||||
ingester.work(context.Background(), *st.jobs[key])
|
||||
|
||||
job := st.jobs[key]
|
||||
if job.State != store.IngestPending || job.Outcome != "not_found" {
|
||||
t.Fatalf("expected a deferral, got %+v", job)
|
||||
}
|
||||
if !job.DueAt.After(before) {
|
||||
t.Fatal("the next attempt was not scheduled into the future")
|
||||
}
|
||||
if len(source.refreshed) != 1 || source.refreshed[0] != "emby-series" {
|
||||
t.Fatalf("expected one rescan nudge at the series, got %v", source.refreshed)
|
||||
}
|
||||
if len(st.items) != 0 {
|
||||
t.Fatal("nothing should have been written")
|
||||
}
|
||||
}
|
||||
|
||||
// Attempts are given up on eventually, because past the last step of the backoff the cause
|
||||
// is not timing and a row retrying for ever is one nobody looks at.
|
||||
func TestRepeatedFailureIsEventuallyGivenUpOn(t *testing.T) {
|
||||
st := newFakeStore()
|
||||
source := &fakeEmby{err: errors.New("emby is not answering")}
|
||||
ingester := testIngester(st, source)
|
||||
enqueueOne(t, ingester, episodeRequest())
|
||||
key := "sonarr:episodefile:8123:551"
|
||||
|
||||
for attempt := 0; attempt < maxAttempts; attempt++ {
|
||||
ingester.work(context.Background(), *st.jobs[key])
|
||||
}
|
||||
if st.jobs[key].State != store.IngestFailed {
|
||||
t.Fatalf("expected the row to be given up on, got %+v", st.jobs[key])
|
||||
}
|
||||
if st.jobs[key].LastError == "" {
|
||||
t.Fatal("a failed row must record why")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRetryDelayWidensAndSettles(t *testing.T) {
|
||||
previous := time.Duration(0)
|
||||
for attempt := 1; attempt <= 6; attempt++ {
|
||||
delay := IngestRetryDelay(attempt)
|
||||
if delay < previous {
|
||||
t.Fatalf("the backoff narrowed at attempt %d: %s after %s", attempt, delay, previous)
|
||||
}
|
||||
previous = delay
|
||||
}
|
||||
if IngestRetryDelay(1) != time.Minute {
|
||||
t.Fatalf("the first retry should be quick, got %s", IngestRetryDelay(1))
|
||||
}
|
||||
}
|
||||
|
||||
// A delete resolves against the catalogue, not against Emby: the file is gone, and Emby is
|
||||
// the least likely thing to still be able to name it.
|
||||
func TestDeleteRemovesTheEpisodeFromTheCatalogue(t *testing.T) {
|
||||
st := newFakeStore()
|
||||
st.series = []store.SeriesRef{{ID: "emby-series", Name: "Blue Bloods", Year: 2010}}
|
||||
st.episodes = []store.CreditsEpisodeRow{
|
||||
{ItemID: "emby-ep-6", SeriesID: "emby-series", Season: 6, Episode: 6},
|
||||
{ItemID: "emby-ep-7", SeriesID: "emby-series", Season: 6, Episode: 7},
|
||||
}
|
||||
source := &fakeEmby{}
|
||||
ingester := testIngester(st, source)
|
||||
request := episodeRequest()
|
||||
request.Action, request.Reason, request.Key = ActionRemove, ReasonDelete, "sonarr:episodefile:8123:delete"
|
||||
enqueueOne(t, ingester, request)
|
||||
|
||||
ingester.work(context.Background(), *st.jobs[request.Key])
|
||||
|
||||
if len(st.deleted) != 1 || st.deleted[0] != "emby-ep-7" {
|
||||
t.Fatalf("unexpected deletions: %v", st.deleted)
|
||||
}
|
||||
if len(source.itemQueries) != 0 || len(source.seasons) != 0 {
|
||||
t.Fatal("a delete must not need to ask Emby anything")
|
||||
}
|
||||
}
|
||||
|
||||
// A delete of something the catalogue never held is settled rather than retried: there is
|
||||
// nothing to remove and no later attempt could change that.
|
||||
func TestDeleteOfSomethingAbsentSettlesQuietly(t *testing.T) {
|
||||
st := newFakeStore()
|
||||
ingester := testIngester(st, &fakeEmby{})
|
||||
request := IngestRequest{
|
||||
Key: "radarr:moviefile:9:delete", Action: ActionRemove, Kind: KindMovie,
|
||||
Reason: ReasonDelete, Title: "Never Imported", Year: 1999,
|
||||
}
|
||||
enqueueOne(t, ingester, request)
|
||||
ingester.work(context.Background(), *st.jobs[request.Key])
|
||||
|
||||
job := st.jobs[request.Key]
|
||||
if job.State != store.IngestDone || job.Outcome != "absent" {
|
||||
t.Fatalf("expected a quiet settle, got %+v", job)
|
||||
}
|
||||
if len(st.deleted) != 0 {
|
||||
t.Fatalf("something was deleted: %v", st.deleted)
|
||||
}
|
||||
}
|
||||
|
||||
// A brand-new show is the case the local index cannot answer, and it is exactly the case
|
||||
// this feature exists for. Emby is asked, and the series row is written beside its episode
|
||||
// so the episode is not a child of a show the catalogue has never heard of.
|
||||
func TestANewSeriesIsResolvedThroughEmbyAndWrittenToo(t *testing.T) {
|
||||
st := newFakeStore()
|
||||
seriesRaw, _ := json.Marshal(map[string]any{
|
||||
"Id": "emby-series", "Name": "Blue Bloods", "Type": "Series", "ProductionYear": 2010,
|
||||
})
|
||||
source := &fakeEmby{
|
||||
items: []json.RawMessage{seriesRaw},
|
||||
episodes: []json.RawMessage{episodePayload("emby-ep-7", 6, 7)},
|
||||
}
|
||||
ingester := testIngester(st, source)
|
||||
enqueueOne(t, ingester, episodeRequest())
|
||||
ingester.work(context.Background(), *st.jobs["sonarr:episodefile:8123:551"])
|
||||
|
||||
if _, written := st.items["emby-series"]; !written {
|
||||
t.Fatalf("the new series row was not written: %v", st.items)
|
||||
}
|
||||
if _, written := st.items["emby-ep-7"]; !written {
|
||||
t.Fatal("the episode was not written")
|
||||
}
|
||||
}
|
||||
|
||||
// The year separates a remake from its original where both systems know it, and the title
|
||||
// alone is the fallback because they disagree about years more often than about names.
|
||||
func TestMovieMatchingPrefersTheYearAndFallsBackToTheTitle(t *testing.T) {
|
||||
original, _ := json.Marshal(map[string]any{
|
||||
"Id": "old", "Name": "The Thing", "Type": "Movie", "ProductionYear": 1982,
|
||||
})
|
||||
remake, _ := json.Marshal(map[string]any{
|
||||
"Id": "new", "Name": "The Thing", "Type": "Movie", "ProductionYear": 2011,
|
||||
})
|
||||
items := []json.RawMessage{original, remake}
|
||||
|
||||
if _, id := pickByTitle(items, "The Thing", 2011); id != "new" {
|
||||
t.Fatalf("the year did not decide: %q", id)
|
||||
}
|
||||
if _, id := pickByTitle(items, "The Thing", 0); id != "old" {
|
||||
t.Fatalf("expected the first title match as the fallback, got %q", id)
|
||||
}
|
||||
if _, id := pickByTitle(items, "Something Else", 0); id != "" {
|
||||
t.Fatalf("an unrelated title matched: %q", id)
|
||||
}
|
||||
}
|
||||
|
||||
// A repeated webhook is one row, and it never pulls the settle delay forward — the whole
|
||||
// point of the delay is that the file has finished being written.
|
||||
func TestRepeatedEnqueueIsOneRowAndKeepsTheSettleDelay(t *testing.T) {
|
||||
st := newFakeStore()
|
||||
ingester := testIngester(st, &fakeEmby{})
|
||||
request := episodeRequest()
|
||||
|
||||
fresh, err := ingester.Enqueue(context.Background(), "sonarr", []IngestRequest{request})
|
||||
if err != nil || fresh != 1 {
|
||||
t.Fatalf("first delivery: fresh=%d err=%v", fresh, err)
|
||||
}
|
||||
first := st.jobs[request.Key].DueAt
|
||||
|
||||
fresh, err = ingester.Enqueue(context.Background(), "sonarr", []IngestRequest{request})
|
||||
if err != nil || fresh != 0 {
|
||||
t.Fatalf("a repeat was treated as news: fresh=%d err=%v", fresh, err)
|
||||
}
|
||||
if len(st.jobs) != 1 {
|
||||
t.Fatalf("a repeat produced %d rows", len(st.jobs))
|
||||
}
|
||||
if st.jobs[request.Key].DueAt.Before(first) {
|
||||
t.Fatal("a repeat pulled the settle delay forward")
|
||||
}
|
||||
}
|
||||
|
||||
// Quiet time stands the worker down without touching the queue, which is the arrangement
|
||||
// that lets the hook accept an event at any hour.
|
||||
func TestQuietTimeStopsTheWorkerAndNotTheQueue(t *testing.T) {
|
||||
st := newFakeStore()
|
||||
st.series = []store.SeriesRef{{ID: "emby-series", Name: "Blue Bloods", Year: 2010}}
|
||||
source := &fakeEmby{episodes: []json.RawMessage{episodePayload("emby-ep-7", 6, 7)}}
|
||||
ingester := testIngester(st, source)
|
||||
ingester.Paused = func() bool { return true }
|
||||
enqueueOne(t, ingester, episodeRequest())
|
||||
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan struct{})
|
||||
go func() { ingester.Run(ctx); close(done) }()
|
||||
time.Sleep(50 * time.Millisecond)
|
||||
cancel()
|
||||
<-done
|
||||
|
||||
if len(st.items) != 0 {
|
||||
t.Fatal("work was done during quiet time")
|
||||
}
|
||||
if st.jobs["sonarr:episodefile:8123:551"].State != store.IngestPending {
|
||||
t.Fatal("the queued work was lost rather than deferred")
|
||||
}
|
||||
}
|
||||
|
||||
// A finished scan is the moment there is something truthful to announce, which is why the
|
||||
// hook fires from here rather than from the webhook. It must carry Emby's own names: the
|
||||
// *arr and Emby disagree about punctuation often enough that a banner built from the
|
||||
// webhook would name the same thing differently from the card underneath it.
|
||||
func TestAFinishedImportIsAnnouncedWithEmbysOwnNames(t *testing.T) {
|
||||
st := newFakeStore()
|
||||
st.series = []store.SeriesRef{{ID: "emby-series", Name: "Blue Bloods", Year: 2010}}
|
||||
source := &fakeEmby{episodes: []json.RawMessage{episodeWithArtwork("emby-ep-7", 6, 7)}}
|
||||
ingester := testIngester(st, source)
|
||||
|
||||
var announced []IngestResult
|
||||
ingester.Announce = func(_ context.Context, result IngestResult) {
|
||||
announced = append(announced, result)
|
||||
}
|
||||
enqueueOne(t, ingester, episodeRequest())
|
||||
ingester.work(context.Background(), *st.jobs["sonarr:episodefile:8123:551"])
|
||||
|
||||
if len(announced) != 1 {
|
||||
t.Fatalf("expected one announcement, got %d", len(announced))
|
||||
}
|
||||
result := announced[0]
|
||||
if result.ItemID != "emby-ep-7" || result.Name != "The Job" {
|
||||
t.Errorf("announced %q/%q, want Emby's id and episode title", result.ItemID, result.Name)
|
||||
}
|
||||
if result.SeriesName != "Blue Bloods" {
|
||||
t.Errorf("series = %q, want Emby's series name", result.SeriesName)
|
||||
}
|
||||
if result.Season != 6 || result.Episode != 7 {
|
||||
t.Errorf("position = S%02dE%02d, want S06E07", result.Season, result.Episode)
|
||||
}
|
||||
if result.ImageTag != "poster-tag" {
|
||||
t.Errorf("image tag = %q, want the poster from the stored payload", result.ImageTag)
|
||||
}
|
||||
if result.Reason != ReasonImport || result.Kind != KindEpisode {
|
||||
t.Errorf("result = %+v, want the import reason and kind carried through", result)
|
||||
}
|
||||
}
|
||||
|
||||
// Nothing is announced for work that did not land. The banner claims the title is in the
|
||||
// catalogue, so a lookup that found nothing must stay silent and simply be retried.
|
||||
func TestNothingIsAnnouncedWhenEmbyHasNotScannedYet(t *testing.T) {
|
||||
st := newFakeStore()
|
||||
st.series = []store.SeriesRef{{ID: "emby-series", Name: "Blue Bloods", Year: 2010}}
|
||||
ingester := testIngester(st, &fakeEmby{})
|
||||
|
||||
announcements := 0
|
||||
ingester.Announce = func(context.Context, IngestResult) { announcements++ }
|
||||
enqueueOne(t, ingester, episodeRequest())
|
||||
ingester.work(context.Background(), *st.jobs["sonarr:episodefile:8123:551"])
|
||||
|
||||
if announcements != 0 {
|
||||
t.Fatalf("announced %d times for an episode Emby has not scanned", announcements)
|
||||
}
|
||||
}
|
||||
|
||||
func episodeWithArtwork(id string, season, episode int) json.RawMessage {
|
||||
raw, _ := json.Marshal(map[string]any{
|
||||
"Id": id, "Name": "The Job", "Type": "Episode", "SeriesId": "emby-series",
|
||||
"SeriesName": "Blue Bloods",
|
||||
"ParentIndexNumber": season, "IndexNumber": episode,
|
||||
"ImageTags": map[string]string{"Primary": "poster-tag"},
|
||||
})
|
||||
return raw
|
||||
}
|
||||
@@ -250,6 +250,14 @@ func (s *Syncer) run(
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// EmbyCredentials is how the ingest worker borrows the same account the scheduled import
|
||||
// uses. One rule for "who does the gateway talk to Emby as" rather than two, so a
|
||||
// household with a service account configured never has an event-driven read appear in
|
||||
// somebody's Emby history as their television.
|
||||
func (s *Syncer) EmbyCredentials(ctx context.Context) (emby.Credentials, error) {
|
||||
return s.credentials(ctx)
|
||||
}
|
||||
|
||||
// credentials prefers the configured service account and otherwise borrows the most
|
||||
// recent TV session.
|
||||
func (s *Syncer) credentials(ctx context.Context) (emby.Credentials, error) {
|
||||
@@ -333,44 +341,74 @@ func (s *Syncer) Find(ctx context.Context, term string, limit int) ([]json.RawMe
|
||||
return found, nil
|
||||
}
|
||||
|
||||
// disabledSyncPoll is how often a switched-off schedule wakes to ask whether it still is.
|
||||
// A setting an operator has just changed must not need a restart, which is the same reason
|
||||
// the Emby reachability probe keeps ticking slowly while it is off.
|
||||
const disabledSyncPoll = 5 * time.Minute
|
||||
|
||||
// Schedule runs an incremental import on an interval until ctx is cancelled.
|
||||
//
|
||||
// New episodes tend to land through the day and films weekly; an hourly incremental pass
|
||||
// covers both without ever asking Emby for the whole catalogue again.
|
||||
func (s *Syncer) Schedule(ctx context.Context, interval time.Duration, paused ...func() bool) {
|
||||
if interval <= 0 {
|
||||
// It is reconciliation now rather than discovery. Where the *arr webhooks are configured, a
|
||||
// file is in the catalogue within a minute of Sonarr or Radarr putting it there and this
|
||||
// pass exists for what they do not manage — media dropped in by hand, a title edited in
|
||||
// Emby, a webhook that never arrived because the gateway was down. Where they are not, it
|
||||
// is still the only thing that notices anything, which is why the interval is the
|
||||
// operator's rather than a constant.
|
||||
//
|
||||
// interval is a function rather than a value because it is read every cycle: an operator
|
||||
// who has just lengthened the sweep must see that take effect without restarting the
|
||||
// container. Zero means switched off, and this keeps waking to ask.
|
||||
func (s *Syncer) Schedule(
|
||||
ctx context.Context, interval func() time.Duration, paused ...func() bool,
|
||||
) {
|
||||
if interval == nil {
|
||||
s.log.Info("library auto-sync disabled")
|
||||
return
|
||||
}
|
||||
ticker := time.NewTicker(interval)
|
||||
defer ticker.Stop()
|
||||
|
||||
s.log.Info("library auto-sync scheduled", "interval", interval.String())
|
||||
s.log.Info("library auto-sync scheduled", "interval", durationLabel(interval()))
|
||||
for {
|
||||
wait := interval()
|
||||
disabled := wait <= 0
|
||||
if disabled {
|
||||
wait = disabledSyncPoll
|
||||
}
|
||||
timer := time.NewTimer(wait)
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
timer.Stop()
|
||||
return
|
||||
case <-ticker.C:
|
||||
if len(paused) > 0 && paused[0] != nil && paused[0]() {
|
||||
s.log.Debug("skipping scheduled sync; quiet time is active")
|
||||
case <-timer.C:
|
||||
}
|
||||
timer.Stop()
|
||||
if disabled {
|
||||
continue
|
||||
}
|
||||
if len(paused) > 0 && paused[0] != nil && paused[0]() {
|
||||
s.log.Debug("skipping scheduled sync; quiet time is active")
|
||||
continue
|
||||
}
|
||||
if s.Running() {
|
||||
s.log.Info("skipping scheduled sync; one is already running")
|
||||
continue
|
||||
}
|
||||
if _, err := s.Sync(ctx, "incremental", "schedule"); err != nil {
|
||||
if errors.Is(err, ErrNoCredentials) {
|
||||
// Nobody has signed in yet. Not worth an error-level log every hour.
|
||||
s.log.Info("skipping scheduled sync; no credentials yet")
|
||||
continue
|
||||
}
|
||||
if s.Running() {
|
||||
s.log.Info("skipping scheduled sync; one is already running")
|
||||
continue
|
||||
}
|
||||
if _, err := s.Sync(ctx, "incremental", "schedule"); err != nil {
|
||||
if errors.Is(err, ErrNoCredentials) {
|
||||
// Nobody has signed in yet. Not worth an error-level log every hour.
|
||||
s.log.Info("skipping scheduled sync; no credentials yet")
|
||||
continue
|
||||
}
|
||||
s.log.Error("scheduled sync failed", "error", err)
|
||||
}
|
||||
s.log.Error("scheduled sync failed", "error", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func durationLabel(value time.Duration) string {
|
||||
if value <= 0 {
|
||||
return "off"
|
||||
}
|
||||
return value.String()
|
||||
}
|
||||
|
||||
// syncItem mirrors the Emby fields promoted to columns.
|
||||
type syncItem struct {
|
||||
ID string `json:"Id"`
|
||||
|
||||
@@ -46,6 +46,14 @@ type GatewaySettings struct {
|
||||
RadarrAlertMinutes int `json:"radarrAlertMinutes"`
|
||||
// EmbyHealthSeconds is how often the reachability probe asks Emby whether it is there.
|
||||
EmbyHealthSeconds int `json:"embyHealthSeconds"`
|
||||
// LibrarySyncMinutes is how often the catalogue sweep asks Emby what changed.
|
||||
//
|
||||
// It is an override worth having because the answer now depends on the household's
|
||||
// wiring rather than on the gateway: with both *arr webhooks configured, a new file is
|
||||
// in the catalogue within a minute of landing and the sweep is reconciliation for
|
||||
// media Sonarr and Radarr do not manage — six hours rather than one. With no webhooks
|
||||
// it is still the only way anything is discovered and must stay frequent.
|
||||
LibrarySyncMinutes int `json:"librarySyncMinutes"`
|
||||
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
UpdatedBy string `json:"updatedBy,omitempty"`
|
||||
@@ -78,6 +86,9 @@ func normalizeGatewaySettings(settings GatewaySettings) GatewaySettings {
|
||||
settings.SonarrAlertMinutes = clampOverride(settings.SonarrAlertMinutes, 1, 24*60, true)
|
||||
settings.RadarrAlertMinutes = clampOverride(settings.RadarrAlertMinutes, 1, 7*24*60, true)
|
||||
settings.EmbyHealthSeconds = clampOverride(settings.EmbyHealthSeconds, 10, 3600, true)
|
||||
// A day is the ceiling rather than a week: however well the webhooks are working, the
|
||||
// sweep is the only thing that ever notices a file somebody moved by hand.
|
||||
settings.LibrarySyncMinutes = clampOverride(settings.LibrarySyncMinutes, 5, 24*60, true)
|
||||
return settings
|
||||
}
|
||||
|
||||
|
||||
@@ -120,6 +120,70 @@ func (s *Store) DeleteLibraryItemsBefore(ctx context.Context, cutoff time.Time)
|
||||
return tag.RowsAffected(), nil
|
||||
}
|
||||
|
||||
// NamedItem is the least a caller can be told about a catalogue row and still identify
|
||||
// it: what it is called and, where the library knows, when it came out.
|
||||
type NamedItem struct {
|
||||
ID string
|
||||
Name string
|
||||
Year int
|
||||
}
|
||||
|
||||
// LibraryItemsByName finds catalogue rows by title, case-insensitively.
|
||||
//
|
||||
// The comparison that decides the answer is not this one: the caller normalises both
|
||||
// sides (punctuation and spacing are where an *arr and Emby actually differ) and picks by
|
||||
// year. This is the narrowing query — a handful of rows out of twenty thousand — so that
|
||||
// the matching rule can stay a pure function with one definition.
|
||||
func (s *Store) LibraryItemsByName(ctx context.Context, itemType, name string) ([]NamedItem, error) {
|
||||
trimmed := strings.TrimSpace(name)
|
||||
if trimmed == "" {
|
||||
return nil, nil
|
||||
}
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT id, name, COALESCE(production_year, 0)
|
||||
FROM library_items
|
||||
WHERE type = $1 AND lower(name) = lower($2)
|
||||
LIMIT 50`, itemType, trimmed)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("store: library items by name: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []NamedItem{}
|
||||
for rows.Next() {
|
||||
var item NamedItem
|
||||
if err := rows.Scan(&item.ID, &item.Name, &item.Year); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, item)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// DeleteLibraryItem removes one item and anything derived from it.
|
||||
//
|
||||
// The credits marker goes with it, and that is the point of doing this in one place: the
|
||||
// marker table is keyed on the item id and nothing else prunes it, so a title deleted from
|
||||
// the library would otherwise leave a Skip Credits position behind for a file that no
|
||||
// longer exists — and if that id were ever reused, in front of the wrong programme.
|
||||
//
|
||||
// Deleting a series takes its episodes with it, because Emby's own hierarchy is the only
|
||||
// thing that made those rows meaningful.
|
||||
func (s *Store) DeleteLibraryItem(ctx context.Context, itemID string) (int64, error) {
|
||||
if strings.TrimSpace(itemID) == "" {
|
||||
return 0, nil
|
||||
}
|
||||
tag, err := s.pool.Exec(ctx,
|
||||
`DELETE FROM library_items WHERE id = $1 OR series_id = $1`, itemID)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("store: delete library item: %w", err)
|
||||
}
|
||||
if _, err := s.pool.Exec(ctx,
|
||||
`DELETE FROM credits_markers WHERE item_id = $1 OR series_id = $1`, itemID); err != nil {
|
||||
return 0, fmt.Errorf("store: delete credits markers: %w", err)
|
||||
}
|
||||
return tag.RowsAffected(), nil
|
||||
}
|
||||
|
||||
// SearchLibrary answers from the imported library rather than Emby.
|
||||
//
|
||||
// Full-text match first, with a trailing ILIKE so partial words ("sever") still hit
|
||||
|
||||
@@ -0,0 +1,216 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// The durable side of event-driven ingest.
|
||||
//
|
||||
// One table, five queries, and the only interesting one is the insert: it is written
|
||||
// ON CONFLICT on a key derived from the file, which is the whole of what makes repeated
|
||||
// webhook delivery safe. Sonarr and Radarr both re-notify on retry and neither guarantees
|
||||
// exactly-once, so "the same news twice" has to be an ordinary event rather than a
|
||||
// duplicate row and a duplicate Emby lookup.
|
||||
|
||||
// Ingest states.
|
||||
const (
|
||||
IngestPending = "pending"
|
||||
IngestDone = "done"
|
||||
IngestFailed = "failed"
|
||||
)
|
||||
|
||||
// IngestRetention is how long settled rows are kept. Long enough that an operator asking
|
||||
// "did the webhook fire when that episode landed last week" gets an answer, short enough
|
||||
// that a household importing all day does not accumulate a table nobody reads.
|
||||
const IngestRetention = 14 * 24 * time.Hour
|
||||
|
||||
// IngestJob is one row of work.
|
||||
type IngestJob struct {
|
||||
Key string `json:"key"`
|
||||
Action string `json:"action"`
|
||||
Kind string `json:"kind"`
|
||||
Reason string `json:"reason"`
|
||||
Source string `json:"source"`
|
||||
Payload json.RawMessage `json:"payload"`
|
||||
State string `json:"state"`
|
||||
Outcome string `json:"outcome"`
|
||||
ItemID string `json:"itemId"`
|
||||
Attempts int `json:"attempts"`
|
||||
LastError string `json:"lastError"`
|
||||
DueAt time.Time `json:"dueAt"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
// IngestCounts is what the console reads beside the list.
|
||||
type IngestCounts struct {
|
||||
Pending int `json:"pending"`
|
||||
Done int `json:"done"`
|
||||
Failed int `json:"failed"`
|
||||
}
|
||||
|
||||
// EnqueueIngest records a piece of work, or refreshes one already waiting.
|
||||
//
|
||||
// The second return reports whether this delivery was news. A repeat is not an error and
|
||||
// not a second row — it moves the existing row's due time no earlier and is logged at
|
||||
// DEBUG, because a Sonarr that retried is an ordinary occurrence and not something an
|
||||
// operator needs told about.
|
||||
//
|
||||
// A key that has already been *settled* is deliberately re-opened: the same file can
|
||||
// legitimately be imported, deleted and imported again, and a row left at 'done' would
|
||||
// swallow the second import for ever.
|
||||
func (s *Store) EnqueueIngest(ctx context.Context, job IngestJob) (bool, error) {
|
||||
if job.Key == "" || job.Action == "" {
|
||||
return false, fmt.Errorf("store: ingest job needs a key and an action")
|
||||
}
|
||||
if len(job.Payload) == 0 {
|
||||
job.Payload = json.RawMessage(`{}`)
|
||||
}
|
||||
if job.DueAt.IsZero() {
|
||||
job.DueAt = time.Now().UTC()
|
||||
}
|
||||
var inserted bool
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
INSERT INTO library_ingest_queue
|
||||
(key, action, kind, reason, source, payload, state, due_at, created_at, updated_at)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, 'pending', $7, now(), now())
|
||||
ON CONFLICT (key) DO UPDATE SET
|
||||
action = EXCLUDED.action,
|
||||
kind = EXCLUDED.kind,
|
||||
reason = EXCLUDED.reason,
|
||||
source = EXCLUDED.source,
|
||||
payload = EXCLUDED.payload,
|
||||
state = 'pending',
|
||||
outcome = '',
|
||||
last_error = '',
|
||||
-- A re-delivery must never pull the settle delay forward: the point of it is
|
||||
-- that the file has finished being written, and an eager retry would ask Emby
|
||||
-- about a file it has not scanned yet.
|
||||
due_at = GREATEST(library_ingest_queue.due_at, EXCLUDED.due_at),
|
||||
-- Attempts reset only when the row had settled. A retry storm against a row
|
||||
-- still being worked must not reset its backoff.
|
||||
attempts = CASE WHEN library_ingest_queue.state = 'pending'
|
||||
THEN library_ingest_queue.attempts ELSE 0 END,
|
||||
updated_at = now()
|
||||
RETURNING (xmax = 0)`,
|
||||
job.Key, job.Action, job.Kind, job.Reason, job.Source, job.Payload, job.DueAt.UTC(),
|
||||
).Scan(&inserted)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("store: enqueue ingest: %w", err)
|
||||
}
|
||||
return inserted, nil
|
||||
}
|
||||
|
||||
// ClaimIngest takes the work that is due, oldest first.
|
||||
//
|
||||
// It marks nothing: the worker is single and in-process, so a claim flag would be state to
|
||||
// get wrong (a row left claimed by a container that was killed) in exchange for protecting
|
||||
// against a second worker that does not exist. FinishIngest is what moves a row on.
|
||||
func (s *Store) ClaimIngest(ctx context.Context, now time.Time, limit int) ([]IngestJob, error) {
|
||||
if limit <= 0 {
|
||||
limit = 10
|
||||
}
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT key, action, kind, reason, source, payload, state, outcome, item_id,
|
||||
attempts, last_error, due_at, created_at, updated_at
|
||||
FROM library_ingest_queue
|
||||
WHERE state = 'pending' AND due_at <= $1
|
||||
ORDER BY due_at
|
||||
LIMIT $2`, now.UTC(), limit)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("store: claim ingest: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
return scanIngestJobs(rows)
|
||||
}
|
||||
|
||||
// FinishIngest settles a row, or schedules the next attempt.
|
||||
//
|
||||
// state is 'done', 'failed' or 'pending' — the last being a deferral, which is the
|
||||
// ordinary answer for a file Emby has not scanned in yet.
|
||||
func (s *Store) FinishIngest(
|
||||
ctx context.Context, key, state, outcome, itemID, errorText string, retryAt time.Time,
|
||||
) error {
|
||||
due := retryAt
|
||||
if due.IsZero() {
|
||||
due = time.Now().UTC()
|
||||
}
|
||||
_, err := s.pool.Exec(ctx, `
|
||||
UPDATE library_ingest_queue
|
||||
SET state = $2, outcome = $3, item_id = $4, last_error = $5,
|
||||
attempts = attempts + 1, due_at = $6, updated_at = now()
|
||||
WHERE key = $1`, key, state, outcome, itemID, errorText, due.UTC())
|
||||
if err != nil {
|
||||
return fmt.Errorf("store: finish ingest: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// RecentIngests is the console's read: newest activity first, whatever its state.
|
||||
func (s *Store) RecentIngests(ctx context.Context, limit int) ([]IngestJob, error) {
|
||||
if limit <= 0 || limit > 200 {
|
||||
limit = 50
|
||||
}
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT key, action, kind, reason, source, payload, state, outcome, item_id,
|
||||
attempts, last_error, due_at, created_at, updated_at
|
||||
FROM library_ingest_queue
|
||||
ORDER BY updated_at DESC
|
||||
LIMIT $1`, limit)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("store: recent ingests: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
return scanIngestJobs(rows)
|
||||
}
|
||||
|
||||
// IngestStateCounts is the summary above that list.
|
||||
func (s *Store) IngestStateCounts(ctx context.Context) (IngestCounts, error) {
|
||||
var counts IngestCounts
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
SELECT
|
||||
COUNT(*) FILTER (WHERE state = 'pending'),
|
||||
COUNT(*) FILTER (WHERE state = 'done'),
|
||||
COUNT(*) FILTER (WHERE state = 'failed')
|
||||
FROM library_ingest_queue`).Scan(&counts.Pending, &counts.Done, &counts.Failed)
|
||||
if err != nil {
|
||||
return IngestCounts{}, fmt.Errorf("store: ingest counts: %w", err)
|
||||
}
|
||||
return counts, nil
|
||||
}
|
||||
|
||||
// PruneIngests removes settled rows past their retention. Pending work is never pruned:
|
||||
// a row still waiting is work nobody has done, however old it is.
|
||||
func (s *Store) PruneIngests(ctx context.Context, retention time.Duration) (int64, error) {
|
||||
if retention <= 0 {
|
||||
return 0, nil
|
||||
}
|
||||
tag, err := s.pool.Exec(ctx, `
|
||||
DELETE FROM library_ingest_queue
|
||||
WHERE state <> 'pending' AND updated_at < $1`, time.Now().UTC().Add(-retention))
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("store: prune ingests: %w", err)
|
||||
}
|
||||
return tag.RowsAffected(), nil
|
||||
}
|
||||
|
||||
func scanIngestJobs(rows pgx.Rows) ([]IngestJob, error) {
|
||||
out := []IngestJob{}
|
||||
for rows.Next() {
|
||||
var job IngestJob
|
||||
if err := rows.Scan(
|
||||
&job.Key, &job.Action, &job.Kind, &job.Reason, &job.Source, &job.Payload,
|
||||
&job.State, &job.Outcome, &job.ItemID, &job.Attempts, &job.LastError,
|
||||
&job.DueAt, &job.CreatedAt, &job.UpdatedAt,
|
||||
); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, job)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
@@ -792,3 +792,42 @@ CREATE INDEX IF NOT EXISTS credits_scan_history_item_time_idx
|
||||
ON credits_scan_history (item_id, finished_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS credits_scan_history_time_idx
|
||||
ON credits_scan_history (finished_at DESC);
|
||||
|
||||
-- Work Sonarr and Radarr told the gateway about.
|
||||
--
|
||||
-- This is the one queue in the schema that is durable, and the reason is that a webhook is
|
||||
-- gone once it has been dropped: a Tracearr-derived credits candidate is rebuilt from one
|
||||
-- query on restart, while "Sonarr imported this at 19:05" cannot be rederived from
|
||||
-- anything. A container restarted during the settle delay must still re-read the file.
|
||||
--
|
||||
-- The key is derived from the *file* rather than from the delivery, so ON CONFLICT is what
|
||||
-- makes repeated webhook delivery safe: two notifications about one import collapse onto
|
||||
-- one row, while a file deleted and re-imported is a different file and its own work.
|
||||
--
|
||||
-- Completed rows are kept rather than deleted. They are the operator's record of why an
|
||||
-- item was re-read, which is the question the Imports page exists to answer; housekeeping
|
||||
-- prunes them.
|
||||
CREATE TABLE IF NOT EXISTS library_ingest_queue (
|
||||
key TEXT PRIMARY KEY,
|
||||
action TEXT NOT NULL,
|
||||
kind TEXT NOT NULL,
|
||||
reason TEXT NOT NULL DEFAULT '',
|
||||
source TEXT NOT NULL DEFAULT '',
|
||||
payload JSONB NOT NULL DEFAULT '{}'::jsonb,
|
||||
state TEXT NOT NULL DEFAULT 'pending',
|
||||
outcome TEXT NOT NULL DEFAULT '',
|
||||
item_id TEXT NOT NULL DEFAULT '',
|
||||
attempts INT NOT NULL DEFAULT 0,
|
||||
last_error TEXT NOT NULL DEFAULT '',
|
||||
due_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
-- The worker's only query: what is due. Partial, because settled rows outnumber pending
|
||||
-- ones by orders of magnitude within a day of the feature being switched on.
|
||||
CREATE INDEX IF NOT EXISTS library_ingest_pending_idx
|
||||
ON library_ingest_queue (due_at)
|
||||
WHERE state = 'pending';
|
||||
CREATE INDEX IF NOT EXISTS library_ingest_recent_idx
|
||||
ON library_ingest_queue (updated_at DESC);
|
||||
|
||||
Reference in New Issue
Block a user