0.2.78

CLAUDE.md

@@ -962,6 +962,47 @@ each page read as a pile of unrelated controls. Things to preserve:
   the page that can do something about it. A screen that both summarises and changes state
   is where an accidental click lives.
 
+**"25 goroutines" was the one figure on the console nobody could act on**, and
+`server/internal/runtimestats` is what replaced it. A count says nothing about what those
+goroutines are doing, which part of Memby they belong to, whether it is normal, or whether
+it has been climbing all week — so the overview's Process card now prints a *verdict* and
+signposts `/admin/runtime`, where the detail lives. The package is organised entirely by
+what a thing costs, and that is the design:
+
+- **The registry is free**, so it runs always. A long-running worker is started through
+  `runtimestats.Go(name, component, fn)` rather than a bare `go`, at every launch site in
+  `main.go` plus the scheduler's loop and the integrations dispatcher, and the console can
+  therefore *name* "Library ingest" instead of inferring it. `Starts` is on the record
+  because a worker being restarted in a loop reads exactly like a healthy one from a single
+  snapshot. Nothing in the package recovers a panic — reporting must never change what it
+  is reporting on.
+- **The sampler is cheap**, so it runs on a minute's tick: a ring of `Sample`s four hours
+  deep. A single instantaneous number cannot show a leak and this can. `trendOf` is pure and
+  compares the median of the window's oldest quarter with its newest rather than fitting a
+  line, because a burst — an import, a household arriving home — is exactly the shape
+  least-squares reports as a trend; the threshold is proportional *with an absolute floor*,
+  or a busy gateway is permanently "rising" and a quiet one calls one extra goroutine a leak.
+  `DirectionUnknown` is a real answer and the honest one for the first quarter of an hour.
+- **The breakdown is expensive**, so it is a button. `CollectGoroutines` walks every stack,
+  which stops the world, and the answer says when it was taken and what it cost.
+  `parseGoroutines` is pure and pinned by a fixture carrying a real dump, and the categories
+  must always **partition the total** — a breakdown whose parts do not add up is worse than
+  none, which is what the unrecognised-state bucket and both admin tests exist for. The
+  `componentRules` order is load-bearing: Memby's own packages are tested before the
+  libraries they call, or every query in the gateway files under "Database pool".
+  `?format=text` hands over the raw dump, which is the alternative to leaving a profiler
+  endpoint permanently mounted.
+- **The verdict is the gateway's wording, not the console's** (`assess`, pure and tested),
+  the stance every label the gateway prints takes: the threshold and the sentence explaining
+  it belong together, and an older console must not be the thing deciding what "watch"
+  means. Every note carries an **area** — a number that has moved is only actionable once it
+  points somewhere — and a rise slower than `goroutineLeakPerHour` is not a note at all, or
+  the card cries wolf on every busy evening.
+- **`Process` is what /proc knows and the runtime does not** — processor time, open sockets,
+  the descriptor limit. `CPUKnown`/`FilesKnown` are separate from the figures because "no
+  open sockets" and "could not look" are different answers, and a developer machine is the
+  second.
+
 **The gateway's own settings are `/admin/settings`**, reached from the account menu in the
 top bar rather than from the rail — every other page decides what the *televisions* do, and
 this one is about the server process. It is `store.GatewaySettings` (one `app_settings` row)
@@ -990,6 +1031,82 @@ the idle sign-out, the two alert windows and the Emby health probe. Things to pr
   believing it did not work. `deployedLogLevel` is remembered because clearing the
   override has to restore *something*, and the variable itself has by then been moved.
 
+**Integrations are their own area, and they are one axis over the scheduler rather than a
+second system.** `/admin/integrations` is the overview, `/admin/integrations/{id}` is one
+service, and `internal/api/integrations_catalogue.go` is the only place a service is
+declared — the `featureCatalogue` shape. Before it, MDBList's settings were on a Movie
+ratings page, the Sonarr and Radarr switches were on a page about Discord webhooks, their
+request policies were beside those, and Tracearr could not be reached from the console at
+all: "is everything Memby depends on working" was four pages and one impossible question.
+Things to preserve:
+
+- **A run history is not a second store.** An integration run *is* a
+  `scheduled_task_runs` row: `scheduler.Task` carries an `Integration` id, the row records
+  it, and `store.IntegrationRuns` reads the same table along the other axis. Giving the
+  area a table of its own would mean two schedulers, two retention jobs and two places one
+  piece of work could be recorded as having failed. What was added is four counters
+  (`store.RunCounts` — processed, changed, skipped, failed) and `scheduler.Work`, the
+  counting form of `Run`; a task declares one or the other and `Register` panics on both.
+  The counters exist because "412 films checked, 7 updated" is the question an operator
+  has and a sentence is a poor place to keep numbers — they cannot be compared between
+  runs or sorted.
+- **Runs and Logs answer different questions and say so.** Logs are the technical events
+  behind a failure; this is the operational record of what Memby attempted and what came of
+  it. A failing service's page links to `/admin/logs?q=<id>` — which is why the log viewer
+  seeds its text filter from `?q=` — so the two are joined rather than merged.
+- **The work is scheduler tasks now, and that is the half that changed behaviour.** The
+  Sonarr lifecycle scan, the Radarr catalogue refresh, the Tracearr import and the daily
+  For You rebuild were four goroutines with tickers of their own in `main`. A ticker could
+  report nothing to an operator, could not be started by hand, and could not be counted
+  against a service. `foryou.Service.Schedule` and `nextDailyRebuild` are gone with them;
+  the "is the rebuild due" rule moved to `api.forYouRebuildDue`, which is pure and tested
+  and compares **local calendar days** rather than subtracting 24 hours.
+- **A switched-off integration reports a *skipped* run, never silence and never an error.**
+  Silence is indistinguishable from a scheduler that has stopped, and an error would put a
+  red row in the console for a state the operator chose.
+- **One stored truth per integration, and one reader.** The switch lives where that
+  service's configuration already lives — Sonarr and Radarr in `ArrIntegrationPolicy`,
+  MDBList in `MDBListSettings.Enabled`, Tracearr in the new `integration_policy` document
+  — and `integrationEnabled`/`SetEnabled` are the only ways in. Consolidating them would
+  be a migration with a window in which two documents disagreed about whether a service
+  was on. Absence means **on**, or the day this shipped would have switched every
+  household's imports off.
+- **`integrationSuppressed` is not the negation of `integrationEnabled`.** The latter is
+  also false for a service this deployment never configured, and a household can perfectly
+  well point Sonarr's *webhook* at Memby without giving Memby Sonarr's API key. The import
+  hooks use the former, and answer 200 while ignoring the payload — neither *arr
+  re-delivers a rejection, so refusing would read as Memby losing imports rather than as
+  the switch the operator set.
+- **A switch consulted once at start-up is not a switch.** `recommend.Engine.TracearrAllowed`
+  is a function read at call time, wired in `main` after the server exists, and nil means
+  allowed so every test behaves as it did. Switching Tracearr off costs the extra signal
+  and never the row — the same degradation an unreachable Tracearr already produced.
+- **Health is probed on a schedule, not on page load.** The console polls; probing per
+  request would make every open tab its own request against somebody's Sonarr. One
+  `integration-health` task every five minutes fills `integrationHealthCache`, and the
+  console's Test button calls the *same* `probeIntegration` — a test taking a different
+  path could report a service as working while the row beside it stayed red. A switched-off
+  service is not probed at all, which is what the switch promises.
+- **MDBList has no probe, and the page says so rather than drawing "never checked".**
+  Its allowance is bought by the day, and spending a request of it to colour a status dot
+  would have the console competing with the televisions for the thing it is reporting on.
+  Its health comes from its own run history.
+- **The status word is the gateway's** (`integrationStatus`, pure and tested), the stance
+  every label Memby prints takes. It resolves in priority order — unconfigured, disabled,
+  running, then the evidence — and a failure followed by a success is *history rather than
+  a verdict*, or one bad night would leave a service red until the retention job removed
+  the row. `idle` is a real answer for a service that has not done anything yet; claiming
+  either verdict would be a guess.
+- **Switching one off states what goes with it.** `Powers` in the catalogue is rendered on
+  the service's page and repeated in the activity event, because "Sonarr disabled" tells an
+  operator nothing they did not just do — what they may not have thought about is the
+  television calendar going with it.
+- **`/admin/ratings` redirects** to the MDBList page rather than 404ing: it was
+  bookmarkable and appears in older activity links. The old Integrations page survives as
+  `/admin/integrations/webhooks`, which answers the opposite question from the rest of the
+  area — those are services Memby *depends on*, that is a place Memby *posts to*, and
+  removing every webhook leaves the gateway unchanged.
+
 **Two things a television does are notifications now**, both in `internal/api/device_activity.go`.
 `TypeDeviceFirstUse` announces the first time a set opened Memby on a household-local day
 and `TypeDeviceUpdated` announces one that finished updating itself. Things to preserve: