fix(api): honour default TTLs and expire stale job ledger rows (#2887)

fluxer_docs/src/content/docs/admin-api/blocklists.mdx

@@ -13,7 +13,7 @@ Each list has its own [Admin ACLs](/admin-api/#acl-registry). A read needs the s
 Fluxer builds each permission name from `ban:`, the list name with each hyphen written as an underscore, and the verb, so the `url-domain` list uses `ban:url_domain:check`, `ban:url_domain:add`, and `ban:url_domain:remove`. The `email-domain-suspicious` list is the one exception and uses `suspicious_email_domain:check`, `suspicious_email_domain:add`, and `suspicious_email_domain:remove`.
 
 :::note[The Admin API exposes no other blocklist]
-Fluxer synchronises disposable email domains from external feeds every six hours. No operation on this page reads or writes them.
+Fluxer synchronises disposable email domains from external feeds every six hours when `FLUXER_BLOCKLIST_FEEDS_ENABLED` is on. No operation on this page reads or writes them.
 :::
 
 ## Blocklist types
@@ -67,6 +67,8 @@ Every `url`, `url-domain`, `file-sha`, and `avatar-hash` row has a category nami
 | gifct | Imported from a GIFCT hash set |
 | stop_ncii | Imported from a StopNCII hash set |
 
+With blocklist feeds off, Fluxer ignores and removes every `file-sha` row in the `malware_bazaar` category that no Admin added.
+
 The request field is a free string of 1 through 64 characters, so a value outside this registry is accepted and stored verbatim. Tolerate a stored category outside the registry.
 
 ## Content blocklist severities

fluxer_docs/src/content/docs/admin-api/jobs.mdx

@@ -11,7 +11,7 @@ These routes list recorded background jobs and request cancellation. Create jobs
 Reads require `jobs:view` and cancellation requires `jobs:cancel`. Every operation on this page shares the `admin:jobs:view` bucket. The three reads record no [Admin audit entry](/admin-api/#admin-audit-entry-object), and cancellation records one.
 
 :::note[Job updates vary by task]
-Mention processing, link previews, and every scheduled task except `syncDisposableEmailDomains`, `syncUrlBlocklists` and `syncFileShaBlocklists` run with no job record and never appear here. A job also runs with no record when Fluxer fails to write that record. When a later write of status, progress or attempts fails, Fluxer logs the failure and the job continues, so the stored values can be behind the real run. Use the [archive routes](/admin-api/archives/) to track archive progress and failures.
+Mention processing, link previews, message delete audit log batching, and every scheduled task except `syncDisposableEmailDomains`, `syncUrlBlocklists` and `syncFileShaBlocklists` run with no job record and never appear here. Fluxer deletes a job record about 90 days after the job was queued. A job also runs with no record when Fluxer fails to write that record. When a later write of status, progress or attempts fails, Fluxer logs the failure and the job continues, so the stored values can be behind the real run. Use the [archive routes](/admin-api/archives/) to track archive progress and failures.
 :::
 
 ## Admin job object
@@ -108,6 +108,8 @@ Pass all three fields from `next_cursor` back to [List jobs](#list-jobs) using t
 
 `queued` and `running` accept cancellation requests. The other statuses are terminal. A cancellation request does not guarantee that the task will stop.
 
+The queue drops a job 7 days after it was queued. A job still recorded as `queued` or `running` after 8 days is set to `deadletter` with the error `Expired from the job queue`.
+
 ## Processing lanes
 
 `jet_stream_lane` identifies the job's processing group.
@@ -139,12 +141,12 @@ Returns a page of [Admin job](#admin-job-object) objects, newest first, within t
 | cursor_bucket_day? | string | `next_cursor.bucket_day` as a `YYYY-MM-DD` UTC date |
 | cursor_created_at? | string | `next_cursor.created_at` as an ISO 8601 timestamp (1-64 characters) |
 | cursor_job_id? | snowflake | The job identifier from `next_cursor.job_id` |
-| max_lookback_days? | integer | The number of days before the cursor date or today to include (1-60, default 14) |
+| max_lookback_days? | integer | The number of days before today to include (1-60, default 14) |
 | status? | string | Filter by recorded [job status](#job-statuses) |
 | task_type? | string | Filter by [task type](#background-job-task-types) (1-128 characters) |
 | requested_by_user_id? | snowflake | Filter by recorded requester |
 
-Supply all three cursor parameters together or omit all three. An incomplete or malformed cursor returns 400 `INVALID_FORM_BODY`. An unknown task type returns an empty page.
+Supply all three cursor parameters together or omit all three. An incomplete or malformed cursor returns 400 `INVALID_FORM_BODY`. An unknown task type returns an empty page. A full page always returns `next_cursor`, so the next request can return an empty page.
 
 ### Response body
 

fluxer_docs/src/content/docs/operator/configuration.mdx

@@ -989,7 +989,9 @@ Default `flag_spammer`. What happens when it fires. `flag_spammer` or `suppress_
 
 #### `FLUXER_BLOCKLIST_FEEDS_ENABLED`
 
-Defaults to the inverse of `FLUXER_SELF_HOSTED`. External blocklist feeds. Off by default on a self-hosted instance.
+Defaults to the inverse of `FLUXER_SELF_HOSTED`. Off by default on a self-hosted instance. With feeds on, the worker downloads disposable email domains and the URLhaus and PhishTank URL lists every six hours, and MalwareBazaar file hashes every twelve hours. Fluxer checks registrations and email changes against the domains, posted links against the URLs, and uploads against the hashes.
+
+With feeds off, Fluxer checks none of this data. The first worker start with feeds off removes every disposable email domain, the URL feed file, and every `malware_bazaar` file-SHA ban that no Admin added. File-SHA bans added through the Admin API stay. Turning feeds back on downloads the disposable domains at the next worker start, the URLs within six hours, and the hashes within twelve.
 
 #### `FLUXER_TOR_EXIT_LIST_ENABLED`
 
@@ -1776,7 +1778,7 @@ Default `256mb`. The ceiling for `edge`.
 
 #### `FLUXER_POSTGRES_MEMORY_LIMIT`
 
-Default `5gb`. The ceiling for `postgres`. Must be at or above `FLUXER_POSTGRES_MEMORY_RESERVATION` or the container fails to create. The 256mb `shm_size` is charged against it.
+Default `5gb`. The ceiling for `postgres`. Must be at or above `FLUXER_POSTGRES_MEMORY_RESERVATION` or the container fails to create. Shared memory in use under `FLUXER_POSTGRES_SHM_SIZE` is charged against it.
 
 #### `FLUXER_VALKEY_MEMORY_LIMIT`
 
@@ -1932,7 +1934,11 @@ Default `256MB`. The budget for one VACUUM, CREATE INDEX, or ALTER TABLE. One su
 
 Default `128MB`. The budget for each autovacuum worker. Postgres runs three workers by default, so budget three times this value.
 
-The remaining Postgres settings are fixed on the command line, with no variable of their own: `min_wal_size=512MB`, `max_wal_size=2GB`, `wal_buffers=16MB`, and `shm_size: 256mb` on the service itself.
+#### `FLUXER_POSTGRES_SHM_SIZE`
+
+Default `1gb`. The shared memory of the `postgres` container, used by parallel queries and parallel maintenance. Keep it well above `FLUXER_POSTGRES_MAINTENANCE_WORK_MEM`. A manual VACUUM sizes its shared memory from that budget, and fails with `could not resize shared memory segment` when it does not fit.
+
+The WAL settings `min_wal_size=512MB`, `max_wal_size=2GB` and `wal_buffers=16MB` are fixed on the command line, with no variable of their own.
 
 The bundled Valkey uses persistent storage. These settings control its memory limit and behaviour when full.
 

fluxer_docs/src/content/docs/operator/upgrading.mdx

@@ -182,6 +182,18 @@ When `api` starts, it writes the new address to the existing `default-server-1`
 
 Voice media is unaffected. It never went through the edge, and `7881/tcp` and `7882/udp` still reach the host directly.
 
+## Short-lived data expires on Postgres
+
+On the Postgres backend, data the schema keeps for a limited time expires the way it does on Cassandra, counted from its last write. After the upgrade, these are the differences most people notice.
+
+- The mentions inbox lists the last 7 days of mentions.
+- A device that has not opened Fluxer in 90 days gets no push notifications until Fluxer is opened on it again.
+- An account's ended sessions in the admin dashboard go back 30 days.
+
+Within minutes of the upgrade, `worker` starts deleting existing data that is past that lifetime. A rollback does not bring it back. [Restore a backup](#restore-a-backup) does. `worker` checks again once a day, so data written while a rollback was in place gets the same treatment after the next upgrade.
+
+Instances on Cassandra already behave this way.
+
 ## What the backup covers
 
 Each upgrade writes one record directory, named `record-` and a UTC stamp, under `backups` or wherever `--backup-dir` points. It holds: