pepepper/fluxer
1
0
Fork 0
mirror of https://github.com/fluxerapp/fluxer synced 2026-10-07 19:22:14 +09:00
Code Issues Packages Projects Releases Wiki Activity

feat(voice): add noise suppression backends and rollout (#2706)

Browse Source
This commit is contained in:
Hampus
2026-09-11 22:24:05 +02:00
committed by GitHub
parent b30a4f5d14
commit 172791316b
97 changed files with 10877 additions and 3157 deletions
Download Patch File Download Diff File Expand all files Collapse all files
fluxer_docs/astro.config.ts
+1 -1
View File
@@ -240,7 +240,7 @@ export default defineConfig({
},
{
label: 'Client surfaces',
items: ['http-api/themes', 'http-api/downloads'],
items: ['http-api/experiments', 'http-api/themes', 'http-api/downloads'],
},
{
label: 'Safety',
fluxer_docs/scripts/VerifyDocsStyle.ts
+1 -1
View File
@@ -303,7 +303,7 @@ const ACCEPTED_TABLE_FINDINGS = new Map<string, Readonly<Partial<Record<TableRul
['admin-api/discovery.mdx', {'table-identifier': 1}],
['admin-api/guilds.mdx', {'table-identifier': 3}],
['admin-api/index.mdx', {'table-fit': 1, 'table-identifier': 2}],
['admin-api/instance.mdx', {'table-identifier': 5}],
['admin-api/instance.mdx', {'table-fit': 1, 'table-identifier': 5}],
['admin-api/messages.mdx', {'table-identifier': 1}],
['admin-api/reports.mdx', {'table-fit': 1, 'table-identifier': 2}],
['admin-api/users.mdx', {'table-fit': 1, 'table-identifier': 1}],
fluxer_docs/src/content/docs/admin-api/instance.mdx
+51 -1
View File
@@ -32,6 +32,8 @@ The complete runtime configuration of the deployment. Every configuration operat
| --- | --- | --- |
| sso | [SSO configuration](#sso-configuration-object) object | Single sign-on settings |
| gateway_rollout | [Gateway rollout configuration](#gateway-rollout-configuration-object) object | Gateway admission and dispatch tuning |
| voice_noise_suppression | [voice noise suppression configuration](#voice-noise-suppression-configuration-object) object | Client-side noise suppression rollout |
| experiment_delivery | [experiment delivery configuration](#experiment-delivery-configuration-object) object | Cadence every client polls the experiments route on |
| registration | [registration configuration](#registration-configuration-object) object | Registration policy, issued URLs, and pending registrations |
| self_hosted | boolean | Whether the deployment runs in self-hosted mode |
| app_public | [public application configuration](#public-application-configuration-object) object | Branding, legal, setup, and registration field policy |
@@ -88,6 +90,48 @@ Admission and dispatch tuning for the Gateway cluster.
Every field is present on read. A deployment that has stored nothing reports the defaults above.
## Voice noise suppression configuration object
The instance rollout of client-side noise suppression. [Experiments](/http-api/experiments/) defines what a client resolves from it and the closed [backend registry](/http-api/experiments/#noise-suppression-backends) every backend field draws on.
### Structure
| Field | Type | Description |
| --- | --- | --- |
| enabled | boolean | Whether the rollout runs at all (default false) |
| config_version | integer | Revision counter, raised by Fluxer and never accepted from a request |
| default_backend | string | Backend given to a drawn account (default `standard`) |
| enabled_backends | array[string] | Backends a client MAY run, up to 7 entries (default every backend) |
| allow_user_override | boolean | Whether an account's own choice replaces the assigned backend (default true) |
| rollout_basis_points | integer | Share of accounts drawn, in basis points (0-10000, default 0) |
| rollout_salt | string | Salt of the sampling hash (1-64 characters, default `voice-ns-v1`) |
| included_user_ids | array[snowflake] | Accounts always drawn, up to 1000 entries (default empty) |
| excluded_user_ids | array[snowflake] | Accounts never drawn, up to 1000 entries (default empty) |
| guild_overrides | array[[guild override](/http-api/experiments/#noise-suppression-guild-override-object) object] | Per-guild replacements, up to 200 entries (default empty) |
| stereo_enabled | boolean | Whether a drawn client publishes a stereo microphone track (default false) |
| suppression_strength | integer | Suppression strength (0-100, default 80) |
Every field is present on read. A deployment that has stored nothing reports the defaults above.
`excluded_user_ids` is applied before `included_user_ids`, so an account in both is never drawn. A `default_backend` or `guild_overrides` entry naming a backend outside `enabled_backends` is dropped from what a client is served, and the stored value is kept as written.
How often a client revalidates this rollout is not set here. It is set once for every experiment in the [experiment delivery configuration](#experiment-delivery-configuration-object) below.
## Experiment delivery configuration object
How often a client polls [Get experiment assignments](/http-api/experiments/#get-experiment-assignments), and how widely those polls are spread. The setting is instance-wide and applies to every experiment at once, so adding an experiment adds no second cadence to tune.
### Structure
| Field | Type | Description |
| --- | --- | --- |
| poll_interval_seconds | integer | Seconds between client revalidations (60-86400, default 300) |
| poll_jitter_percent | integer | Spread applied to each revalidation (0-50, default 15) |
Every field is present on read. A deployment that has stored nothing reports the defaults above.
Both fields are served to every account, whether or not any experiment targets that account, and neither one is versioned by `config_version`. A client that has never reached the experiments route holds the same two values as built-in defaults, 300 seconds and 15 percent, so neither field reaches a client that cannot read the route.
## Registration configuration object
Registration policy in force, plus every issued registration URL and every account awaiting a decision.
@@ -472,6 +516,8 @@ The body has one optional object for each section. Fluxer leaves an absent secti
| --- | --- | --- |
| sso?<sup>1</sup> | object | Every [SSO configuration](#sso-configuration-object) field except `client_secret_set` and `redirect_uri`, plus `client_secret` |
| gateway_rollout? | object | Any subset of the [Gateway rollout configuration](#gateway-rollout-configuration-object) fields, each bound as documented there |
| voice_noise_suppression? | object | Any subset of the [noise suppression](#voice-noise-suppression-configuration-object) fields |
| experiment_delivery? | object | Any subset of the [experiment delivery](#experiment-delivery-configuration-object) fields |
| registration? | object | `mode` and `admin_registration_urls_enabled` |
| app_public?<sup>2</sup> | object | `branding`, `setup`, `legal`, and `registration` sub-objects, each merged field by field |
| integrations?<sup>3</sup> | object | `gif`, `youtube`, `captcha`, `email`, and `bluesky` sub-objects, the last of which also has the `keys` array |
@@ -482,6 +528,10 @@ The body has one optional object for each section. Fluxer leaves an absent secti
<sup>2</sup> `branding.product_name` is 1 to 80 characters, every branding and legal URL is at most 2048 characters and nullable, `branding.theme_color` is at most 64 characters and nullable, and `setup.configured` and `registration.collect_date_of_birth` are booleans. Every string is trimmed before it is stored
`voice_noise_suppression` takes every [voice noise suppression configuration](#voice-noise-suppression-configuration-object) field except `config_version`, each bound as documented there. Fluxer raises `config_version` by one on each request that supplies at least one of them. A section that is absent, or present with no field set, writes nothing and leaves `config_version` alone.
`experiment_delivery` takes both [experiment delivery configuration](#experiment-delivery-configuration-object) fields, each bound as documented there. It is a section of its own, so a write to it changes no `config_version` and changes no assignment, only the cadence on which clients ask for one.
<sup>3</sup> A secret such as `klipy_api_key`, `api_key`, `hcaptcha_secret_key`, `turnstile_secret_key`, or the SMTP `password` is written when supplied and left alone when absent. `integrations.bluesky.keys` is the only way to write the Bluesky signing keys counted as `bluesky.key_count`. It takes up to 8 entries of `kid` (1-255 characters) and nullable `private_key` (up to 10000 characters), and replaces the stored key set outright
:::note[Single sign-on URL validation is conditional]
@@ -516,7 +566,7 @@ Fluxer skips URL validation while the merged configuration leaves single sign-on
| 400 | [error response](/admin-api/#error-response) | A policy transition is refused, returned as `INSTANCE_POLICY_TRANSITION_NOT_ALLOWED` |
:::caution[Sections are applied one after another]
The order is `gateway_rollout`, `sso`, `registration`, `app_public` branding, legal, and registration fields, `integrations`, `media`, `policy`, and finally `app_public.setup`. A failure part way through leaves the earlier sections written.
The order is `gateway_rollout`, `voice_noise_suppression`, `experiment_delivery`, `sso`, `registration`, `app_public` branding, legal, and registration fields, `integrations`, `media`, `policy`, and finally `app_public.setup`. A failure part way through leaves the earlier sections written.
:::
### Side effects
fluxer_docs/src/content/docs/http-api/experiments.mdx
+150
View File
@@ -0,0 +1,150 @@
---
# SPDX-License-Identifier: AGPL-3.0-or-later
title: Experiments
description: The experiment assignments envelope, the revalidation and polling contract, and the noise suppression experiment.
---
import RouteHeader from '@/components/RouteHeader.astro';
An experiment is one instance-wide rollout the operator configures and Fluxer resolves against one account. The single route on this page resolves every experiment the server defines and returns them in one envelope, together with the polling cadence they share. `voice_noise_suppression` is the only experiment defined today, and [Voice](/voice/) defines the placement protocol its assignment applies to.
Every assignment is advice. A client that ignores one behaves as it does with the rollout off, and no route and no Gateway event reports what a client actually ran.
## Experiment assignments object
One resolution of every defined experiment against one account. Every field is present on every response.
### Structure
| Field | Type | Description |
| --- | --- | --- |
| poll_interval_seconds | integer | Seconds to wait before revalidating, from 60 through 86400 |
| poll_jitter_percent | integer | How far to spread the wait around the interval, from 0 through 50 |
| assignments | [assignment map](#assignment-map-object) object | One entry for each experiment the server defines |
The two polling fields sit on the envelope rather than on any one experiment, because the cadence is a property of the route and not of a rollout. They are the operator's [experiment delivery configuration](/admin-api/instance/#experiment-delivery-configuration-object) read back unchanged, so they hold the same values for every account and every experiment, whatever those experiments resolve to.
## Assignment map object
One entry per experiment. The envelope reports this object even when it is empty.
### Structure
| Field | Type | Description |
| --- | --- | --- |
| voice_noise_suppression? | [noise suppression assignment](#noise-suppression-assignment-object) object | The caller's noise suppression assignment |
Every key here is optional in the schema so that a peer running a different server version still validates the envelope. A client reading a server that has not defined an experiment it knows about, or that defines one it does not, drops the unknown key and treats the missing one as off rather than as an error.
This server version writes `voice_noise_suppression` on every response, including while the rollout is disabled. The disabled value is the first [resolution outcome](#resolution-outcomes) below, which reports `enabled` false and the stored `config_version`, so a client can tell an operator write from a no-op without a second request.
## Noise suppression backends
Noise suppression runs in the client, on the microphone track, before that track is published. Fluxer processes no audio for it. The registry is closed, and a backend outside it is not a value this API produces or accepts.
| Value | Description |
| --- | --- |
| none | No processing |
| standard | The browser or platform suppressor the client already has |
| gate | A noise gate keyed on input level |
| speex | The Speex preprocessor |
| rnnoise | The RNNoise recurrent model |
| gtcrn | The GTCRN model |
| deep_filter | The DeepFilterNet model |
A client MUST treat a backend absent from `enabled_backends` as unavailable, including one named by `backend` or by a [guild override](#noise-suppression-guild-override-object).
## Noise suppression assignment object
One resolution of the instance noise suppression rollout against one account. Every field is present whenever the key is written.
### Structure
| Field | Type | Description |
| --- | --- | --- |
| enabled | boolean | Whether the rollout is running on this instance |
| config_version | integer | The revision of the instance configuration this assignment was resolved from |
| user_targeted | boolean | Whether the caller is inside the rollout |
| backend | ?string | The [backend](#noise-suppression-backends) the caller applies, and null where the caller is not targeted |
| source | ?string | Which rule targeted the caller, one of `user_rule` or `canary`, and null where the caller is not targeted |
| guild_overrides | array[[guild override](#noise-suppression-guild-override-object) object] | Per-guild backend replacements that apply to the caller |
| enabled_backends | array[string] | The [backends](#noise-suppression-backends) the client MAY run |
| allow_user_override | boolean | Whether the account's own stored choice replaces `backend` |
| stereo_enabled | boolean | Whether the client publishes a stereo microphone track |
| suppression_strength | integer | Suppression strength from 0 through 100, which only a backend that reads it applies |
`config_version` counts operator writes, not assignment changes. It is raised by every [Update instance configuration](/admin-api/instance/#update-instance-configuration) request that sets at least one noise suppression field, so it can advance while the caller's assignment stays byte for byte the same.
### Resolution outcomes
Three outcomes set `user_targeted` to false, and they differ in what else they report.
1. The rollout is off. `enabled` is false, `enabled_backends` and `guild_overrides` are empty, and `allow_user_override` and `stereo_enabled` are false.
2. The operator has excluded the caller. `enabled` is true, and every other field is as in the first outcome.
3. The caller was not drawn. `enabled` is true, and `enabled_backends`, `guild_overrides`, `allow_user_override`, and `stereo_enabled` all hold their configured values.
A caller is drawn either by the operator's allowlist, which sets `source` to `user_rule`, or by the sampled share of the account population, which sets `source` to `canary`. A caller that is drawn while `backend` is absent from `enabled_backends` is reported as not drawn, with `user_targeted` false and both `backend` and `source` null.
A client branches on `user_targeted` rather than on `enabled_backends`, because the third outcome keeps the array populated. A `guild_overrides` entry applies in its guild whether or not the caller was drawn.
`config_version` reports the stored revision in all three outcomes, the rollout being off included.
## Noise suppression guild override object
One backend replacement scoped to one guild. A guild named here replaces `backend` while the caller is connected to a voice channel of that guild.
### Structure
| Field | Type | Description |
| --- | --- | --- |
| guild_id | snowflake | The guild the replacement applies in |
| backend | string | The [backend](#noise-suppression-backends) to run in that guild |
An override naming a backend that is absent from `enabled_backends` is dropped before the response is written, so every entry is runnable.
## Get experiment assignments
<RouteHeader method="GET" path="/v1/experiments" bot />
Resolves every defined experiment for the caller and returns an [experiment assignments](#experiment-assignments-object) object. The response is derived per account, so it is never shared between accounts.
### Response body
[Experiment assignments](#experiment-assignments-object) object.
### Response
| Status | Body | Condition |
| --- | --- | --- |
| 200 | response body | The envelope was resolved |
| 304 | empty | The request sent a matching `If-None-Match` |
### Request headers
| Field | Type | Description |
| --- | --- | --- |
| If-None-Match? | string | An `ETag` from an earlier response to this route |
A client sends the `ETag` it last received. Fluxer compares it against the tag of the envelope it just resolved and answers 304 with no body on a match. `*` matches any current tag. A weak comparison is used, so a `W/` prefix on either side does not defeat the match.
### Response headers
| Field | Type | Description |
| --- | --- | --- |
| ETag | string | A strong tag over the envelope body, sent on 200 and on 304 |
| Cache-Control | string | The literal value `private, no-cache` |
| Vary | string | The literal value `Authorization`, replaced by `Origin` where the [cross-origin policy](/http-api/#cross-origin-requests) echoed an allowed origin |
The tag is a hash of the body alone, so two accounts resolving to the same envelope receive the same tag. It changes whenever any field changes, the polling fields and `config_version` included.
`ETag` is listed in `Access-Control-Expose-Headers` and `If-None-Match` in `Access-Control-Allow-Headers`, so a cross-origin client reads the tag and revalidates with it.
### Polling
A client reads this route once per session and then again every `poll_interval_seconds`, offset by a random amount up to `poll_jitter_percent` of that interval in either direction. It sends the last `ETag` on every request after the first. A client MUST NOT poll faster than the value it was served.
Raising `poll_interval_seconds` sheds request volume. Raising `poll_jitter_percent` spreads a fleet that has synchronised on one interval. A client that has never reached this route holds the built-in defaults of 300 seconds and 15 percent, so neither value reaches a client that cannot read it.
### Rate limit
60 requests per 10 seconds for each authenticated user, on the `default` bucket.
fluxer_docs/src/content/docs/http-api/index.md
+2 -2
View File
@@ -137,7 +137,7 @@ An `X-Audit-Log-Reason` normalised to more than 512 characters is discarded, and
| Content-Type? | string | The media type of the representation, absent from a response with no body |
| Cache-Control?<sup>4</sup> | string | The literal value `no-cache` unless the operation sets its own directive |
| Access-Control-Allow-Origin?<sup>5</sup> | string | The request `Origin` when it is a configured application origin, and the literal `*` on routes that set their own wildcard |
| Access-Control-Expose-Headers?<sup>5</sup> | string | The literal value `X-Fluxer-Version` |
| Access-Control-Expose-Headers?<sup>5</sup> | string | The literal value `X-Fluxer-Version, ETag` |
| Vary?<sup>5</sup> | string | The literal value `Origin`, sent whenever the allowed origin was echoed |
| Retry-After?<sup>6</sup> | string | Whole seconds to wait, sent on a rate limit denial, a slowmode denial, a resource lock, and the in-flight ceiling 503 |
| X-RateLimit-Limit?<sup>7</sup> | string | Present on a route denial and on a successful bot or webhook request |
@@ -188,7 +188,7 @@ Five paths are readable from any origin. `/v1/webhooks/{webhook_id}/{token}` and
[Get instance discovery](/http-api/instance/#get-instance-discovery) on `/.well-known/fluxer`, [Get OpenAPI document](/http-api/instance/#get-openapi-document) on `/v1/openapi.json`, and [Get client geolocation](/http-api/instance/#get-client-geolocation) on `/v1/ip` set `Access-Control-Allow-Origin: *` in the operation itself. The wildcard stands for any origin outside the allow-list, and for an allowed origin the policy replaces it with that exact origin and sends `Vary: Origin`.
`Access-Control-Expose-Headers` is the single value `X-Fluxer-Version`. Every other Fluxer response header, the rate limit headers and `X-Request-ID` included, is hidden from cross-origin script.
`Access-Control-Expose-Headers` is the value `X-Fluxer-Version, ETag`. Every other Fluxer response header, the rate limit headers and `X-Request-ID` included, is hidden from cross-origin script. `Access-Control-Allow-Headers` is `Content-Type, Authorization, X-Requested-With, Accept-Language, X-Request-ID, If-None-Match`, so a cross-origin client revalidates an [ETag](/http-api/experiments/#get-experiment-assignments) it was served.
:::caution[Same-host mutations require a matching origin]
A production deployment rejects a non-`GET` request whose `Host` is `web.fluxer.app` or `web.canary.fluxer.app` unless its `Origin` is exactly `https://` followed by that same host, returning 403 `INVALID_API_ORIGIN`. A request sent to the API host is unaffected.