mirror of
https://github.com/fluxerapp/fluxer
synced 2026-10-19 05:55:34 +09:00
fix(api): dedupe and budget ipinfo lookups across api pods (#2584)
This commit is contained in:
1 parent
a2a68847fd
commit
824b5c86c9
24 files changed
+2125
-66
No files matched your search
@@ -1010,6 +1010,12 @@ Defaults to the inverse of `FLUXER_SELF_HOSTED`. External blocklist feeds. Off b
|
||||
|
||||
A second family, unrelated to the rules above, tunes the IP auto-banner: `FLUXER_ABUSE_WINDOW_MS`, four `FLUXER_ABUSE_THRESHOLD_` names, four `FLUXER_ABUSE_TOKEN_DIVERSITY_` names, `FLUXER_ABUSE_BAN_TTL_SEC`, `FLUXER_ABUSE_BATCH_FLUSH_MS`, `FLUXER_ABUSE_MAX_BATCH_TICKS`, `FLUXER_ABUSE_MAX_NEW_TOKENS_PER_TICK`, `FLUXER_ABUSE_MAX_TRACKED_IPS`, `FLUXER_ABUSE_MAX_TOKEN_HASHES_PER_IP`, `FLUXER_ABUSE_MIN_SCORE_FOR_LOOKUP`, `FLUXER_ABUSE_MIN_TOKENS_FOR_LOOKUP`, and `FLUXER_ABUSE_REQUIRED_SCORE_WINDOWS_FOR_AUTO_BAN`. All are read directly from the environment, none are in `.env.example` or the Compose file, and a non-finite value or one at or below zero falls back to the default.
|
||||
|
||||
Five more names in that family bound the IP classification the auto-banner buys from ipinfo. Before them, every API replica looked the same attacking IP up at the same moment, so one IP cost one lookup per replica. `FLUXER_ABUSE_IP_CLASS_CLAIM_ENABLED` defaults to `1` and gates the shared claim that now lets one replica do the lookup for all of them. `FLUXER_ABUSE_IP_CLASS_CLAIM_TTL_SEC` defaults to `15` and sets how long a replica holds that claim. `FLUXER_ABUSE_IP_CLASS_PENDING_TTL_MS` defaults to `20000` and sets how long a replica that lost the claim waits before it tries again. `FLUXER_ABUSE_IP_CLASS_NEGATIVE_TTL_MS` defaults to `300000` and sets how long a failed classification is remembered. `FLUXER_ABUSE_IP_CLASS_HINT_TTL_MS` defaults to `600000` and sets how long a class sent by another replica stays usable. The claim key is `abuse:ipclass:claim:` plus the ban key, and replicas send classes to each other on the `abuse_tracker:ipclass` key-value channel. The four numeric names read through the same helper as the names above, so a non-finite value or one at or below zero falls back to the default. `FLUXER_ABUSE_IP_CLASS_CLAIM_ENABLED` is read as a string instead, and only `0` turns the claim off.
|
||||
|
||||
A `FLUXER_IPINFO_BUDGET_` family caps what the instance spends at ipinfo. `FLUXER_IPINFO_BUDGET_ENABLED` defaults to `1`, and `0` turns off all shedding. `FLUXER_IPINFO_BUDGET_MONTHLY_MAX` defaults to `140000` and is the ceiling for one UTC calendar month. Lookups run at three priorities. Admin IP bans and scheduled deletion checks are critical and reach the full ceiling. Registration risk is standard and stops at `FLUXER_IPINFO_BUDGET_STANDARD_MONTHLY_PCT` percent of it, default `90`. The IP auto-banner is background and stops at `FLUXER_IPINFO_BUDGET_BACKGROUND_MONTHLY_PCT` percent, default `60`. The lower ceilings mean background lookups stop first and critical lookups stop last. Each priority also has a token bucket for bursts, sized by `FLUXER_IPINFO_BUDGET_CRITICAL_BURST` (`60`), `FLUXER_IPINFO_BUDGET_STANDARD_BURST` (`240`) and `FLUXER_IPINFO_BUDGET_BACKGROUND_BURST` (`120`), refilled once a minute by `FLUXER_IPINFO_BUDGET_CRITICAL_REFILL_PER_MIN` (`60`), `FLUXER_IPINFO_BUDGET_STANDARD_REFILL_PER_MIN` (`120`) and `FLUXER_IPINFO_BUDGET_BACKGROUND_REFILL_PER_MIN` (`30`). The counters live in the key-value store under `ipinfo:budget:burst:` and `ipinfo:budget:month:`. A shed lookup returns an unavailable result instead of an error, and any key-value failure admits the lookup at every priority, so a cache outage never stops an admin ban or the auto-banner.
|
||||
|
||||
Two names let the local MaxMind databases answer the registration risk lookup instead of ipinfo. `FLUXER_RISK_IPINFO_PRESCREEN_ENABLED` is off by default and only `1` or `true`, in any case, turns it on. `FLUXER_RISK_IPINFO_PRESCREEN_ALLOW_ASNS` is a comma-separated list of ASN numbers, empty by default, and non-numeric entries are dropped. The list is empty out of the box, so the pre-screen does nothing until an operator fills it in. An IP skips ipinfo only when the local city database returns a country, the local ASN database returns a number, that number is in the list, and the ASN organization name is not a commercial privacy provider, an education network or a cellular network. Everything else still goes to ipinfo.
|
||||
|
||||
## Limits
|
||||
|
||||
No environment variable changes an instance limit. Fluxer keeps the limits in the key-value store under `limit_config:self_hosted` or `limit_config:saas`, seeded from `FLUXER_SELF_HOSTED`, and an operator edits them through the admin dashboard's Limit Config page or the [Admin API](/admin-api/). The published values are the [limit configuration object](/http-api/instance/#limit-configuration-object).
|
||||
@@ -1180,7 +1186,7 @@ Default `0`. The Bunny pull zone. Integer.
|
||||
|
||||
#### `FLUXER_GEOIP_DB_PATH`
|
||||
|
||||
Default empty. The GeoIP database. A filesystem path, or an `s3://bucket/key` URL whose `download_path` query parameter is mandatory and must be absolute. `app-proxy` also accepts `MAXMIND_DB_PATH`.
|
||||
Default empty. The GeoIP database. A filesystem path, or an `s3://bucket/key` URL whose `download_path` query parameter is mandatory and must be absolute. `app-proxy` also accepts `MAXMIND_DB_PATH`. The API also reads an ASN database. On a filesystem path it looks for `GeoLite2-ASN.mmdb` in the same directory. On an `s3://` URL it needs an `asn_key` query parameter, and takes an optional absolute `asn_download_path` next to it. A missing ASN database is not an error, and the API falls back to ipinfo for ASN data.
|
||||
|
||||
## Instance identity and branding
|
||||
|
||||
@@ -1648,7 +1654,7 @@ Branding is otherwise admin-dashboard only.
|
||||
|
||||
Supplies the initial setup state on a self-hosted instance, until the first write of the stored `app_public_config` row takes over.
|
||||
|
||||
#### Every `FLUXER_ABUSE_` auto-banner name and every Media Proxy performance knob
|
||||
#### Every `FLUXER_ABUSE_` auto-banner name, every `FLUXER_IPINFO_` budget name, and every Media Proxy performance knob
|
||||
|
||||
No example coverage at all.
|
||||
|
||||
|
||||
Reference in new issue
Block a user