v1.23.0: Lyse Hext

[Xe]

Sorry this took so long, work has been wiping me out. If you know of any companies that are hiring for someone of my skillset, please let me know.

• Add default tencent cloud DENY rule.
• Added (data)/meta/default-config.yaml for importing the entire default configuration at once.
• Add -custom-real-ip-header flag to get the original request IP from a different header than x-real-ip.
• Add contentLength variable to bot expressions.
• Add COOKIE_SAME_SITE_MODE to force anubis cookies SameSite value, and downgrade automatically from None to Lax if cookie is insecure.
• Fix lock convoy problem in decaymap (#1103).
• Fix lock convoy problem in bbolt by implementing the actor pattern (#1103).
• Remove bbolt actorify implementation due to causing production issues.
• Document missing environment variables in installation guide: SLOG_LEVEL, COOKIE_PREFIX, FORCED_LANGUAGE, and TARGET_DISABLE_KEEPALIVE (#1086).
• Add validation warning when persistent storage is used without setting signing keys.
• Fixed robots2policy to properly group consecutive user agents into any: instead of only processing the last one (#925).
• Make the fast algorithm prefer purejs when running in an insecure context.
• Add the s3api storage backend to allow Anubis to use S3 API compatible object storage as its storage backend.
• Fix a "stutter" in the cookie name prefix so the auth cookie is named techaro.lol-anubis-auth instead of techaro.lol-anubis-auth-auth.
• Make cmd/containerbuild support commas for separating elements of the --docker-tags argument as well as newlines.
• Add the DIFFICULTY_IN_JWT option, which allows one to add the difficulty field in the JWT claims which indicates the difficulty of the token (#1063).
• Ported the client-side JS to TypeScript to avoid egregious errors in the future.
• Fixes concurrency problems with very old browsers (#1082).
• Randomly use the Refresh header instead of the meta refresh tag in the metarefresh challenge.
• Update OpenRC service to truncate the runtime directory before starting Anubis.
• Make the git client profile more strictly match how the git client behaves.
• Make the default configuration reward users using normal browsers.
• Allow multiple consecutive slashes in a row in application paths (#754).
• Add option to set targetSNI to special keyword 'auto' to indicate that it should be automatically set to the request Host name (424).
• The Preact challenge has been removed from the default configuration. It will be deprecated in the future.
• An open redirect when in subrequest mode has been fixed.

Potentially breaking changes

Multiple checks at once has and-like semantics instead of or-like semantics

Anubis lets you stack multiple checks at once with blocks like this:

name: allow-prometheus
action: ALLOW
user_agent_regex: ^prometheus-probe$
remote_addresses:
  - 192.168.2.0/24

Previously, this only returned ALLOW if any one of the conditions matched. This behaviour has changed to only return ALLOW if all of the conditions match. I expect this to have some issues with user configs, however this fix is grave enough that it's worth the risk of breaking configs. If this bites you, please let me know so we can make an escape hatch.

Better error messages

In order to make it easier for legitimate clients to debug issues with their browser configuration and Anubis, Anubis will emit internal error detail in base 64 so that administrators can chase down issues. Future versions of this may also include a variant that encrypts the error detail messages.

Bug Fixes

Sometimes the enhanced temporal assurance in #1038 and #1068 could backfire because Chromium and its ilk randomize the amount of time they wait in order to avoid a timing side channel attack. This has been fixed by both increasing the amount of time a client has to wait for the metarefresh and preact challenges as well as making the server side logic more permissive.

What's Changed

• docs(installation): add SLOG_LEVEL environment variable to configuration by @JasonLovesDoggo in docs(installation): add SLOG_LEVEL environment variable to configuration #1086
• docs: document some missing env vars by @JasonLovesDoggo in docs: document some missing env vars #1087
• build(deps): bump the github-actions group across 1 directory with 8 updates by @dependabot[bot] in build(deps): bump the github-actions group across 1 directory with 8 updates #1071
• fix(robots2policy): handle multiple user agents under one block by @JasonLovesDoggo in fix(robots2policy): handle multiple user agents under one block #925
• feat(lib/store): add s3api storage backend by @Xe in feat(lib/store): add s3api storage backend #1089
• Xe/demote temporal assurance by @Xe in Xe/demote temporal assurance #1090
• feat: Warn on missing signing keys when persisting challenges by @JasonLovesDoggo in feat: Warn on missing signing keys when persisting challenges #1088
• docs: add reminder for verified signatures in PR template by @JasonLovesDoggo in docs: add reminder for verified signatures in PR template #1092
• build(deps): bump the github-actions group with 4 updates by @dependabot[bot] in build(deps): bump the github-actions group with 4 updates #1093
• security: npm audit fix for GHSA-hfm8-9jrf-7g9w et. al by @Xe in security: npm audit fix for GHSA-hfm8-9jrf-7g9w et. al #1098
• fix(cmd/containerbuild): support commas in --docker-tags by @Xe in fix(cmd/containerbuild): support commas in --docker-tags #1099
• feat(lib): Add option for adding difficulty field to JWT claims by @Earl0fPudding in feat(lib): Add option for adding difficulty field to JWT claims #1063
• chore: port client-side JS to TypeScript by @Xe in chore: port client-side JS to TypeScript #1100
• fix(decaymap): fix lock convoy by @Xe in fix(decaymap): fix lock convoy #1106
• feat(store/bbolt): implement actor pattern by @Xe in feat(store/bbolt): implement actor pattern #1107
• feat: allow to set cookie sameSite mode and fallback to Lax mode if cookie is not secure by @vaab in feat: allow to set cookie sameSite mode and fallback to Lax mode if cookie is not secure #1105
• docs: add link to preact in challenge list by @agoujot in docs: add link to preact in challenge list #1111
• ci: add aarch64 for ssh CI by @Xe in ci: add aarch64 for ssh CI #1112
• ci(ssh): don't print uname -av output by @Xe in ci(ssh): don't print uname -av output #1114
• feat(expressions): add contentLength to bot expressions by @Xe in feat(expressions): add contentLength to bot expressions #1120
• fix(run/openrc): truncate runtime directory before starting Anubis by @CyberTailor in fix(run/openrc): truncate runtime directory before starting Anubis #1122
• build(deps): bump the npm group with 2 updates by @dependabot[bot] in build(deps): bump the npm group with 2 updates #1117
• build(deps): bump the github-actions group with 3 updates by @dependabot[bot] in build(deps): bump the github-actions group with 3 updates #1118
• Update nl.json removing literal translated cookie 'koekje' with 'cookie' by @jieter in Update nl.json removing literal translated cookie 'koekje' with 'cookie' #1126
• convert issue templates into issue forms by @NetSysFire in convert issue templates into issue forms #1115
• build(deps): bump github.com/docker/docker from 28.3.2+incompatible to 28.3.3+incompatible in /test by @dependabot[bot] in build(deps): bump github.com/docker/docker from 28.3.2+incompatible to 28.3.3+incompatible in /test #1130
• feat(metarefresh): randomly use the Refresh header by @Xe in feat(metarefresh): randomly use the Refresh header #1133
• Add Door43 link to known instances documentation by @richmahn in Add Door43 link to known instances documentation #1136
• fix: mend auth cookie name stutter by @Xe in fix: mend auth cookie name stutter #1139
• Update Nynorsk translation by @turtlegarden in Update Nynorsk translation #1143
• feat: support reading real client IP from a custom header by @avioletheart in feat: support reading real client IP from a custom header #1138
• enable auto setting of SNI based on host header by @jmcclelland in enable auto setting of SNI based on host header #1129
• fix(lib): enable multiple consecutive slash support by @Xe in fix(lib): enable multiple consecutive slash support #1155
• build(deps-dev): bump esbuild from 0.25.9 to 0.25.10 in the npm group by @dependabot[bot] in build(deps-dev): bump esbuild from 0.25.9 to 0.25.10 in the npm group #1147
• build(deps): bump github.com/ulikunitz/xz from 0.5.12 to 0.5.14 by @dependabot[bot] in build(deps): bump github.com/ulikunitz/xz from 0.5.12 to 0.5.14 #1132
• build(deps): bump github.com/docker/docker from 28.3.2+incompatible to 28.3.3+incompatible by @dependabot[bot] in build(deps): bump github.com/docker/docker from 28.3.2+incompatible to 28.3.3+incompatible #1131
• fix(lib): serve CSS properly by @Xe in fix(lib): serve CSS properly #1158
• fix(default-config): make the default config far less paranoid by @Xe in fix(default-config): make the default config far less paranoid #1179
• fix(default-config): remove preact challenge by @Xe in fix(default-config): remove preact challenge #1184
• feat: default config macro by @Xe in feat: default config macro #1186
• fix(lib): de-flake package lib tests by @Xe in fix(lib): de-flake package lib tests #1187
• Updated REDIRECT_DOMAINS documentation by @zc-devs in Updated REDIRECT_DOMAINS documentation #1171
• fix(default-config): sometimes browsers don't send Upgrade-Insecure-Requests by @Xe in fix(default-config): sometimes browsers don't send Upgrade-Insecure-Requests #1189
• fix(algorithms/fast): fix fast challenge on insecure contexts by @Xe in fix(algorithms/fast): fix fast challenge on insecure contexts #1198
• Xe/show error state by @Xe in Xe/show error state #1203
• locale: Update Nynorsk translation by @turtlegarden in locale: Update Nynorsk translation #1204
• docs: point get started button to the per-environment setup docs by @Xe in docs: point get started button to the per-environment setup docs #1213
• fix(store/bbolt): remove actorify by @Xe in fix(store/bbolt): remove actorify #1215
• feat(default-config): block tencent cloud by default by @Xe in feat(default-config): block tencent cloud by default #1216
• link to docs site from readme by @pushcx in link to docs site from readme #1214
• fix!(policy/checker): make List and-like by @Xe in fix!(policy/checker): make List and-like #1217
• chore: remove copilot instructions by @Xe in chore: remove copilot instructions #1218
• build(deps): bump the github-actions group across 1 directory with 6 updates by @dependabot[bot] in build(deps): bump the github-actions group across 1 directory with 6 updates #1221
• fix(lib): close open redirect when in subrequest mode by @Xe in fix(lib): close open redirect when in subrequest mode #1222

New Contributors

• @vaab made their first contribution in feat: allow to set cookie sameSite mode and fallback to Lax mode if cookie is not secure #1105
• @agoujot made their first contribution in docs: add link to preact in challenge list #1111
• @NetSysFire made their first contribution in convert issue templates into issue forms #1115
• @richmahn made their first contribution in Add Door43 link to known instances documentation #1136
• @avioletheart made their first contribution in feat: support reading real client IP from a custom header #1138
• @jmcclelland made their first contribution in enable auto setting of SNI based on host header #1129
• @zc-devs made their first contribution in Updated REDIRECT_DOMAINS documentation #1171
• @pushcx made their first contribution in link to docs site from readme #1214

Full Changelog: v1.22.0...v1.23.0

---

This discussion was created from the release v1.23.0: Lyse Hext.

[alex3305]

Thanks for the great release ❤️ . I have one regression.

Pulling (Docker) containers from my private Forgejo instance yields an 'Unauthorized' with the new version with Traefik and Anubis configured as middleware😒 . Rolling back to 1.22 fixes the issue instantly. Docker login seems to work fine either way.

Tests are done on a private network, so I'm at a loss where the issue occurs. Anubis only gives me a "New challenge issued" log.

[rebelliouswhiz]

Thanks for the release! Lots of work indeed!

After the upgrade, I noticed the cute little fox(?) no longer appears. Was it the browser solving it too quickly, or was it no longer present by design?

When I curl with the user-agent, I can correctly receive the HTML of the inspecting page.

Is there any way that I can see the fox again? I kinda miss him/her/them (not sure).

[Xe]

You can see the jackal again by making a custom policy file that removes some of the rules I added to better detect common browsers and change their weight so that you don't see the Anubis challenge as much:

anubis/data/botPolicies.yaml

Lines 99 to 141 in 83a83e9

	# Assert behaviour that only genuine browsers display. This ensures that Chrome
	# or Firefox versions
	- name: realistic-browser-catchall
	expression:
	all:
	- '"User-Agent" in headers'
	- '( userAgent.contains("Firefox") ) || ( userAgent.contains("Chrome") ) || ( userAgent.contains("Safari") )'
	- '"Accept" in headers'
	- '"Sec-Fetch-Dest" in headers'
	- '"Sec-Fetch-Mode" in headers'
	- '"Sec-Fetch-Site" in headers'
	- '"Accept-Encoding" in headers'
	- '( headers["Accept-Encoding"].contains("zstd") || headers["Accept-Encoding"].contains("br") )'
	- '"Accept-Language" in headers'
	action: WEIGH
	weight:
	adjust: -10
	# The Upgrade-Insecure-Requests header is typically sent by browsers, but not always
	- name: upgrade-insecure-requests
	expression: '"Upgrade-Insecure-Requests" in headers'
	action: WEIGH
	weight:
	adjust: -2
	# Chrome should behave like Chrome
	- name: chrome-is-proper
	expression:
	all:
	- userAgent.contains("Chrome")
	- '"Sec-Ch-Ua" in headers'
	- 'headers["Sec-Ch-Ua"].contains("Chromium")'
	- '"Sec-Ch-Ua-Mobile" in headers'
	- '"Sec-Ch-Ua-Platform" in headers'
	action: WEIGH
	weight:
	adjust: -5
	- name: should-have-accept
	expression: '!("Accept" in headers)'
	action: WEIGH
	weight:
	adjust: 5

[rebelliouswhiz]

Yeah I saw that in the change log, and was wondering why curl even setting the same user-agent as my browser does not bypass it.

Looking at the headers in the policy, I see it's a good balance of weighting between legit traffic and the cradler traffic.

Thanks Xe, great work! Kinda miss the (essentially a) jackal though.