Weighs the soul of incoming HTTP requests to stop AI crawlers

v1.18.0: Varis zos Galvus

The big ticket feature in this release is [CEL expression matching support](https://anubis.techaro.lol/docs/admin/configuration/expressions). This allows you to tailor your approach for the individual services you are protecting.

These can be as simple as:

```yaml
- name: allow-api-requests
action: ALLOW
expression:
all:
- '"Accept" in headers'
- 'headers["Accept"] == "application/json"'
- 'path.startsWith("/api/")'
```

Or as complicated as:

```yaml
- name: allow-git-clients
action: ALLOW
expression:
all:
- >-
(
userAgent.startsWith("git/") ||
userAgent.contains("libgit") ||
userAgent.startsWith("go-git") ||
userAgent.startsWith("JGit/") ||
userAgent.startsWith("JGit-")
)
- '"Git-Protocol" in headers'
- headers["Git-Protocol"] == "version=2"
```

The docs have more information, but here's a tl;dr of the variables you have access to in expressions:

| Name | Type | Explanation | Example |
| :-------------- | :-------------------- | :---------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------- |
| `headers` | `map[string, string]` | The [headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers) of the request being processed. | `{"User-Agent": "Mozilla/5.0 Gecko/20100101 Firefox/137.0"}` |
| `host` | `string` | The [HTTP hostname](https://web.dev/articles/url-parts#host) the request is targeted to. | `anubis.techaro.lol` |
| `method` | `string` | The [HTTP method](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Methods) in the request being processed. | `GET`, `POST`, `DELETE`, etc. |
| `path` | `string` | The [path](https://web.dev/articles/url-parts#pathname) of the request being processed. | `/`, `/api/memes/create` |
| `query` | `map[string, string]` | The [query parameters](https://web.dev/articles/url-parts#query) of the request being processed. | `?foo=bar` -> `{"foo": "bar"}` |
| `remoteAddress` | `string` | The IP address of the client. | `1.1.1.1` |
| `userAgent` | `string` | The [`User-Agent`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/User-Agent) string in the request being processed. | `Mozilla/5.0 Gecko/20100101 Firefox/137.0` |

This will be made more elaborate in the future. Give me time. This is a [simple, lovable, and complete](https://longform.asmartbear.com/slc/) implementation of this feature so that administrators can get hacking ASAP.

Other changes:

- Use CSS variables to deduplicate styles
- Fixed native packages not containing the stdlib and botPolicies.yaml
- Change import syntax to allow multi-level imports
- Changed the startup logging to use JSON formatting as all the other logs do.
- Added the ability to do [expression matching with CEL](./admin/configuration/expressions.mdx)
- Add a warning for clients that don't store cookies
- Disable Open Graph passthrough by default ([#435](https://github.com/TecharoHQ/anubis/issues/435))
- Clarify the license of the mascot images ([#442](https://github.com/TecharoHQ/anubis/issues/442))
- Started Suppressing 'Context canceled' errors from http in the logs ([#446](https://github.com/TecharoHQ/anubis/issues/446))

Signed-off-by: Xe Iaso <me@xeiaso.net>

Xe Iaso 8c7640aa b1c276db

+57 -6
+1 -1
VERSION
··· 1 - 1.18.0-pre1 1 + 1.18.0
+51
docs/docs/CHANGELOG.md
··· 11 11 12 12 ## [Unreleased] 13 13 14 + ## v1.18.0: Varis zos Galvus 15 + 16 + The big ticket feature in this release is [CEL expression matching support](https://anubis.techaro.lol/docs/admin/configuration/expressions). This allows you to tailor your approach for the individual services you are protecting. 17 + 18 + These can be as simple as: 19 + 20 + ```yaml 21 + - name: allow-api-requests 22 + action: ALLOW 23 + expression: 24 + all: 25 + - '"Accept" in headers' 26 + - 'headers["Accept"] == "application/json"' 27 + - 'path.startsWith("/api/")' 28 + ``` 29 + 30 + Or as complicated as: 31 + 32 + ```yaml 33 + - name: allow-git-clients 34 + action: ALLOW 35 + expression: 36 + all: 37 + - >- 38 + ( 39 + userAgent.startsWith("git/") || 40 + userAgent.contains("libgit") || 41 + userAgent.startsWith("go-git") || 42 + userAgent.startsWith("JGit/") || 43 + userAgent.startsWith("JGit-") 44 + ) 45 + - '"Git-Protocol" in headers' 46 + - headers["Git-Protocol"] == "version=2" 47 + ``` 48 + 49 + The docs have more information, but here's a tl;dr of the variables you have access to in expressions: 50 + 51 + | Name | Type | Explanation | Example | 52 + | :-------------- | :-------------------- | :---------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------- | 53 + | `headers` | `map[string, string]` | The [headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers) of the request being processed. | `{"User-Agent": "Mozilla/5.0 Gecko/20100101 Firefox/137.0"}` | 54 + | `host` | `string` | The [HTTP hostname](https://web.dev/articles/url-parts#host) the request is targeted to. | `anubis.techaro.lol` | 55 + | `method` | `string` | The [HTTP method](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Methods) in the request being processed. | `GET`, `POST`, `DELETE`, etc. | 56 + | `path` | `string` | The [path](https://web.dev/articles/url-parts#pathname) of the request being processed. | `/`, `/api/memes/create` | 57 + | `query` | `map[string, string]` | The [query parameters](https://web.dev/articles/url-parts#query) of the request being processed. | `?foo=bar` -> `{"foo": "bar"}` | 58 + | `remoteAddress` | `string` | The IP address of the client. | `1.1.1.1` | 59 + | `userAgent` | `string` | The [`User-Agent`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/User-Agent) string in the request being processed. | `Mozilla/5.0 Gecko/20100101 Firefox/137.0` | 60 + 61 + This will be made more elaborate in the future. Give me time. This is a [simple, lovable, and complete](https://longform.asmartbear.com/slc/) implementation of this feature so that administrators can get hacking ASAP. 62 + 63 + Other changes: 64 + 14 65 - Use CSS variables to deduplicate styles 15 66 - Fixed native packages not containing the stdlib and botPolicies.yaml 16 67 - Change import syntax to allow multi-level imports
+3 -3
package-lock.json
··· 1 1 { 2 2 "name": "@techaro/anubis", 3 - "version": "1.18.0-pre1", 3 + "version": "1.18.0", 4 4 "lockfileVersion": 3, 5 5 "requires": true, 6 6 "packages": { 7 7 "": { 8 8 "name": "@techaro/anubis", 9 - "version": "1.18.0-pre1", 9 + "version": "1.18.0", 10 10 "license": "ISC", 11 11 "devDependencies": { 12 12 "cssnano": "^7.0.6", ··· 2739 2739 } 2740 2740 } 2741 2741 } 2742 - } 2742 + }
+2 -2
package.json
··· 1 1 { 2 2 "name": "@techaro/anubis", 3 - "version": "1.18.0-pre1", 3 + "version": "1.18.0", 4 4 "description": "", 5 5 "main": "index.js", 6 6 "scripts": { ··· 25 25 "postcss-import-url": "^7.2.0", 26 26 "postcss-url": "^10.1.3" 27 27 } 28 - } 28 + }