microcosm.blue
commits
"forward links" now align a little on the "other link" terminology used in the client-facing api
(kind of regret "other" as a term but oh well!)
the derives came in from base64 in tangled#7, whic was removed.
the cursor is a composite, but we might have other composite cursors that are different -- renaming this one to reflect its specific purpose.
see https://tangled.org/microcosm.blue/microcosm-rs/pulls/9/round/3#comment-1707
Renaming from `getDistinct` to `getBacklinkDids` to better reflect the
output type and the close relationship to the `getBacklinks` method.
Simple conversion of the existing endpoint from REST to XRPC.
Consequtively marked the pre-existing REST endpoint as deprecated.
# Conflicts:
# constellation/templates/try-it-macros.html.j2
# Conflicts:
# constellation/src/server/mod.rs
# constellation/templates/hello.html.j2
Bump time from 0.3.41 to 0.3.47
Bumps [time](https://github.com/time-rs/time) from 0.3.41 to 0.3.47.
- [Release notes](https://github.com/time-rs/time/releases)
- [Changelog](https://github.com/time-rs/time/blob/main/CHANGELOG.md)
- [Commits](https://github.com/time-rs/time/compare/v0.3.41...v0.3.47)
---
updated-dependencies:
- dependency-name: time
dependency-version: 0.3.47
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Most importantly we transformed the composite cursor into an `Option<(u64, u64)>`. This removes a couple of invalid cursor combinations automatically: if the composite cursor is provided it can be reasonable expected that both parts are present and valid (especially considering that we're the ones building the cursor in the first place). When this is not the case we bail early.
In addition, we removed the redundant base64 dependency (the `OpaqueApiCursor` handles the url-safe hex-encoded representation, and addressed some minor instances where we could have written some parts in a more idiomatic way.
No tests were changed and all tests still pass.
Same issue as the get_many_to_many cursor fix in the previous commit,
but simpler: the counts endpoint groups by subject so each subject
appears exactly once, meaning a subject-only cursor is sufficient.
The only problem was that when items.len() == limit, the code couldn't
distinguish "exactly limit items exist" from "more items exist but we
stopped at limit", causing a false cursor on the final page.
Fix: accumulate limit+1 items, only emit a cursor when more than limit
exist, then truncate. Applied to both MemStorage (take N+1 in the
iterator chain) and RocksDB (allow one extra group in the BTreeMap,
pop it before building results).
The previous cursor was a plain subject string, which broke pagination
when multiple records shared the same secondary subject. Depending on
the comparison used in skip_while, this caused either duplicates (items
returned on both pages) or skipped items (items returned on neither
page) - these edge cases are now covered by the added collection of new
test.
The root cause is that a subject-only cursor cannot uniquely identify
a position in the result set when multiple (RecordId, subject) pairs
share the same subject value. Five issues were fixed:
1. Composite sort: items are now sorted by (subject, RecordId) instead
of subject alone, establishing a deterministic total order.
2. Composite cursor: the cursor is now a base64-encoded "did|rkey|subject"
string. Shared encode/decode helpers in storage/mod.rs ensure both
MemStorage and RocksDB produce and consume the same format. DID and
rkey are placed first in the encoding so that splitn(3, '|') correctly
handles subjects that may contain the delimiter.
3. Correct skip direction: skip_while now uses <= (skip items at or
before the cursor), fixing the comparison that previously caused
items to be re-included or over-skipped.
4. Post-filter cursor computation: the next cursor is now derived from
the last item after skip+take, not from the unfiltered list.
5. Fetch N+1: take limit+1 items, only emit a cursor when more than
limit items exist, then truncate. This avoids emitting a false cursor
that leads to an empty final page when items exactly equals limit.
As
[discussed](https://tangled.org/microcosm.blue/microcosm-rs/pulls/7#comment-1528)
we replaced the existing nested structure of multiple record ids mapped
to one subject with a flat vector of record_id/subject tuples.
Compared to the proposed double-cursor we would need to impelement for
the latter, the former allows for simpler single-cursor and possible
confusion on the user side on how to use these.
Replace (String, Vec<RecordId>) tuples with a proper
RecordsBySubject struct throughout the many-to-many query
implementation as Lexicons per ATProto spec do not offer a tuple type.
Add lexicon definition for the getManyToMany XRPC endpoint.
# Conflicts:
# constellation/src/server/mod.rs
Bump bytes from 1.10.1 to 1.11.1
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.10.1 to 1.11.1.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.10.1...v1.11.1)
---
updated-dependencies:
- dependency-name: bytes
dependency-version: 1.11.1
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
this should probably be fixed in atrium's handle type but oh well
- Add --cache-memory-mb and --cache-disk-gb flags
bind options and general app configs
not likely to get reused as library code // the deprecation is more a network-user-facing thing
make clap a workspace dependency (finally, todo: more of them)
add env configs for args
add ipv6 support to slingshot's acme config
Bump rsa from 0.9.8 to 0.9.10
This belongs to the xrpc_many2many branch.
tangled.sh links give "Unable to connect error"
Now matches the existing XRPC getBacklinks endpoint naming scheme.
This should fix the white-space issue for the` try-it-macros` template as
well.
Simple conversion of the existing endpoint from REST to XRPC.
Consequtively marked the pre-existing REST endpoint as deprecated.
In addition we now ignore rocks.test
# Conflicts:
# constellation/src/storage/mod.rs
# lexicons/blue.microcosm/links/getManyToMany.json
merges https://tangled.org/microcosm.blue/microcosm-rs/pulls/6
slice the links list via Iterator::skip and ::take. i changed some of the bounds setup too. not sure if it's clearer or if i just have a better understanding of it at this specific moment.
throw an `empty` static function on some paging things for less verbose early return and tests.
make get_links_paged test a bit clearer.
https://tangled.org/microcosm.blue/microcosm-rs/pulls/6#comment-1444
many-to-many order is deterministic (could be reversed) but the order itself is arbitrary, so reversing it gives a different arbitrary order, but not a meaningfully different order.
the rocks impl also embeds assumptions about the paging rules into its join loop which might not hold
the regular /links endpoint won't get the reverse param because it's deprecated, but reversing backlinks is kept for the non-deprectated xrpc backlinks endpoint!
Bumps [rsa](https://github.com/RustCrypto/RSA) from 0.9.8 to 0.9.10.
- [Changelog](https://github.com/RustCrypto/RSA/blob/v0.9.10/CHANGELOG.md)
- [Commits](https://github.com/RustCrypto/RSA/compare/v0.9.8...v0.9.10)
---
updated-dependencies:
- dependency-name: rsa
dependency-version: 0.9.10
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
As outlined in the description of issue #4 we increase:
- the default response limit from 16 to __100__
- the maximum from 100 to __1000_
Adds a new --collect-metrics (presence) flag to make metrics collection
opt-in. The new default is to __not__ collect any metrics, i.e. neither
spawning the monitoring thread nor starting the metrics server server
that exposes these.
Bumps [time](https://github.com/time-rs/time) from 0.3.41 to 0.3.47.
- [Release notes](https://github.com/time-rs/time/releases)
- [Changelog](https://github.com/time-rs/time/blob/main/CHANGELOG.md)
- [Commits](https://github.com/time-rs/time/compare/v0.3.41...v0.3.47)
---
updated-dependencies:
- dependency-name: time
dependency-version: 0.3.47
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Most importantly we transformed the composite cursor into an `Option<(u64, u64)>`. This removes a couple of invalid cursor combinations automatically: if the composite cursor is provided it can be reasonable expected that both parts are present and valid (especially considering that we're the ones building the cursor in the first place). When this is not the case we bail early.
In addition, we removed the redundant base64 dependency (the `OpaqueApiCursor` handles the url-safe hex-encoded representation, and addressed some minor instances where we could have written some parts in a more idiomatic way.
No tests were changed and all tests still pass.
Same issue as the get_many_to_many cursor fix in the previous commit,
but simpler: the counts endpoint groups by subject so each subject
appears exactly once, meaning a subject-only cursor is sufficient.
The only problem was that when items.len() == limit, the code couldn't
distinguish "exactly limit items exist" from "more items exist but we
stopped at limit", causing a false cursor on the final page.
Fix: accumulate limit+1 items, only emit a cursor when more than limit
exist, then truncate. Applied to both MemStorage (take N+1 in the
iterator chain) and RocksDB (allow one extra group in the BTreeMap,
pop it before building results).
The previous cursor was a plain subject string, which broke pagination
when multiple records shared the same secondary subject. Depending on
the comparison used in skip_while, this caused either duplicates (items
returned on both pages) or skipped items (items returned on neither
page) - these edge cases are now covered by the added collection of new
test.
The root cause is that a subject-only cursor cannot uniquely identify
a position in the result set when multiple (RecordId, subject) pairs
share the same subject value. Five issues were fixed:
1. Composite sort: items are now sorted by (subject, RecordId) instead
of subject alone, establishing a deterministic total order.
2. Composite cursor: the cursor is now a base64-encoded "did|rkey|subject"
string. Shared encode/decode helpers in storage/mod.rs ensure both
MemStorage and RocksDB produce and consume the same format. DID and
rkey are placed first in the encoding so that splitn(3, '|') correctly
handles subjects that may contain the delimiter.
3. Correct skip direction: skip_while now uses <= (skip items at or
before the cursor), fixing the comparison that previously caused
items to be re-included or over-skipped.
4. Post-filter cursor computation: the next cursor is now derived from
the last item after skip+take, not from the unfiltered list.
5. Fetch N+1: take limit+1 items, only emit a cursor when more than
limit items exist, then truncate. This avoids emitting a false cursor
that leads to an empty final page when items exactly equals limit.
As
[discussed](https://tangled.org/microcosm.blue/microcosm-rs/pulls/7#comment-1528)
we replaced the existing nested structure of multiple record ids mapped
to one subject with a flat vector of record_id/subject tuples.
Compared to the proposed double-cursor we would need to impelement for
the latter, the former allows for simpler single-cursor and possible
confusion on the user side on how to use these.
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.10.1 to 1.11.1.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.10.1...v1.11.1)
---
updated-dependencies:
- dependency-name: bytes
dependency-version: 1.11.1
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
slice the links list via Iterator::skip and ::take. i changed some of the bounds setup too. not sure if it's clearer or if i just have a better understanding of it at this specific moment.
throw an `empty` static function on some paging things for less verbose early return and tests.
make get_links_paged test a bit clearer.
https://tangled.org/microcosm.blue/microcosm-rs/pulls/6#comment-1444
many-to-many order is deterministic (could be reversed) but the order itself is arbitrary, so reversing it gives a different arbitrary order, but not a meaningfully different order.
the rocks impl also embeds assumptions about the paging rules into its join loop which might not hold
the regular /links endpoint won't get the reverse param because it's deprecated, but reversing backlinks is kept for the non-deprectated xrpc backlinks endpoint!
Bumps [rsa](https://github.com/RustCrypto/RSA) from 0.9.8 to 0.9.10.
- [Changelog](https://github.com/RustCrypto/RSA/blob/v0.9.10/CHANGELOG.md)
- [Commits](https://github.com/RustCrypto/RSA/compare/v0.9.8...v0.9.10)
---
updated-dependencies:
- dependency-name: rsa
dependency-version: 0.9.10
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>