Constellation, Spacedust, Slingshot, UFOs: atproto crates and services for microcosm

Constellation: add order query to page links from oldest->newest #1

open opened by microcosm.blue

Right now the /links and /xrpc/blue.microcosm.links.getBacklinks always return backlinks in newest-to-oldest order.

Would be nice to have a query parameter making it go from oldest-to-newest instead!

feel free to ping fig or hop in the microcosm discord to discuss!

Opened another PR for this one as well.

Seems I chose the wrong target repo. Here's the correct one!

thank you for getting this going! some old thoughts from a previous private discussion about this:

thing 1: i'm wondering if the cursor should hold the initial length of the backlinks vec, pass it along to subsequent calls, and consider that the end, even if newer links have become available since beginning paging. (could even have a way to flag to clients that newer links are available, and perhaps a way to let clients start paging again starting from the end of a cursor?)

this would give something closer to a consistent view of a point in time, which is nice. of course it's not actually fully consistent: a link could be deleted while we're paging, so the total set that a client sees is "all live when they started, plus some undefined combination of links deleted while paging". but it's probably close enough.

(we could do an actual db snapshot and identify it with the cursor, but that's adding a lot of new stuff to handle, so, not for now at least.)

thing 2 (related): the way record updates are handled right now is by looking up the previous version, generating deletes for all links in it, and then generating creates for all links in the new version. this winds up being correct-ish, but for updates that don't change some links, it churns those links and pushes the backlinks to the end of the vec.

in the context of this PR, that means you can actually get duplicate backlinks from paging reversed, since you might see a link from a record that then gets updated causing it to reappear later.

that might be acceptable, but would need some documentation to note the behaviour. (or, if we stop iterating at the end of the original vec size, it's a non-issue because the duplicates would be after the end)

(separately: we should fix record update handling so it actually diffs the set of links, to avoid churning unchanged links)

...the current cursor handling is a little weird right now with alive/gone/total but i think it should already carry enough information to figure out stopping at the initial vec size (excluding backlinks added since paging began)

sign up or login to add to the discussion
Labels
constellation
good-first-issue
Participants 3
AT URI
at://did:plc:lulmyldiq4sb2ikags5sfb25/sh.tangled.repo.issue/3m2cb5emiys22