+46

docs/frontpage-data-layers.md

··· 1 1 + # Frontpage Data Layers 2 2 + 3 3 + Frontpage uses a layered data architecture to manage and interact with data from various sources, including the AT protocol and its own database. This document outlines the key data layers used in the Frontpage application. 4 4 + 5 5 + ## Why? 6 6 + 7 7 + AT has two sources of truth for data: the PDS of the actor, and the Frontpage app¹ itself. The canonical store for the data is the PDS and the Frontpage app acts as a kind of index that can efficiently query and aggregate data from every actor's PDS. 8 8 + 9 9 + This makes the data architecture and flow more complex. 10 10 + 11 11 + ## Data Layers 12 12 + 13 13 + ### API Layer 14 14 + 15 15 + Code for this layer is in `packages/frontpage/lib/api`. This layer is responsible for handling incoming API requests, validating input, and orchestrating calls to the underlying data layers (DB and Atproto). 16 16 + 17 17 + This layer is essentially the public API of the Frontpage app, although in reality we don't expose this API directly right now - we just call these functions from within our Next.js server-side code. In the future we will likely expose this API via [XRPC](https://atproto.com/specs/xrpc) so that other apps can interact with Frontpage programmatically. 18 18 + 19 19 + Generally, the Next.js app should only speak to this layer, and not directly to the DB or Atproto layers. There are some exceptions to this for legacy reasons, but the goal is to eventually have all data access go through this layer. 20 20 + 21 21 + As this layer is designed to speak XRPC, it uses AT protocol types and concepts where possible. It borrows many conventions from the Bsky API eg. Using AT URIs to identify resources. Unlike the Bsky API it also offers write operations, not just read operations. This is because Frontpage chooses option 2 in [Paul's Leaflet](https://pfrazee.leaflet.pub/3m5hwua4sh22v) while Bsky chooses option 1. The main reason for this is to allow Frontpage to read it's own writes, Bsky is able to do this via specific logic that it injects into the PDS itself - we don't have that option. 22 22 + 23 23 + ### DB Layer 24 24 + 25 25 + Code for this layer is in `packages/frontpage/lib/data/db`. This layer is responsible for interacting with the Frontpage database. It's structured like a traditional database access layer, with functions for creating, reading, updating, and deleting records in the database. 26 26 + 27 27 + ### Atproto Layer 28 28 + 29 29 + Code for this layer is in `packages/frontpage/lib/data/atproto`. This layer is responsible for interacting with the PDS directly (CRUD operations on AT protocol records). It uses the AT protocol client to perform these operations. 30 30 + 31 31 + Implementation is handled almost entirely by the `@repo/frontpage-atproto-client` package (contained in this repo) which is a generated client based on the AT protocol schemas in `lexicons/`. 32 32 + 33 33 + ### Indexing Layer 34 34 + 35 35 + Code for this layer is in `packages/frontpage/app/api/receive_hook`. This layer is responsible for receiving and processing events from AT protocol PDSes when users create or update records. It processes these events and updates the Frontpage database accordingly. 36 36 + 37 37 + This layer is crucial for keeping the Frontpage app's data in sync with the AT protocol network as it's always possible for users to update their data directly on their PDS or via another app without going through Frontpage. 38 38 + 39 39 + ## Example data flow: Creating a Post 40 40 + 41 41 +  42 42 +  43 43 + 44 44 + --- 45 45 + 46 46 + ¹ In this context, "app" refers to the Frontpage application and its associated backend services including the database. In AT this is often referred to as the "appview" but this can be confusing: https://pfrazee.leaflet.pub/3lyucxaykg22w

docs/images/create-post-data-flow-dark.png

This is a binary file and will not be displayed.

docs/images/create-post-data-flow-light.png

This is a binary file and will not be displayed.