+65 -36

README.md

··· 1 1 # Slices 2 2 3 3 - An open-source platform for building AT Protocol appviews with custom data 3 3 + An open-source platform for building AT Protocol AppViews with custom data 4 4 schemas, automatic SDK generation, and built-in sync capabilities. 5 5 6 6 + ⚠️ **Alpha Release** Version 0.x - This project is in active development 7 7 + approaching stability. Core features are implemented and functional, though APIs 8 8 + may undergo refinements. Suitable for early adoption and development use. 9 9 + Production deployment is possible with thorough testing for your specific use 10 10 + case. 11 11 + 6 12 ## Overview 7 13 8 14 Slices enables developers to create "slices" - custom appviews within the AT ··· 20 26 - **OAuth Integration**: Built-in AT Protocol authentication 21 27 - **Multi-tenant Architecture**: Each slice operates independently with its own 22 28 data validated against its lexicons 23 23 - - **Dynamic API Endpoints**: CRUD operations automatically created for each 24 24 - lexicon record type (collection) 29 29 + - **Dynamic API Endpoints**: CRUD operations with SQL-like queries automatically 30 30 + created for each lexicon record type (collection) 25 31 26 32 ## Documentation 27 33 ··· 32 38 - [API Reference](./docs/api-reference.md) - Complete API documentation 33 39 - [SDK Usage](./docs/sdk-usage.md) - Using generated TypeScript clients 34 40 35 35 - ## Quick Start 41 41 + ## Development Quick Start 36 42 37 43 ### Prerequisites 38 44 ··· 46 52 1. Clone the repository: 47 53 48 54 ```bash 49 49 - git clone https://tangled.sh/justslices.net/core 55 55 + git clone https://tangled.sh/slices.network/slices 50 56 cd core 51 57 ``` 52 58 53 59 2. Set up environment variables: 54 60 55 55 - Create `.env` files in both `/api` and `/frontend` directories (see 56 56 - [Getting Started](./docs/getting-started.md) for details). 61 61 + Create `.env` files in both `/api` and `/frontend` directories (see environment 62 62 + variables section below). 63 63 + 64 64 + 3. Start the required infrastructure services: 65 65 + 66 66 + > **Note**: This section is a work in progress. Currently using Cloudflare 67 67 + > tunnels, but support for alternative solutions like ngrok or Tailscale would 68 68 + > be welcome contributions. 69 69 + 70 70 + ```bash 71 71 + # Start PostgreSQL, AIP OAuth service, Cloudflare tunnel, and Redis 72 72 + CLOUDFLARE_TUNNEL_TOKEN=<your-token> AIP_EXTERNAL_BASE=<your-tunnel-url> docker compose up -d 73 73 + ``` 57 74 58 58 - 3. Start the services: 75 75 + You'll need to provide: 76 76 + 77 77 + - `CLOUDFLARE_TUNNEL_TOKEN`: Your Cloudflare tunnel token for secure access 78 78 + - `AIP_EXTERNAL_BASE`: The external URL for the AIP OAuth service (e.g., 79 79 + `https://tunnel.example.com`) 80 80 + 81 81 + 4. Start the application services: 59 82 60 83 ```bash 61 84 # Start the API ··· 67 90 deno task dev 68 91 ``` 69 92 70 70 - 4. Visit `http://localhost:8000` to access the web interface. 93 93 + 5. Visit `http://localhost:8080` to access the web interface. 71 94 72 95 ## Project Structure 73 96 ··· 78 101 79 102 - **AT Protocol XRPC Handlers**: Dynamic endpoints for slice-specific 80 103 collections with full CRUD operations 104 104 + - **Lexicon Validation**: Validates all records against defined lexicon schemas 81 105 - **Sync Engine**: Bulk synchronization from AT Protocol repositories 82 106 - **Jetstream Integration**: Real-time data streaming from AT Protocol firehose 83 107 - **Database Layer**: PostgreSQL integration with slice-aware queries 84 84 - - **SDK Generation**: Automatically generates type-safe TypeScript clients and 85 85 - OpenAPI specifications 86 108 - **OAuth Integration**: Handles AT Protocol OAuth flows and token management 87 109 88 110 ### Frontend (`/frontend`) ··· 168 190 169 191 ### API Configuration 170 192 171 171 - | Variable | Description | Required | 172 172 - | --------------- | ---------------------------- | -------- | 173 173 - | `DATABASE_URL` | PostgreSQL connection string | Yes | 174 174 - | `AUTH_BASE_URL` | AIP OAuth service URL | Yes | 175 175 - | `PORT` | Server port (default: 3000) | No | 193 193 + | Variable | Description | Required | Default | 194 194 + | -------------------------------------- | -------------------------------------------------------------- | -------- | ------------------------------------- | 195 195 + | `DATABASE_URL` | PostgreSQL connection string | Yes | - | 196 196 + | `AUTH_BASE_URL` | Authentication service base URL | No | `http://localhost:8081` | 197 197 + | `PORT` | Server port | No | `3000` | 198 198 + | `PROCESS_TYPE` | Process type: `all` (HTTP + Jetstream), `app`, or `worker` | No | `all` | 199 199 + | `RELAY_ENDPOINT` | AT Protocol relay endpoint for backfill | No | `https://relay1.us-west.bsky.network` | 200 200 + | `JETSTREAM_HOSTNAME` | AT Protocol Jetstream hostname | No | - | 201 201 + | `SYSTEM_SLICE_URI` | System slice URI | No | Default slice URI | 202 202 + | `DEFAULT_MAX_SYNC_REPOS` | Maximum repositories per sync operation | No | `5000` | 203 203 + | `REDIS_URL` | Redis connection URL (optional, falls back to in-memory cache) | No | - | 204 204 + | `REDIS_TTL_SECONDS` | Redis cache TTL in seconds | No | `3600` | 205 205 + | `JETSTREAM_CURSOR_WRITE_INTERVAL_SECS` | Interval for writing Jetstream cursor position | No | `30` | 206 206 + | `RUST_LOG` | Rust logging level | No | `debug` | 176 207 177 208 ### Frontend Configuration 178 209 179 179 - | Variable | Description | Required | 180 180 - | --------------------- | ------------------------------- | -------- | 181 181 - | `OAUTH_CLIENT_ID` | OAuth application client ID | Yes | 182 182 - | `OAUTH_CLIENT_SECRET` | OAuth application client secret | Yes | 183 183 - | `OAUTH_REDIRECT_URI` | OAuth callback URL | Yes | 184 184 - | `OAUTH_AIP_BASE_URL` | AIP OAuth service URL | Yes | 185 185 - | `API_URL` | Backend API base URL | Yes | 186 186 - | `SLICE_URI` | Default slice URI for queries | Yes | 210 210 + | Variable | Description | Required | Default | 211 211 + | --------------------- | --------------------------------------------- | -------- | ----------- | 212 212 + | `OAUTH_CLIENT_ID` | OAuth application client ID | Yes | - | 213 213 + | `OAUTH_CLIENT_SECRET` | OAuth application client secret | Yes | - | 214 214 + | `OAUTH_REDIRECT_URI` | OAuth callback URL | Yes | - | 215 215 + | `OAUTH_AIP_BASE_URL` | AIP OAuth service URL | Yes | - | 216 216 + | `API_URL` | Backend API base URL | Yes | - | 217 217 + | `SLICE_URI` | Default slice URI for queries | Yes | - | 218 218 + | `ADMIN_DID` | Admin DID for privileged operations | No | - | 219 219 + | `DATABASE_URL` | SQLite database path (frontend session store) | No | `slices.db` | 187 220 188 221 ## Roadmap 189 222 ··· 192 225 - Documentation 193 226 - Frontend UX improvements/social features 194 227 - Support more search and filtering params in collection xrpx handlers and SDK 195 195 - - Surface jetstream and sync logs in the UI 196 196 - - Improve sync and jetstream reliability 228 228 + - SDK error handling improvements (i.e. fuzzy search, date ranges, geo?? etc) 229 229 + - Improve sync and jetstream performace, logging, error handling, and 230 230 + reliability 197 231 - Monitor api container performance and resource usage 198 232 199 233 ### Planned Features 200 234 201 235 - Labeler service integration 202 202 - - CLI tool 203 203 - - API rate limiting 204 236 - Enhanced lexicon management UI 205 237 - Lexicon discovery and sharing 238 238 + - More cli templates (React, Expo, Astro, etc) and examples 206 239 207 240 ## Community 208 241 209 209 - - **Bluesky**: [@justslices.net](https://bsky.app/profile/justslices.net) 210 210 - - **Discord**: [Join our server](https://discord.gg/your-invite) 242 242 + - **Bluesky**: [@slices.network](https://bsky.app/profile/slices.network) 243 243 + - **Discord**: [Join our server](https://discord.gg/NqSd3eW8S8) 211 244 212 245 ## Support 213 246 214 247 - [Documentation](./docs/) 215 215 - - [Discord Community](https://discord.gg/your-invite) 248 248 + - [Discord Community](https://discord.gg/NqSd3eW8S8) 216 249 217 250 ## License 218 251 ··· 223 256 224 257 - Built on the [AT Protocol](https://atproto.com) 225 258 - Inspired by the AT Protocol community 226 226 - - Thanks to all contributors 227 227 - 228 228 - ## Status 229 229 - 230 230 - This project is in active development. APIs may change as we approach v1.0. 259 259 + - Thanks to all contributors and early adopters 231 260 232 261 --- 233 262

+340 -483

docs/api-reference.md

··· 1 1 # API Reference 2 2 3 3 - Complete reference for Slices API endpoints. 4 4 - 5 3 ## Base URL 6 4 7 7 - ```bash 5 5 + ``` 8 6 https://api.slices.network/xrpc/ 9 7 ``` 10 8 11 11 - ## Authentication 12 12 - 13 13 - Most write operations require OAuth 2.0 authentication. Include the access token 14 14 - in the Authorization header: 15 15 - 9 9 + For local development: 16 10 ``` 17 17 - Authorization: Bearer YOUR_ACCESS_TOKEN 11 11 + http://localhost:3000/xrpc/ 18 12 ``` 19 13 20 20 - Read operations typically work without authentication. 21 21 - 22 22 - ## Dynamic Collection Endpoints 23 23 - 24 24 - For each collection in your slice, the following endpoints are automatically 25 25 - generated: 26 26 - 27 27 - ### `[collection].getRecords` 28 28 - 29 29 - Get records in a collection. 30 30 - 31 31 - **Method**: GET 32 32 - 33 33 - **Parameters**: 34 34 - 35 35 - - `slice` (string, required): Slice URI 36 36 - - `limit` (number, optional): Maximum records (default: 50) 37 37 - - `cursor` (string, optional): Pagination cursor 38 38 - - `where` (object, optional): Filter conditions using field-specific queries 39 39 - - `sortBy` (array, optional): Sort specification with field and direction 40 40 - objects 41 41 - 42 42 - ### `[collection].getRecord` 43 43 - 44 44 - Get a single record. 45 45 - 46 46 - **Method**: GET 14 14 + ## Authentication 47 15 48 48 - **Parameters**: 16 16 + Write operations require OAuth 2.0 authentication with a Bearer token: 49 17 50 50 - - `slice` (string, required): Slice URI 51 51 - - `uri` (string, required): Record URI 52 52 - 53 53 - ### `[collection].countRecords` 54 54 - 55 55 - Count records in a collection. 56 56 - 57 57 - **Method**: GET 58 58 - 59 59 - **Parameters**: 60 60 - 61 61 - - `slice` (string, required): Slice URI 62 62 - - `where` (object, optional): Filter conditions using field-specific queries 63 63 - - Other filter parameters (no limit/cursor) 64 64 - 65 65 - **Response**: 66 66 - 67 67 - ```json Code 68 68 - { 69 69 - "count": 150 70 70 - } 71 18 ``` 72 72 - 73 73 - ### `[collection].createRecord` 74 74 - 75 75 - Create a new record. 76 76 - 77 77 - **Method**: POST 78 78 - 79 79 - **Authentication**: Required 80 80 - 81 81 - **Body**: 82 82 - 83 83 - ```json Code 84 84 - { 85 85 - "slice": "at://your-slice-uri", 86 86 - "record": { 87 87 - "$type": "com.recordcollector.album", 88 88 - "title": "Superunknown", 89 89 - "artist": "Soundgarden", 90 90 - "releaseDate": "1994-03-08", 91 91 - "condition": "Near Mint", 92 92 - "genre": ["grunge", "alternative metal"] 93 93 - }, 94 94 - "rkey": "3jklmno456" 95 95 - } 19 19 + Authorization: Bearer YOUR_ACCESS_TOKEN 96 20 ``` 97 21 98 98 - ### `[collection].updateRecord` 99 99 - 100 100 - Update an existing record. 101 101 - 102 102 - **Method**: POST 103 103 - 104 104 - **Authentication**: Required 105 105 - 106 106 - **Body**: 107 107 - 108 108 - ```json Code 109 109 - { 110 110 - "slice": "at://your-slice-uri", 111 111 - "rkey": "3xyz789abc", 112 112 - "record": { 113 113 - "$type": "com.recordcollector.album", 114 114 - "title": "Dirt", 115 115 - "artist": "Alice in Chains", 116 116 - "releaseDate": "1992-09-29", 117 117 - "condition": "Very Good Plus", 118 118 - "notes": "Minor sleeve wear, vinyl plays perfectly" 119 119 - } 120 120 - } 121 121 - ``` 22 22 + Read operations are public by default. 122 23 123 123 - ### `[collection].deleteRecord` 24 24 + ## Collection Endpoints 124 25 125 125 - Delete a record. 26 26 + For each collection in your slice (e.g., `com.recordcollector.album`), the following endpoints are automatically generated: 126 27 127 127 - **Method**: POST 28 28 + # **{collection}.getRecords** 128 29 129 129 - **Authentication**: Required 30 30 + > List records with filtering, sorting, and pagination. 130 31 131 131 - **Body**: 32 32 + **Method:** `POST` 33 33 + **Endpoint:** `/xrpc/{collection}.getRecords` 34 34 + **Example:** `/xrpc/com.recordcollector.album.getRecords` 132 35 133 133 - ```json Code 36 36 + **Request Body:** 37 37 + ```json 134 38 { 135 135 - "rkey": "3abc123xyz" 39 39 + "slice": "at://did:plc:abc123/network.slices.slice/xyz789", 40 40 + "limit": 20, 41 41 + "cursor": "optional-pagination-cursor", 42 42 + "where": { 43 43 + "genre": { "contains": "grunge" }, 44 44 + "condition": { "in": ["Mint", "Near Mint"] } 45 45 + }, 46 46 + "sortBy": [ 47 47 + { "field": "releaseDate", "direction": "desc" } 48 48 + ] 136 49 } 137 50 ``` 138 51 139 139 - ## Core Endpoints 140 140 - 141 141 - ### Slice Management 52 52 + **Parameters:** 53 53 + - `slice` (string, required): The slice URI to query 54 54 + - `limit` (number, optional): Maximum records to return (default: 50, max: 100) 55 55 + - `cursor` (string, optional): Pagination cursor from previous response 56 56 + - `where` (object, optional): Filter conditions. Field filters support operators: `eq`, `contains`, `in`. Special field `json` searches across all fields 57 57 + - `sortBy` (array, optional): Sort specification. Each item has `field` and `direction` ("asc" or "desc") 142 58 143 143 - ### `network.slices.slice.getRecords` 144 144 - 145 145 - Get all slices. 146 146 - 147 147 - **Method**: GET 148 148 - 149 149 - **Parameters**: 150 150 - 151 151 - - `limit` (number, optional): Maximum records to return (default: 50) 152 152 - - `cursor` (string, optional): Pagination cursor 153 153 - - `where` (object, optional): Filter conditions using field-specific queries 154 154 - - `sortBy` (array, optional): Sort specification with field and direction 155 155 - objects 156 156 - 157 157 - **Response**: 158 158 - 159 159 - ```json Code 59 59 + **Response:** 60 60 + ```json 160 61 { 161 62 "records": [ 162 63 { 163 163 - "uri": "at://did:plc:abc/network.slices.slice/xyz", 164 164 - "cid": "bafyrei...", 165 165 - "did": "did:plc:abc", 166 166 - "collection": "network.slices.slice", 64 64 + "uri": "at://did:plc:user123/com.recordcollector.album/3l2w4x5y6z", 65 65 + "cid": "bafyreigbtj4x7ip5legnfznufuopl4sg4knzc2cof6duas4b3q2fy6swua", 66 66 + "did": "did:plc:user123", 67 67 + "collection": "com.recordcollector.album", 167 68 "value": { 168 168 - "name": "My Slice", 169 169 - "domain": "com.example", 170 170 - "createdAt": "2024-01-01T00:00:00Z" 69 69 + "$type": "com.recordcollector.album", 70 70 + "title": "Nevermind", 71 71 + "artist": "Nirvana", 72 72 + "releaseDate": "1991-09-24T00:00:00.000Z", 73 73 + "genre": ["grunge", "alternative rock"], 74 74 + "condition": "Near Mint", 75 75 + "notes": "Original pressing, includes poster" 171 76 }, 172 172 - "indexedAt": "2024-01-01T00:00:00Z" 77 77 + "indexedAt": "2024-03-15T10:30:15.123Z" 173 78 } 174 79 ], 175 175 - "cursor": "next-page-cursor" 80 80 + "cursor": "next-page-cursor-xyz789" 176 81 } 177 82 ``` 178 83 179 179 - ### `network.slices.slice.getRecord` 180 180 - 181 181 - Get a specific slice by URI. 84 84 + # **{collection}.getRecord** 182 85 183 183 - **Method**: GET 184 184 - 185 185 - **Parameters**: 186 186 - 187 187 - - `uri` (string, required): AT Protocol URI of the slice 188 188 - 189 189 - **Response**: Single record object (same structure as getRecords item) 190 190 - 191 191 - ### `network.slices.slice.createRecord` 192 192 - 193 193 - Create a new slice. 194 194 - 195 195 - **Method**: POST 86 86 + > Get a single record by URI. 196 87 197 197 - **Authentication**: Required 88 88 + **Method:** `GET` 89 89 + **Endpoint:** `/xrpc/{collection}.getRecord` 90 90 + **Example:** `/xrpc/com.recordcollector.album.getRecord` 198 91 199 199 - **Body**: 92 92 + **Query Parameters:** 93 93 + - `slice` (string, required): The slice URI 94 94 + - `uri` (string, required): The AT Protocol URI of the record 200 95 201 201 - ```json Code 202 202 - { 203 203 - "slice": "at://your-slice-uri", 204 204 - "record": { 205 205 - "$type": "network.slices.slice", 206 206 - "name": "My New Slice", 207 207 - "domain": "com.example", 208 208 - "createdAt": "2024-01-01T00:00:00Z" 209 209 - }, 210 210 - "rkey": "optional-record-key" 211 211 - } 96 96 + **Example Request:** 97 97 + ``` 98 98 + GET /xrpc/com.recordcollector.album.getRecord?slice=at://did:plc:abc123/network.slices.slice/xyz789&uri=at://did:plc:user123/com.recordcollector.album/3l2w4x5y6z 212 99 ``` 213 100 214 214 - **Response**: 215 215 - 216 216 - ```json Code 101 101 + **Response:** 102 102 + ```json 217 103 { 218 218 - "uri": "at://did:plc:abc/network.slices.slice/xyz", 219 219 - "cid": "bafyrei..." 104 104 + "uri": "at://did:plc:user123/com.recordcollector.album/3l2w4x5y6z", 105 105 + "cid": "bafyreigbtj4x7ip5legnfznufuopl4sg4knzc2cof6duas4b3q2fy6swua", 106 106 + "did": "did:plc:user123", 107 107 + "collection": "com.recordcollector.album", 108 108 + "value": { 109 109 + "$type": "com.recordcollector.album", 110 110 + "title": "Nevermind", 111 111 + "artist": "Nirvana", 112 112 + "releaseDate": "1991-09-24T00:00:00.000Z", 113 113 + "genre": ["grunge", "alternative rock"], 114 114 + "condition": "Near Mint", 115 115 + "notes": "Original pressing, includes poster" 116 116 + }, 117 117 + "indexedAt": "2024-03-15T10:30:15.123Z" 220 118 } 221 119 ``` 222 120 223 223 - ### Slice Operations 121 121 + # **{collection}.countRecords** 224 122 225 225 - ### `network.slices.slice.stats` 123 123 + > Count records matching filter criteria. 226 124 227 227 - Get statistics for a slice. 125 125 + **Method:** `POST` 126 126 + **Endpoint:** `/xrpc/{collection}.countRecords` 127 127 + **Example:** `/xrpc/com.recordcollector.album.countRecords` 228 128 229 229 - **Method**: POST 230 230 - 231 231 - **Body**: 232 232 - 233 233 - ```json Code 129 129 + **Request Body:** 130 130 + ```json 234 131 { 235 235 - "slice": "at://your-slice-uri" 132 132 + "slice": "at://did:plc:abc123/network.slices.slice/xyz789", 133 133 + "where": { 134 134 + "condition": { "in": ["Mint", "Near Mint"] }, 135 135 + "genre": { "contains": "grunge" } 136 136 + } 236 137 } 237 138 ``` 238 139 239 239 - **Response**: 240 240 - 241 241 - ```json Code 140 140 + **Response:** 141 141 + ```json 242 142 { 243 143 "success": true, 244 244 - "collections": ["com.recordcollector.album", "com.recordcollector.review"], 245 245 - "collectionStats": [ 246 246 - { 247 247 - "collection": "com.recordcollector.album", 248 248 - "recordCount": 427, 249 249 - "uniqueActors": 23 250 250 - } 251 251 - ], 252 252 - "totalLexicons": 5, 253 253 - "totalRecords": 500, 254 254 - "totalActors": 25, 255 255 - "message": "Statistics retrieved successfully" 144 144 + "count": 42, 145 145 + "message": "Count retrieved successfully" 256 146 } 257 147 ``` 258 148 259 259 - ### `network.slices.slice.listSliceRecords` 149 149 + # **{collection}.createRecord** 260 150 261 261 - List records across multiple collections in a slice. 151 151 + > Create a new record. 262 152 263 263 - **Method**: POST 264 264 - 265 265 - **Body**: 153 153 + **Method:** `POST` 154 154 + **Endpoint:** `/xrpc/{collection}.createRecord` 155 155 + **Example:** `/xrpc/com.recordcollector.album.createRecord` 156 156 + **Authentication:** Required 266 157 267 267 - ```json Code 158 158 + **Request Body:** 159 159 + ```json 268 160 { 269 269 - "slice": "at://your-slice-uri", 270 270 - "collections": ["com.recordcollector.album", "com.recordcollector.review"], 271 271 - "authors": ["did:plc:optional-filter"], 272 272 - "limit": 20, 273 273 - "cursor": "pagination-cursor" 161 161 + "slice": "at://did:plc:abc123/network.slices.slice/xyz789", 162 162 + "rkey": "optional-custom-key", 163 163 + "record": { 164 164 + "$type": "com.recordcollector.album", 165 165 + "title": "In Utero", 166 166 + "artist": "Nirvana", 167 167 + "releaseDate": "1993-09-21T00:00:00.000Z", 168 168 + "genre": ["grunge", "alternative rock"], 169 169 + "condition": "Very Good Plus", 170 170 + "notes": "Some light wear on sleeve" 171 171 + } 274 172 } 275 173 ``` 276 174 277 277 - **Response**: 175 175 + **Parameters:** 176 176 + - `slice` (string, required): The slice URI 177 177 + - `rkey` (string, optional): Custom record key (auto-generated if omitted) 178 178 + - `record` (object, required): The record data matching your lexicon schema 278 179 279 279 - ```json Code 180 180 + **Response:** 181 181 + ```json 280 182 { 281 281 - "success": true, 282 282 - "records": [ 283 283 - { 284 284 - "uri": "at://did:plc:abc/com.recordcollector.album/xyz", 285 285 - "cid": "bafyrei...", 286 286 - "did": "did:plc:abc", 287 287 - "collection": "com.recordcollector.album", 288 288 - "value": {/* record data */}, 289 289 - "indexedAt": "2024-01-01T00:00:00Z" 290 290 - } 291 291 - ], 292 292 - "cursor": "next-page-cursor" 183 183 + "uri": "at://did:plc:user123/com.recordcollector.album/3abc456def", 184 184 + "cid": "bafyreihj7x5legnfznufuopl4sg4knzc2cof6duas4b3q2fy6swua" 293 185 } 294 186 ``` 295 187 296 296 - ### `network.slices.slice.searchSliceRecords` 297 297 - 298 298 - Search records across multiple collections in a slice by content. 188 188 + # **{collection}.updateRecord** 299 189 300 300 - **Method**: POST 190 190 + > Update an existing record. 301 191 302 302 - **Body**: 192 192 + **Method:** `POST` 193 193 + **Endpoint:** `/xrpc/{collection}.updateRecord` 194 194 + **Example:** `/xrpc/com.recordcollector.album.updateRecord` 195 195 + **Authentication:** Required 303 196 304 304 - ```json Code 197 197 + **Request Body:** 198 198 + ```json 305 199 { 306 306 - "slice": "at://your-slice-uri", 307 307 - "collections": ["com.recordcollector.album", "com.recordcollector.review"], 308 308 - "search": "search term", 309 309 - "authors": ["did:plc:optional-filter"], 310 310 - "limit": 20, 311 311 - "cursor": "pagination-cursor" 200 200 + "slice": "at://did:plc:abc123/network.slices.slice/xyz789", 201 201 + "rkey": "3abc456def", 202 202 + "record": { 203 203 + "$type": "com.recordcollector.album", 204 204 + "title": "In Utero", 205 205 + "artist": "Nirvana", 206 206 + "releaseDate": "1993-09-21T00:00:00.000Z", 207 207 + "genre": ["grunge", "alternative rock", "noise rock"], 208 208 + "condition": "Very Good", 209 209 + "notes": "Updated: slight ring wear visible, plays perfectly" 210 210 + } 312 211 } 313 212 ``` 314 213 315 315 - **Response**: 214 214 + **Parameters:** 215 215 + - `slice` (string, required): The slice URI 216 216 + - `rkey` (string, required): The record key to update 217 217 + - `record` (object, required): The complete updated record data 316 218 317 317 - ```json Code 219 219 + **Response:** 220 220 + ```json 318 221 { 319 319 - "success": true, 320 320 - "records": [ 321 321 - { 322 322 - "uri": "at://did:plc:abc/com.recordcollector.album/xyz", 323 323 - "cid": "bafyrei...", 324 324 - "did": "did:plc:abc", 325 325 - "collection": "com.recordcollector.album", 326 326 - "value": {/* record data */}, 327 327 - "indexedAt": "2024-01-01T00:00:00Z" 328 328 - } 329 329 - ], 330 330 - "cursor": "next-page-cursor" 222 222 + "uri": "at://did:plc:user123/com.recordcollector.album/3abc456def", 223 223 + "cid": "bafyreiabc123legnfznufuopl4sg4knzc2cof6duas4b3q2fy6swua" 331 224 } 332 225 ``` 333 226 334 334 - ### `network.slices.slice.syncUserCollections` 227 227 + # **{collection}.deleteRecord** 335 228 336 336 - Synchronously sync collections for the authenticated user. 229 229 + > Delete a record. 337 230 338 338 - **Method**: POST 339 339 - 340 340 - **Authentication**: Required 341 341 - 342 342 - **Body**: 231 231 + **Method:** `POST` 232 232 + **Endpoint:** `/xrpc/{collection}.deleteRecord` 233 233 + **Example:** `/xrpc/com.recordcollector.album.deleteRecord` 234 234 + **Authentication:** Required 343 235 344 344 - ```json Code 236 236 + **Request Body:** 237 237 + ```json 345 238 { 346 346 - "slice": "at://your-slice-uri", 347 347 - "timeoutSeconds": 30 239 239 + "rkey": "3abc456def" 348 240 } 349 241 ``` 350 242 351 351 - **Response**: 352 352 - 353 353 - ```json Code 354 354 - { 355 355 - "success": true, 356 356 - "reposProcessed": 1, 357 357 - "recordsSynced": 45, 358 358 - "timedOut": false, 359 359 - "message": "Sync completed successfully" 360 360 - } 243 243 + **Response:** 244 244 + ```json 245 245 + {} 361 246 ``` 362 247 363 363 - ### `network.slices.slice.startSync` 248 248 + ## Filtering 364 249 365 365 - Start an asynchronous bulk sync job. 250 250 + The `where` parameter supports powerful filtering: 366 251 367 367 - **Method**: POST 252 252 + ### Filter Operators 368 253 369 369 - **Authentication**: Required 370 370 - 371 371 - **Body**: 372 372 - 373 373 - ```json Code 374 374 - { 375 375 - "slice": "at://your-slice-uri", 376 376 - "collections": ["com.recordcollector.album"], 377 377 - "externalCollections": ["app.bsky.actor.profile"], 378 378 - "repos": ["did:plc:abc", "did:plc:xyz"], 379 379 - "limitPerRepo": 100 380 380 - } 254 254 + - **`eq`**: Exact match 255 255 + ```json 256 256 + { "condition": { "eq": "Mint" } } 381 257 ``` 382 258 383 383 - **Response**: 259 259 + - **`contains`**: Partial text match (case-insensitive) 260 260 + ```json 261 261 + { "artist": { "contains": "pearl jam" } } 262 262 + ``` 384 263 385 385 - ```json Code 386 386 - { 387 387 - "success": true, 388 388 - "jobId": "job-uuid", 389 389 - "message": "Sync job started" 390 390 - } 264 264 + - **`in`**: Match any value in array 265 265 + ```json 266 266 + { "condition": { "in": ["Mint", "Near Mint", "Very Good Plus"] } } 391 267 ``` 392 268 393 393 - ### `network.slices.slice.codegen` 269 269 + ### Special Fields 394 270 395 395 - Generate TypeScript client code. 396 396 - 397 397 - **Method**: POST 398 398 - 399 399 - **Body**: 400 400 - 401 401 - ```json Code 402 402 - { 403 403 - "target": "typescript", 404 404 - "slice": "at://your-slice-uri" 405 405 - } 271 271 + - **`json`**: Search across all fields 272 272 + ```json 273 273 + { "json": { "contains": "nirvana" } } 406 274 ``` 407 275 408 408 - **Response**: 409 409 - 410 410 - ```json Code 276 276 + - **System fields**: Filter by record metadata 277 277 + ```json 411 278 { 412 412 - "success": true, 413 413 - "generatedCode": "// Generated TypeScript client code..." 279 279 + "did": { "eq": "did:plc:user123" }, 280 280 + "collection": { "eq": "com.recordcollector.album" }, 281 281 + "indexedAt": { "contains": "2024-03" } 414 282 } 415 283 ``` 416 284 417 417 - ## Lexicon Management 285 285 + ### Complex Filtering Examples 418 286 419 419 - ### `network.slices.lexicon.getRecords` 420 420 - 421 421 - Get lexicons in a slice. 422 422 - 423 423 - **Method**: GET 424 424 - 425 425 - **Parameters**: Same as collection.getRecords 426 426 - 427 427 - ### `network.slices.lexicon.countRecords` 428 428 - 429 429 - Count lexicons in a slice. 430 430 - 431 431 - **Method**: GET 432 432 - 433 433 - **Parameters**: Same as collection.getRecords (except limit and cursor) 434 434 - 435 435 - **Response**: 436 436 - 437 437 - ```json Code 287 287 + **Multiple conditions (AND logic):** 288 288 + ```json 438 289 { 439 439 - "count": 10 290 290 + "where": { 291 291 + "genre": { "contains": "grunge" }, 292 292 + "condition": { "in": ["Mint", "Near Mint"] }, 293 293 + "releaseDate": { "contains": "1991" } 294 294 + } 440 295 } 441 296 ``` 442 297 443 443 - ### `network.slices.lexicon.createRecord` 444 444 - 445 445 - Add a lexicon to a slice. 446 446 - 447 447 - **Method**: POST 448 448 - 449 449 - **Authentication**: Required 450 450 - 451 451 - **Body**: 452 452 - 453 453 - ```json Code 298 298 + **Array field filtering:** 299 299 + ```json 454 300 { 455 455 - "slice": "at://your-slice-uri", 456 456 - "record": { 457 457 - "$type": "network.slices.lexicon", 458 458 - "nsid": "com.recordcollector.album", 459 459 - "definitions": "{\"lexicon\": 1, ...}", 460 460 - "createdAt": "2024-01-01T00:00:00Z", 461 461 - "slice": "at://your-slice-uri" 301 301 + "where": { 302 302 + "genre": { "contains": "alternative" } 462 303 } 463 304 } 464 305 ``` 465 306 466 466 - ## Actor Management 467 467 - 468 468 - ### `network.slices.slice.getActors` 469 469 - 470 470 - Get actors (users) in a slice. 471 471 - 472 472 - **Method**: GET 473 473 - 474 474 - **Parameters**: 475 475 - 476 476 - - `slice` (string, required): Slice URI 477 477 - - `search` (string, optional): Search query 478 478 - - `dids` (string[], optional): Filter by DIDs 479 479 - - `limit` (number, optional): Maximum results 480 480 - - `cursor` (string, optional): Pagination cursor 307 307 + ## Sorting 481 308 482 482 - **Response**: 309 309 + Sort results using the `sortBy` parameter: 483 310 484 484 - ```json Code 311 311 + ```json 485 312 { 486 486 - "actors": [ 487 487 - { 488 488 - "did": "did:plc:abc", 489 489 - "handle": "user.bsky.social", 490 490 - "sliceUri": "at://slice-uri", 491 491 - "indexedAt": "2024-01-01T00:00:00Z" 492 492 - } 493 493 - ], 494 494 - "cursor": "next-page" 313 313 + "sortBy": [ 314 314 + { "field": "releaseDate", "direction": "desc" }, 315 315 + { "field": "artist", "direction": "asc" } 316 316 + ] 495 317 } 496 318 ``` 497 319 498 498 - ## Blob Upload 320 320 + **Common sort patterns:** 321 321 + - Newest releases first: `[{ "field": "releaseDate", "direction": "desc" }]` 322 322 + - Alphabetical by artist: `[{ "field": "artist", "direction": "asc" }]` 323 323 + - By condition (best first): `[{ "field": "condition", "direction": "asc" }]` 324 324 + - Recently indexed: `[{ "field": "indexedAt", "direction": "desc" }]` 499 325 500 500 - ### `com.atproto.repo.uploadBlob` 326 326 + ## Pagination 501 327 502 502 - Upload a blob (image, file). 328 328 + Use cursor-based pagination for large result sets: 503 329 504 504 - **Method**: POST 330 330 + ```javascript 331 331 + // First request 332 332 + const page1 = await fetch('/xrpc/com.recordcollector.album.getRecords', { 333 333 + method: 'POST', 334 334 + body: JSON.stringify({ 335 335 + slice: 'at://your-slice-uri', 336 336 + limit: 20 337 337 + }) 338 338 + }); 505 339 506 506 - **Authentication**: Required 507 507 - 508 508 - **Headers**: 509 509 - 510 510 - - `Content-Type`: MIME type of the blob 511 511 - 512 512 - **Body**: Raw binary data 513 513 - 514 514 - **Response**: 515 515 - 516 516 - ```json Code 517 517 - { 518 518 - "blob": { 519 519 - "$type": "blob", 520 520 - "ref": { "$link": "bafkrei..." }, 521 521 - "mimeType": "image/jpeg", 522 522 - "size": 127198 523 523 - } 524 524 - } 340 340 + // Next page using cursor 341 341 + const page2 = await fetch('/xrpc/com.recordcollector.album.getRecords', { 342 342 + method: 'POST', 343 343 + body: JSON.stringify({ 344 344 + slice: 'at://your-slice-uri', 345 345 + limit: 20, 346 346 + cursor: page1.cursor 347 347 + }) 348 348 + }); 525 349 ``` 526 350 527 351 ## Error Responses 528 352 529 529 - All endpoints may return error responses: 353 353 + All endpoints return consistent error format: 530 354 531 531 - ```json Code 355 355 + ```json 532 356 { 533 357 "error": "InvalidRequest", 534 534 - "message": "Detailed error message" 358 358 + "message": "Missing required parameter: slice" 535 359 } 536 360 ``` 537 361 538 538 - Common HTTP status codes: 539 539 - 362 362 + **Common HTTP Status Codes:** 540 363 - `200`: Success 541 541 - - `400`: Bad request 542 542 - - `401`: Authentication required 543 543 - - `403`: Forbidden 544 544 - - `404`: Not found 364 364 + - `400`: Bad request (invalid parameters) 365 365 + - `401`: Unauthorized (missing/invalid auth token) 366 366 + - `403`: Forbidden (insufficient permissions) 367 367 + - `404`: Not found (record/collection doesn't exist) 545 368 - `500`: Internal server error 546 369 547 547 - ## Pagination 370 370 + ## OpenAPI Specification 548 371 549 549 - List endpoints support cursor-based pagination: 550 550 - 551 551 - 1. Make initial request without cursor 552 552 - 2. Use returned cursor for next page 553 553 - 3. Continue until no cursor returned 372 372 + Get the OpenAPI spec for your slice: 554 373 555 555 - Example: 374 374 + **Method:** `GET` 375 375 + **Endpoint:** `/xrpc/network.slices.slice.openapi` 556 376 557 557 - ```javascript 558 558 - let cursor = undefined; 559 559 - do { 560 560 - const response = await fetch(`/xrpc/collection.getRecords?cursor=${cursor}`); 561 561 - const data = await response.json(); 562 562 - // Process records 563 563 - cursor = data.cursor; 564 564 - } while (cursor); 565 565 - ``` 377 377 + **Query Parameters:** 378 378 + - `slice` (string, required): The slice URI 566 379 567 567 - ## Filtering 380 380 + **Response:** OpenAPI 3.0 specification with all available endpoints 568 381 569 569 - List endpoints support filtering using the `where` parameter with field-specific 570 570 - query operators: 382 382 + ## Example: Record Collector Application 571 383 572 572 - ### Filter Operators 384 384 + Here's a complete example using the record collector lexicon: 573 385 574 574 - - `eq`: Exact match 575 575 - - `contains`: Partial text match (case-insensitive) 576 576 - - `in`: Match any value in array 577 577 - 578 578 - ### Examples 579 579 - 580 580 - **Exact match filtering:** 581 581 - 582 582 - ```json Code 386 386 + ### Lexicon Definition 387 387 + ```json 583 388 { 584 584 - "where": { 585 585 - "artist": { "eq": "Nirvana" }, 586 586 - "condition": { "eq": "Mint" } 389 389 + "lexicon": 1, 390 390 + "id": "com.recordcollector.album", 391 391 + "defs": { 392 392 + "main": { 393 393 + "type": "record", 394 394 + "record": { 395 395 + "type": "object", 396 396 + "required": ["title", "artist", "releaseDate"], 397 397 + "properties": { 398 398 + "title": { "type": "string", "description": "Album title" }, 399 399 + "artist": { "type": "string", "description": "Artist or band name" }, 400 400 + "releaseDate": { 401 401 + "type": "string", 402 402 + "format": "datetime", 403 403 + "description": "Original release date" 404 404 + }, 405 405 + "genre": { 406 406 + "type": "array", 407 407 + "items": { "type": "string" }, 408 408 + "description": "Music genres" 409 409 + }, 410 410 + "condition": { 411 411 + "type": "string", 412 412 + "description": "Vinyl condition (Mint, Near Mint, etc.)" 413 413 + }, 414 414 + "notes": { 415 415 + "type": "string", 416 416 + "description": "Collector notes" 417 417 + } 418 418 + } 419 419 + } 420 420 + } 587 421 } 588 422 } 589 423 ``` 590 424 591 591 - **Text search filtering:** 592 592 - 593 593 - ```json Code 594 594 - { 595 595 - "where": { 596 596 - "title": { "contains": "nevermind" }, 597 597 - "genre": { "contains": "grunge" } 598 598 - } 599 599 - } 425 425 + ### List Albums by Genre 426 426 + ```bash 427 427 + curl -X POST "https://api.slices.network/xrpc/com.recordcollector.album.getRecords" \ 428 428 + -H "Content-Type: application/json" \ 429 429 + -d '{ 430 430 + "slice": "at://did:plc:abc123/network.slices.slice/xyz789", 431 431 + "where": { "genre": { "contains": "grunge" } }, 432 432 + "sortBy": [{ "field": "releaseDate", "direction": "desc" }], 433 433 + "limit": 10 434 434 + }' 600 435 ``` 601 436 602 602 - **Array filtering:** 603 603 - 604 604 - ```json Code 605 605 - { 606 606 - "where": { 607 607 - "condition": { "in": ["Mint", "Near Mint", "Very Good Plus"] }, 608 608 - "artist": { "in": ["Nirvana", "Pearl Jam", "Soundgarden"] } 609 609 - } 610 610 - } 437 437 + ### Add New Album to Collection 438 438 + ```bash 439 439 + curl -X POST "https://api.slices.network/xrpc/com.recordcollector.album.createRecord" \ 440 440 + -H "Authorization: Bearer YOUR_TOKEN" \ 441 441 + -H "Content-Type: application/json" \ 442 442 + -d '{ 443 443 + "slice": "at://did:plc:abc123/network.slices.slice/xyz789", 444 444 + "record": { 445 445 + "$type": "com.recordcollector.album", 446 446 + "title": "Superunknown", 447 447 + "artist": "Soundgarden", 448 448 + "releaseDate": "1994-03-08T00:00:00.000Z", 449 449 + "genre": ["grunge", "alternative metal"], 450 450 + "condition": "Near Mint", 451 451 + "notes": "Limited edition orange vinyl" 452 452 + } 453 453 + }' 611 454 ``` 612 455 613 613 - **Global search across all fields:** 614 614 - 615 615 - ```json Code 616 616 - { 617 617 - "where": { 618 618 - "json": { "contains": "grunge" } 619 619 - } 620 620 - } 456 456 + ### Search Collection by Condition 457 457 + ```bash 458 458 + curl -X POST "https://api.slices.network/xrpc/com.recordcollector.album.getRecords" \ 459 459 + -H "Content-Type: application/json" \ 460 460 + -d '{ 461 461 + "slice": "at://did:plc:abc123/network.slices.slice/xyz789", 462 462 + "where": { 463 463 + "condition": { "in": ["Mint", "Near Mint"] }, 464 464 + "releaseDate": { "contains": "199" } 465 465 + }, 466 466 + "sortBy": [{ "field": "artist", "direction": "asc" }] 467 467 + }' 621 468 ``` 622 469 623 623 - ## Sorting 470 470 + ### Count Albums in Collection 471 471 + ```bash 472 472 + curl -X POST "https://api.slices.network/xrpc/com.recordcollector.album.countRecords" \ 473 473 + -H "Content-Type: application/json" \ 474 474 + -d '{ 475 475 + "slice": "at://did:plc:abc123/network.slices.slice/xyz789", 476 476 + "where": { "condition": { "eq": "Mint" } } 477 477 + }' 478 478 + ``` 624 479 625 625 - Sort parameter uses an array format with field and direction: 626 626 - 627 627 - ```json Code 628 628 - { 629 629 - "sortBy": [ 630 630 - { "field": "releaseDate", "direction": "desc" }, 631 631 - { "field": "title", "direction": "asc" } 632 632 - ] 633 633 - } 480 480 + ### Update Album Condition 481 481 + ```bash 482 482 + curl -X POST "https://api.slices.network/xrpc/com.recordcollector.album.updateRecord" \ 483 483 + -H "Authorization: Bearer YOUR_TOKEN" \ 484 484 + -H "Content-Type: application/json" \ 485 485 + -d '{ 486 486 + "slice": "at://did:plc:abc123/network.slices.slice/xyz789", 487 487 + "rkey": "3abc456def", 488 488 + "record": { 489 489 + "$type": "com.recordcollector.album", 490 490 + "title": "Ten", 491 491 + "artist": "Pearl Jam", 492 492 + "releaseDate": "1991-08-27T00:00:00.000Z", 493 493 + "genre": ["grunge", "alternative rock"], 494 494 + "condition": "Very Good", 495 495 + "notes": "Updated after closer inspection - slight scuffs on Side B" 496 496 + } 497 497 + }' 634 498 ``` 635 499 636 636 - Examples: 637 637 - 638 638 - - `[{ "field": "releaseDate", "direction": "desc" }]` - Newest releases first 639 639 - - `[{ "field": "artist", "direction": "asc" }]` - Alphabetical by artist 640 640 - - `[{ "field": "releaseDate", "direction": "desc" }, { "field": "title", "direction": "asc" }]` - 641 641 - Newest first, then alphabetical by title 642 642 - 643 500 ## Next Steps 644 501 645 502 - [SDK Usage](./sdk-usage.md) - Using generated TypeScript clients 646 503 - [Getting Started](./getting-started.md) - Build your first application 647 647 - - [Concepts](./concepts.md) - Understand the architecture 504 504 + - [Concepts](./concepts.md) - Understand the architecture

+285 -216

docs/concepts.md

··· 1 1 # Core Concepts 2 2 3 3 - Understanding these core concepts will help you effectively use Slices. 3 3 + Slices is built on four fundamental concepts that work together to create a 4 4 + powerful AT Protocol development platform. 5 5 + 6 6 + ``` 7 7 + Slices → Lexicons → Sync → APIs 8 8 + (Container) → (Schema) → (Data) → (Access) 9 9 + ``` 10 10 + 11 11 + ## **Slices**: Your Data Universe 12 12 + 13 13 + ### What is a Slice? 14 14 + 15 15 + A slice is your own independent data space within the AT Protocol network. Think 16 16 + of it as a database with built-in APIs, authentication, and real-time sync, all 17 17 + isolated from other slices. 4 18 5 5 - ## Slices 19 19 + ### Why Slices Matter 20 20 + 21 21 + In the AT Protocol ecosystem, data lives on Personal Data Servers (PDS) 22 22 + scattered across the network. Slices acts as an **AppView** that: 23 23 + 24 24 + - Aggregates data from across the network 25 25 + - Applies your custom schemas 26 26 + - Provides instant APIs for your data 27 27 + - Maintains complete data isolation 28 28 + 29 29 + ### Creating Your Slice 30 30 + 31 31 + When you create a slice for `com.recordcollector`: 32 32 + 33 33 + ``` 34 34 + Slice Name: My Vinyl Collection 35 35 + Domain: com.recordcollector 36 36 + URI: at://did:plc:abc123/network.slices.slice/xyz789 37 37 + ``` 6 38 7 7 - A slice is an independent appview within the AT Protocol ecosystem. Think of it 8 8 - as your own data universe with custom schemas and records. 39 39 + This slice becomes the container for all your vinyl collection data, completely 40 40 + isolated from other slices. 9 41 10 42 ### Key Properties 11 43 12 12 - - **URI**: Unique AT Protocol URI (e.g., 13 13 - `at://did:plc:abc123/network.slices.slice/3xyz`) 14 14 - - **Name**: Human-readable identifier 15 15 - - **Domain**: Namespace for lexicons (e.g., `com.example`, `social.grain`) 16 16 - - **Creation Date**: When the slice was created 44 44 + - **URI**: Unique AT Protocol identifier 45 45 + - **Domain**: Your namespace (e.g., `com.recordcollector`) 46 46 + - **Isolation**: Complete data separation between slices 47 47 + - **Multi-tenancy**: Multiple users can have data in the same slice 17 48 18 18 - ### Slice Isolation 49 49 + ### Example: Record Collector Slice 19 50 20 20 - Each slice maintains complete data isolation: 51 51 + ```json 52 52 + { 53 53 + "uri": "at://did:plc:abc123/network.slices.slice/xyz789", 54 54 + "name": "Vinyl Collection Manager", 55 55 + "domain": "com.recordcollector", 56 56 + "createdAt": "2024-01-15T10:00:00Z" 57 57 + } 58 58 + ``` 21 59 22 22 - - Records are filtered by slice URI in all queries 23 23 - - Sync operations respect slice boundaries 24 24 - - Statistics are calculated per-slice 25 25 - - Users can have different data in different slices 60 60 + ## **Lexicons**: Data Schemas 26 61 27 27 - ## Lexicons 62 62 + ### What are Lexicons? 28 63 29 29 - Lexicons are JSON schemas that define record types in AT Protocol. They specify 30 30 - the structure, validation rules, and metadata for records. 64 64 + Lexicons are JSON schemas that define the structure of your data in AT Protocol. 65 65 + They're like database tables with built-in validation. In Slices, these lexicons 66 66 + automatically generate APIs for you. 31 67 32 32 - ### Lexicon Structure 68 68 + ### Defining Your Data 33 69 34 34 - ```json Code 70 70 + For a record collector app, you might define an album lexicon: 71 71 + 72 72 + ```json 35 73 { 36 74 "lexicon": 1, 37 75 "id": "com.recordcollector.album", 38 76 "defs": { 39 77 "main": { 40 78 "type": "record", 41 41 - "description": "A vinyl album record", 79 79 + "description": "A vinyl album in the collection", 42 80 "record": { 43 81 "type": "object", 82 82 + "required": ["title", "artist", "releaseDate"], 44 83 "properties": { 45 45 - "title": { "type": "string" }, 46 46 - "artist": { "type": "string" }, 47 47 - "releaseDate": { "type": "string", "format": "datetime" }, 48 48 - "condition": { "type": "string" } 49 49 - }, 50 50 - "required": ["title", "artist"] 84 84 + "title": { 85 85 + "type": "string", 86 86 + "description": "Album title" 87 87 + }, 88 88 + "artist": { 89 89 + "type": "string", 90 90 + "description": "Artist or band name" 91 91 + }, 92 92 + "releaseDate": { 93 93 + "type": "string", 94 94 + "format": "datetime", 95 95 + "description": "Original release date" 96 96 + }, 97 97 + "genre": { 98 98 + "type": "array", 99 99 + "items": { "type": "string" }, 100 100 + "description": "Musical genres" 101 101 + }, 102 102 + "condition": { 103 103 + "type": "string", 104 104 + "enum": [ 105 105 + "Mint", 106 106 + "Near Mint", 107 107 + "Very Good Plus", 108 108 + "Very Good", 109 109 + "Good", 110 110 + "Fair", 111 111 + "Poor" 112 112 + ], 113 113 + "description": "Vinyl condition grading" 114 114 + }, 115 115 + "notes": { 116 116 + "type": "string", 117 117 + "maxLength": 1000, 118 118 + "description": "Collector's notes" 119 119 + } 120 120 + } 51 121 } 52 122 } 53 123 } 54 124 } 55 125 ``` 56 126 57 57 - ### Supported Types 127 127 + ### Primary vs External Lexicons 58 128 59 59 - - **Primitives**: string, number, integer, boolean 60 60 - - **Complex**: object, array, union, ref 61 61 - - **Special**: blob (for media), cid-link, at-uri 62 62 - - **Formats**: datetime, at-identifier, did, handle 129 129 + **Primary Lexicons**: Match your slice domain: 63 130 64 64 - ### Lexicon Namespacing 131 131 + - `com.recordcollector.album` 132 132 + - `com.recordcollector.review` 133 133 + - `com.recordcollector.wishlist` 65 134 66 66 - Lexicons follow reverse domain naming: 135 135 + **External Lexicons**: From other namespaces: 67 136 68 68 - - `com.recordcollector.album` - An album in the recordcollector.com namespace 69 69 - - `com.recordcollector.review` - A vinyl review record 70 70 - - `network.slices.slice` - Core slice record type 71 71 - - `app.bsky.actor.profile` - Bluesky profile (external) 137 137 + - `app.bsky.actor.profile` (Bluesky profiles) 138 138 + - `sh.tangled.repo` (Tangled Repo) 139 139 + - `social.grain.photo` (Grain Photos) 140 140 + - `com.cassettecollector.tape` (Cassette Collector) 72 141 73 73 - ## Collections 142 142 + ### Automatic API Generation 74 143 75 75 - Collections are groups of records with the same lexicon type. They map directly 76 76 - to XRPC endpoints. 144 144 + Each lexicon with `type: "record"` automatically creates endpoints. Record types 145 145 + are the only lexicons that generate CRUD APIs: 77 146 78 78 - ### Primary Collections 147 147 + ``` 148 148 + com.recordcollector.album (type: "record") → 149 149 + /xrpc/com.recordcollector.album.getRecords 150 150 + /xrpc/com.recordcollector.album.createRecord 151 151 + /xrpc/com.recordcollector.album.updateRecord 152 152 + /xrpc/com.recordcollector.album.deleteRecord 153 153 + ``` 79 154 80 80 - Collections that match your slice's domain namespace. For example, if your slice 81 81 - domain is `com.recordcollector`, then `com.recordcollector.album` would be a 82 82 - primary collection. 155 155 + Note: Other lexicon types (like `query`, `procedure`, or `subscription`) serve 156 156 + different purposes and don't create these standard CRUD endpoints. 83 157 84 84 - ### External Collections 158 158 + ## **Sync**: Data Flow 85 159 86 86 - Collections from other namespaces that you've synced into your slice. For 87 87 - example: 160 160 + ### How Data Enters Your Slice 88 161 89 89 - - Bluesky profiles (`app.bsky.actor.profile`) 90 90 - - Bluesky posts (`app.bsky.feed.post`) 91 91 - - Collections from other slices 162 162 + The sync engine manages how data flows into your slice from the AT Protocol 163 163 + network. 92 164 93 93 - ### Collection Operations 165 165 + ### Three Sync Strategies 94 166 95 95 - Both primary and external collections support the same operations: 167 167 + #### 1. Bulk Sync: Historical Import 96 168 97 97 - - `*.getRecords` - Get records with pagination, filtering, and search 98 98 - - `*.getRecord` - Get single record by URI 99 99 - - `*.createRecord` - Create new record 100 100 - - `*.updateRecord` - Update existing record 101 101 - - `*.deleteRecord` - Remove record 102 102 - - `*.countRecords` - Count records with filtering 169 169 + Perfect for initial data loading or periodic updates. When you're first starting 170 170 + out, you might only be syncing your own records from your PDS. But as more 171 171 + people adopt your app and write records to their own PDSs, you can sync from 172 172 + their repositories too, growing your network. 103 173 104 104 - The key difference is conceptual: primary collections are "native" to your 105 105 - slice's domain, while external collections are imported from other namespaces. 174 174 + Specify: 106 175 107 107 - ## Records 176 176 + - Collections to sync (e.g., `com.recordcollector.album`) 177 177 + - External collections (e.g., `app.bsky.actor.profile` for user profiles) 178 178 + - Specific repositories (DIDs) to import from 108 179 109 109 - Records are individual data items stored in collections. 180 180 + #### 2. User Sync: On-Demand 110 181 111 111 - ### Record Properties 182 182 + Automatically syncs when users log in. This is primarily for discovering and 183 183 + syncing external collections. When a new user authenticates for the first time, 184 184 + they become an actor in your slice, allowing you to discover what external 185 185 + collections they have (like Bluesky profiles or posts) and sync that data from 186 186 + their PDS. 112 187 113 113 - - **URI**: Unique AT Protocol URI 114 114 - - **CID**: Content identifier (hash) 115 115 - - **DID**: Owner's decentralized identifier 116 116 - - **Collection**: Lexicon type 117 117 - - **Value**: Actual record data 118 118 - - **IndexedAt**: When record was indexed 188 188 + #### 3. Jetstream: Real-Time Updates 119 189 120 120 - ### Record Keys (rkeys) 190 190 + Connects to the AT Protocol firehose for live updates. This tracks create, 191 191 + update, and delete events as they happen across the network. 121 192 122 122 - Records use keys for identification: 193 193 + ``` 194 194 + Firehose Event → Filter by Collection → Validate → Store in Slice 195 195 + ``` 123 196 124 124 - - **Self**: Special key for singleton records (e.g., profiles) 125 125 - - **TID**: Timestamp-based identifiers 126 126 - - **Custom**: User-defined keys 197 197 + ### Syncing External Data 127 198 128 128 - ### Record Lifecycle 199 199 + Import Bluesky profiles to show who owns each album and their Bluesky avatar. 200 200 + You can sync external collections like `app.bsky.actor.profile` to enrich your 201 201 + slice with user information from the broader AT Protocol network. 129 202 130 130 - 1. **Creation**: Via API or sync 131 131 - 2. **Indexing**: Stored in PostgreSQL 132 132 - 3. **Updates**: New versions with new CIDs 133 133 - 4. **Deletion**: Soft or hard delete 203 203 + ### Performance Features 134 204 135 135 - ## Sync Engine 205 205 + - **CID Deduplication**: Skips unchanged records 206 206 + - **Bulk Operations**: Processes thousands of records efficiently 207 207 + - **Actor Caching**: Reduces database lookups 208 208 + - **Auto-Recovery**: Handles network interruptions 136 209 137 137 - The sync engine imports AT Protocol data into your slice using multiple 138 138 - strategies for optimal performance and reliability. 210 210 + ### Example: Syncing a Vinyl Community 139 211 140 140 - ### Sync Types 212 212 + To sync all albums from a vinyl collecting community, you would: 141 213 142 142 - **Bulk Sync**: One-time import of historical data 214 214 + - List primary collections (`com.recordcollector.album`, 215 215 + `com.recordcollector.review`) 216 216 + - Include external collections (`app.bsky.actor.profile`) to show collector 217 217 + information 143 218 144 144 - - Specify collections to sync 145 145 - - Filter by repositories (DIDs) 146 146 - - Set limits per repository 147 147 - - Uses optimized bulk database operations 219 219 + ## **Code Generation & XRPC Endpoints**: APIs & SDKs 148 220 149 149 - **User Sync**: Sync data for authenticated user 221 221 + ### Dynamic API Creation 150 222 151 151 - - Automatic on login 152 152 - - Timeout protection (30 seconds default) 153 153 - - External collection discovery 154 154 - - Synchronous operation for immediate feedback 223 223 + Every record-type lexicon automatically generates REST-like XRPC endpoints. 155 224 156 156 - **Jetstream Sync**: Real-time updates via WebSocket 225 225 + ### Generated Endpoints 157 226 158 158 - - Subscribe to AT Protocol firehose 159 159 - - Filter relevant events by slice collections 160 160 - - Automatic record updates and deletions 161 161 - - Built-in reconnection with exponential backoff 227 227 + For `com.recordcollector.album`: 162 228 163 163 - ### Sync Process 229 229 + #### List Albums 164 230 165 165 - 1. **Discovery**: Find available records via AT Protocol relay 166 166 - 2. **Filtering**: Apply slice and collection filters 167 167 - 3. **Validation**: Check lexicon compliance against slice schemas 168 168 - 4. **Storage**: Index in database using bulk operations 169 169 - 5. **Deduplication**: Skip existing records (by CID comparison) 231 231 + ```http 232 232 + POST /xrpc/com.recordcollector.album.getRecords 233 233 + { 234 234 + "slice": "at://your-slice-uri", 235 235 + "where": { "genre": { "contains": "jazz" } }, 236 236 + "sortBy": [{ "field": "releaseDate", "direction": "desc" }], 237 237 + "limit": 20 238 238 + } 239 239 + ``` 170 240 171 171 - ### Performance Optimizations 241 241 + #### Add Album 172 242 173 173 - **CID-Based Deduplication** 243 243 + ```http 244 244 + POST /xrpc/com.recordcollector.album.createRecord 245 245 + Authorization: Bearer YOUR_TOKEN 246 246 + { 247 247 + "slice": "at://your-slice-uri", 248 248 + "record": { 249 249 + "title": "Kind of Blue", 250 250 + "artist": "Miles Davis", 251 251 + "releaseDate": "1959-08-17T00:00:00Z", 252 252 + "genre": ["jazz", "modal jazz"], 253 253 + "condition": "Very Good Plus" 254 254 + } 255 255 + } 256 256 + ``` 174 257 175 175 - - Compare Content Identifiers (CIDs) before processing 176 176 - - Skip records that haven't changed since last sync 177 177 - - Reduces unnecessary database operations and validation overhead 258 258 + ### TypeScript SDK Generation 178 259 179 179 - **Actor Caching** 260 260 + Slices generates a fully-typed TypeScript client: 180 261 181 181 - - Pre-load actor lookup cache to avoid database hits during Jetstream processing 182 182 - - Cache (DID, slice_uri) mappings for external collection filtering 183 183 - - Periodic cache refresh every 5 minutes 262 262 + ```typescript 263 263 + // Generated SDK with full type safety 264 264 + import { AtprotoClient } from "./generated_client.ts"; 184 265 185 185 - ### Jetstream Reliability 266 266 + const client = new AtprotoClient({ 267 267 + baseUrl: "https://api.slices.network", 268 268 + sliceUri: "at://your-slice-uri", 269 269 + auth: oauthClient, 270 270 + }); 186 271 187 187 - **Automatic Recovery** 188 188 - 189 189 - - Infinite retry loop with exponential backoff (5 seconds → 5 minutes max) 190 190 - - Fresh consumer instance creation on each retry 191 191 - - Database connectivity monitoring and recovery 192 192 - - Connection status tracking via atomic flags 193 193 - 194 194 - **Error Handling** 195 195 - 196 196 - - Graceful degradation when database connections fail 197 197 - - Validation fallback with fresh lexicon loading from database 198 198 - - Separate error handling for primary vs external collections 199 199 - 200 200 - **Configuration Reloading** 201 201 - 202 202 - - Automatic slice configuration refresh every 5 minutes 203 203 - - Dynamic collection filtering based on slice lexicons 204 204 - - Actor cache updates to reflect new slice membership 205 205 - 206 206 - ## XRPC Handlers 207 207 - 208 208 - XRPC (Cross-Protocol Remote Procedure Call) handlers provide the API layer. 209 209 - 210 210 - ### Dynamic Handlers 211 211 - 212 212 - Automatically generated from lexicons: 213 213 - 214 214 - - No manual endpoint creation 215 215 - - Type-safe request/response 216 216 - - Automatic validation 217 217 - - OAuth integration 272 272 + // Fully typed operations 273 273 + const albums = await client.com.recordcollector.album.getRecords({ 274 274 + where: { 275 275 + condition: { in: ["Mint", "Near Mint"] }, 276 276 + genre: { contains: "jazz" }, 277 277 + }, 278 278 + sortBy: [{ field: "artist", direction: "asc" }], 279 279 + limit: 50, 280 280 + }); 218 281 219 219 - ### Core Handlers 220 220 - 221 221 - Built-in endpoints for slice management: 282 282 + // Type-safe record creation 283 283 + const newAlbum = await client.com.recordcollector.album.createRecord({ 284 284 + title: "Blue Train", 285 285 + artist: "John Coltrane", 286 286 + releaseDate: "1958-01-01T00:00:00Z", 287 287 + genre: ["jazz", "hard bop"], 288 288 + condition: "Near Mint", 289 289 + notes: "Original Blue Note pressing", 290 290 + }); 291 291 + ``` 222 292 223 223 - - `network.slices.slice.stats` - Slice statistics 224 224 - - `network.slices.slice.records` - Browse records 225 225 - - `network.slices.slice.codegen` - Generate SDKs 226 226 - - `network.slices.slice.sync` - Trigger sync 227 227 - 228 228 - ### Handler Authentication 229 229 - 230 230 - - **Read Operations**: Optional auth (public by default) 231 231 - - **Write Operations**: Require OAuth tokens 232 232 - - **Admin Operations**: Require slice ownership 293 293 + ### OAuth Integration 233 294 234 234 - ## Generated SDKs 295 295 + Built-in OAuth 2.0 with PKCE: 235 296 236 236 - Type-safe client libraries generated from lexicons. 297 297 + - Read operations: Public by default 298 298 + - Write operations: Require authentication 299 299 + - Automatic token refresh 300 300 + - Secure session management 237 301 238 302 ### SDK Features 239 303 240 240 - - **Type Safety**: Full TypeScript types 241 241 - - **Nested Structure**: Matches lexicon namespacing 242 242 - - **OAuth Integration**: Automatic token handling 243 243 - - **Error Handling**: Retry logic and graceful failures 304 304 + - **Type Safety**: Full TypeScript types from lexicons 305 305 + - **Nested APIs**: `client.com.recordcollector.album.*` 306 306 + - **Error Handling**: Automatic retries and graceful failures 307 307 + - **Blob Support**: Handle images and media files 244 308 245 245 - ### SDK Generation Process 309 309 + ## How It All Works Together 246 310 247 247 - 1. Parse slice lexicons 248 248 - 2. Generate TypeScript interfaces 249 249 - 3. Create client classes 250 250 - 4. Add utility functions 251 251 - 5. Format and validate 311 311 + ``` 312 312 + 1. Create Slice → Define namespace (com.recordcollector) 313 313 + ↓ 314 314 + 2. Add Lexicons → Define data structure (album, review, wishlist) 315 315 + ↓ 316 316 + 3. Sync Data → Import existing vinyl collections 317 317 + → Subscribe to real-time updates 318 318 + ↓ 319 319 + 4. Use APIs → Generated endpoints for all operations 320 320 + → Type-safe SDK for your app 321 321 + ``` 252 322 253 253 - ### Using Generated SDKs 323 323 + ## Practical Example: Building a Vinyl Collector App 254 324 255 255 - ```typescript Code 256 256 - // Initialize client 257 257 - const client = new AtProtoClient(apiUrl, sliceUri, oauthClient); 325 325 + ### Step 1: Create Your Slice 258 326 259 259 - // Use nested structure matching lexicons 260 260 - await client.com.recordcollector.album.getRecords(); 261 261 - await client.app.bsky.actor.profile.getRecord({ uri }); 262 262 - ``` 327 327 + Create a slice using the Slices CLI or web interface: 263 328 264 264 - ## Authentication 329 329 + - Name: "Vintage Vinyl Collectors" 330 330 + - Domain: "com.recordcollector" 265 331 266 266 - OAuth 2.0 with PKCE for secure authentication. 332 332 + ### Step 2: Define Lexicons 267 333 268 268 - ### OAuth Flow 334 334 + Upload your lexicon definitions through the web UI or CLI: 269 335 270 270 - 1. **Authorization**: Redirect to AT Protocol provider 271 271 - 2. **Callback**: Exchange code for tokens 272 272 - 3. **Token Storage**: Secure client-side storage 273 273 - 4. **Refresh**: Automatic token renewal 336 336 + - Album schema (`com.recordcollector.album`) 337 337 + - Review schema (`com.recordcollector.review`) 338 338 + - Wishlist schema (`com.recordcollector.wishlist`) 274 339 275 275 - ### Session Management 340 340 + ### Step 3: Sync Existing Data 276 341 277 277 - - Encrypted cookies for web sessions 278 278 - - Token refresh before expiration 279 279 - - Graceful degradation for read-only access 342 342 + Start a sync job to import your existing collection: 280 343 281 281 - ## Blob Handling 344 344 + - Use the Sync tab in the web UI 345 345 + - Select collections to sync 346 346 + - Specify repositories (or leave empty for all) 282 347 283 283 - Media files use blob references with CDN URLs. 348 348 + ### Step 4: Build Your App 284 349 285 285 - ### Blob Structure 350 350 + ```javascript 351 351 + // Use the generated SDK 352 352 + const client = new AtprotoClient(slice.uri); 286 353 287 287 - ```json Code 288 288 - { 289 289 - "$type": "blob", 290 290 - "ref": { "$link": "bafkreig5bcb..." }, 291 291 - "mimeType": "image/jpeg", 292 292 - "size": 127198 293 293 - } 294 294 - ``` 295 295 - 296 296 - ### CDN URL Generation 297 297 - 298 298 - Convert blob references to CDN URLs using Bluesky's CDN: 354 354 + // List all jazz albums in mint condition 355 355 + const jazzMint = await client.com.recordcollector.album.getRecords({ 356 356 + where: { 357 357 + genre: { contains: "jazz" }, 358 358 + condition: { eq: "Mint" }, 359 359 + }, 360 360 + }); 299 361 300 300 - ```typescript Code 301 301 - recordBlobToCdnUrl(record, blobRef, "avatar"); 302 302 - // -> https://cdn.bsky.app/img/avatar/plain/did:plc:abc/bafkrei...@jpeg 362 362 + // Add a new album to the collection 363 363 + await client.com.recordcollector.album.createRecord({ 364 364 + title: "A Love Supreme", 365 365 + artist: "John Coltrane", 366 366 + releaseDate: "1965-02-01T00:00:00Z", 367 367 + genre: ["jazz", "spiritual jazz"], 368 368 + condition: "Near Mint", 369 369 + }); 303 370 ``` 304 371 305 305 - ### Bluesky CDN Presets 372 372 + ## Summary 306 373 307 307 - - `avatar` - Profile pictures 308 308 - - `banner` - Cover images 309 309 - - `feed_thumbnail` - Small previews 310 310 - - `feed_fullsize` - Full resolution 374 374 + | Concept | Purpose | Record Collector Example | 375 375 + | ------------ | ------------------------------- | ----------------------------------------------------------- | 376 376 + | **Slices** | Isolated data container | `com.recordcollector` namespace with all your vinyl data | 377 377 + | **Lexicons** | Schema definitions | `album`, `review`, `wishlist` record types | 378 378 + | **Sync** | Data import & real-time updates | Import collections from network, live updates via Jetstream | 379 379 + | **Code Gen** | Auto-generated APIs & SDKs | TypeScript client with `getRecords`, `createRecord`, etc. | 311 380 312 381 ## Next Steps 313 382 314 314 - - [API Reference](./api-reference.md) - Detailed endpoint documentation 315 315 - - [SDK Usage](./sdk-usage.md) - Advanced client patterns 316 316 - - [Getting Started](./getting-started.md) - Build your first slice 383 383 + - [Getting Started](./getting-started.md): Create your first slice 384 384 + - [API Reference](./api-reference.md): Detailed endpoint documentation 385 385 + - [SDK Usage](./sdk-usage.md): Advanced SDK patterns

-23

docs/deployment.md

··· 1 1 - # Deployment Guide 2 2 - 3 3 - Documentation for deploying Slices to production is coming soon. 4 4 - 5 5 - ## Basic Requirements 6 6 - 7 7 - - [AIP server](https://github.com/graze-social/aip) running and accessible 8 8 - - Registered OAuth client with AIP 9 9 - - PostgreSQL database 10 10 - 11 11 - ## Environment Setup 12 12 - 13 13 - See the [environment variables](../README.md#environment-variables) section in the README for required configuration. 14 14 - 15 15 - ## Docker 16 16 - 17 17 - Docker images can be built using the provided Dockerfile in each directory. 18 18 - 19 19 - ## More Information 20 20 - 21 21 - For now, refer to: 22 22 - - [Getting Started](./getting-started.md) for local setup 23 23 - - [README](../README.md) for environment configuration

+379 -69

docs/getting-started.md

··· 1 1 - # Getting Started with Slices 1 1 + # Getting Started 2 2 3 3 - This guide will help you set up Slices and create your first slice. 3 3 + Build your first AT Protocol app in minutes with the Slices CLI. 4 4 5 5 - ## Creating Your First Slice 5 5 + ## Quick Start 6 6 7 7 - ### 1. Log In 7 7 + ### Install the CLI 8 8 9 9 - Click "Login" and authenticate with your AT Protocol account. 9 9 + You'll need Deno installed first. Get it at [deno.com](https://deno.com/). 10 10 11 11 - ### 2. Create a Slice 11 11 + ```bash 12 12 + # Install from JSR 13 13 + deno install -g jsr:@slices/cli --name slices 14 14 + ``` 12 15 13 13 - Click "Create Slice" and provide: 16 16 + ### Create Your Project 14 17 15 15 - - **Name**: A friendly name for your slice 16 16 - - **Domain**: Your namespace (e.g., `com.recordcollector`) 18 18 + ```bash 19 19 + # Create a new project with automatic setup 20 20 + slices init my-vinyl-app 17 21 18 18 - ### 3. Define a Lexicon 22 22 + # Or let us generate a name/domain for you 23 23 + slices init 24 24 + ``` 19 25 20 20 - Navigate to your slice and go to the Lexicon tab. Create a lexicon for your 21 21 - first record type: 26 26 + The `slices init` command does everything for you: 22 27 23 23 - ```json Code 28 28 + - Creates a full-stack Deno app with OAuth authentication 29 29 + - Automatically creates a slice on the network 30 30 + - Sets up OAuth credentials 31 31 + - Pulls standard lexicons (including Bluesky profiles) 32 32 + - Generates a TypeScript SDK 33 33 + - Initializes a git repository 34 34 + 35 35 + ### Start Developing 36 36 + 37 37 + ```bash 38 38 + cd my-vinyl-app 39 39 + deno task dev 40 40 + ``` 41 41 + 42 42 + Visit http://localhost:8080 and you're live! 43 43 + 44 44 + ## What You Get 45 45 + 46 46 + The `slices init` command creates a production-ready app with: 47 47 + 48 48 + > More templates/examples coming soon (i.e. React, Expo, Astro, etc) 49 49 + 50 50 + ### Full-Stack Deno Application 51 51 + 52 52 + - **Server-side rendering** with Preact and JSX 53 53 + - **OAuth authentication** with PKCE flow and automatic token refresh 54 54 + - **HTMX integration** for dynamic UI without complex JavaScript 55 55 + - **Tailwind CSS** for beautiful, responsive styling 56 56 + - **SQLite sessions** for secure session management 57 57 + - **Feature-based architecture** for scalable code organization 58 58 + 59 59 + ### AT Protocol Integration 60 60 + 61 61 + - **Configured slice** with your own namespace 62 62 + - **Generated TypeScript SDK** from your lexicons 63 63 + - **Automatic sync** capabilities 64 64 + - **Real-time updates** via Jetstream 65 65 + 66 66 + ### Development Experience 67 67 + 68 68 + - **Hot reload** in development 69 69 + - **Type safety** throughout 70 70 + - **Environment variables** pre-configured 71 71 + - **Git repository** initialized 72 72 + 73 73 + ## Project Structure 74 74 + 75 75 + Here's what the generated project structure looks like: 76 76 + 77 77 + ``` 78 78 + my-vinyl-app/ 79 79 + ├── slices.json # Slice configuration 80 80 + ├── .env # Your credentials (auto-generated) 81 81 + ├── deno.json # Deno configuration 82 82 + ├── lexicons/ # AT Protocol schemas 83 83 + │ ├── com/ 84 84 + │ │ └── recordcollector/ 85 85 + │ │ └── album.json 86 86 + │ └── app/ 87 87 + │ └── bsky/ 88 88 + │ └── actor/ 89 89 + │ └── profile.json 90 90 + └── src/ 91 91 + ├── main.ts # Server entry point 92 92 + ├── config.ts # App configuration 93 93 + ├── generated_client.ts # Your TypeScript SDK 94 94 + ├── routes/ # HTTP routes 95 95 + ├── features/ # Feature modules 96 96 + │ ├── auth/ # OAuth implementation 97 97 + │ └── dashboard/ # Main app UI 98 98 + ├── shared/ # Reusable components 99 99 + └── utils/ # Helper functions 100 100 + ``` 101 101 + 102 102 + ## CLI Commands 103 103 + 104 104 + The Slices CLI is your command center: 105 105 + 106 106 + ### Project Management 107 107 + 108 108 + ```bash 109 109 + # Create a new project 110 110 + slices init my-app 111 111 + 112 112 + # Check your authentication status 113 113 + slices status 114 114 + 115 115 + # Authenticate with Slices network 116 116 + slices login 117 117 + ``` 118 118 + 119 119 + ### Lexicon Management 120 120 + 121 121 + ```bash 122 122 + # Pull lexicons from your slice 123 123 + slices lexicon pull 124 124 + 125 125 + # Push local lexicons to your slice 126 126 + slices lexicon push 127 127 + 128 128 + # List lexicons in your slice 129 129 + slices lexicon list 130 130 + ``` 131 131 + 132 132 + ### Code Generation 133 133 + 134 134 + ```bash 135 135 + # Generate TypeScript SDK from lexicons 136 136 + slices codegen 137 137 + 138 138 + # The SDK is created at src/generated_client.ts 139 139 + ``` 140 140 + 141 141 + ### Monitoring 142 142 + 143 143 + ```bash 144 144 + # View real-time Jetstream logs 145 145 + slices logs 146 146 + 147 147 + # See sync activity and data flow 148 148 + slices logs --verbose 149 149 + ``` 150 150 + 151 151 + ## Working with Lexicons 152 152 + 153 153 + Lexicons define your data structure. The init command includes Bluesky profile 154 154 + lexicons to get you started, but you'll want to add your own custom lexicons for 155 155 + your app. 156 156 + 157 157 + ### Modify an Existing Lexicon 158 158 + 159 159 + Edit `lexicons/com/recordcollector/album.json`: 160 160 + 161 161 + ```json 24 162 { 25 163 "lexicon": 1, 26 164 "id": "com.recordcollector.album", 27 165 "defs": { 28 166 "main": { 29 167 "type": "record", 30 30 - "description": "A vinyl album record", 168 168 + "description": "A vinyl album in my collection", 31 169 "record": { 32 170 "type": "object", 171 171 + "required": ["title", "artist", "releaseDate"], 33 172 "properties": { 34 34 - "title": { 35 35 - "type": "string", 36 36 - "description": "Album title" 37 37 - }, 38 38 - "artist": { 39 39 - "type": "string", 40 40 - "description": "Artist or band name" 41 41 - }, 173 173 + "title": { "type": "string" }, 174 174 + "artist": { "type": "string" }, 42 175 "releaseDate": { 43 176 "type": "string", 44 44 - "format": "datetime", 45 45 - "description": "Original release date" 177 177 + "format": "datetime" 46 178 }, 47 179 "genre": { 48 180 "type": "array", 49 49 - "items": { 50 50 - "type": "string" 51 51 - }, 52 52 - "description": "Music genres" 181 181 + "items": { "type": "string" } 53 182 }, 54 183 "condition": { 55 184 "type": "string", 56 56 - "description": "Vinyl condition (Mint, Near Mint, Very Good, etc.)" 185 185 + "enum": [ 186 186 + "Mint", 187 187 + "Near Mint", 188 188 + "Very Good Plus", 189 189 + "Very Good", 190 190 + "Good", 191 191 + "Fair", 192 192 + "Poor" 193 193 + ] 57 194 }, 58 58 - "notes": { 195 195 + "notes": { "type": "string" } 196 196 + } 197 197 + } 198 198 + } 199 199 + } 200 200 + } 201 201 + ``` 202 202 + 203 203 + ### Create a New Lexicon 204 204 + 205 205 + Create `lexicons/com/recordcollector/review.json`: 206 206 + 207 207 + ```json 208 208 + { 209 209 + "lexicon": 1, 210 210 + "id": "com.recordcollector.review", 211 211 + "defs": { 212 212 + "main": { 213 213 + "type": "record", 214 214 + "description": "Album review", 215 215 + "record": { 216 216 + "type": "object", 217 217 + "required": ["albumUri", "rating", "content"], 218 218 + "properties": { 219 219 + "albumUri": { 59 220 "type": "string", 60 60 - "description": "Collector notes" 221 221 + "format": "at-uri" 222 222 + }, 223 223 + "rating": { 224 224 + "type": "integer", 225 225 + "minimum": 1, 226 226 + "maximum": 5 227 227 + }, 228 228 + "content": { "type": "string" }, 229 229 + "createdAt": { 230 230 + "type": "string", 231 231 + "format": "datetime" 61 232 } 62 62 - }, 63 63 - "required": ["title", "artist", "releaseDate"] 233 233 + } 64 234 } 65 235 } 66 236 } 67 237 } 68 238 ``` 69 239 70 70 - ### 4. Generate TypeScript Client 240 240 + ### Update Your Slice 71 241 72 72 - Navigate to the Code Generation tab and click "Generate TypeScript Client". This 73 73 - creates a type-safe client library for your slice. 242 242 + After modifying lexicons: 74 243 75 75 - ### 5. Use the Generated Client 244 244 + ```bash 245 245 + # Push changes to your slice 246 246 + slices lexicon push 76 247 77 77 - In your application: 248 248 + # Regenerate the TypeScript SDK 249 249 + slices codegen 250 250 + ``` 78 251 79 79 - ```typescript Code 80 80 - import { AtProtoClient } from "./generated-client.ts"; 252 252 + Your SDK at `src/generated_client.ts` now includes the new types and methods! 81 253 82 82 - const client = new AtProtoClient( 83 83 - "http://localhost:3000", 84 84 - "at://did:plc:your-did/network.slices.slice/your-slice-id", 85 85 - ); 254 254 + **Important:** You must push your lexicons to the slice before you can create 255 255 + records. The slice needs to know about your schema in order to validate and 256 256 + store records. 86 257 87 87 - // Get albums 88 88 - const albums = await client.com.recordcollector.album.getRecords(); 258 258 + ## Using the Generated SDK 259 259 + 260 260 + The generated SDK provides a type-safe client for your slice: 89 261 90 90 - // Add a new album to your collection 262 262 + ```typescript 263 263 + import { AtprotoClient } from "./generated_client.ts"; 264 264 + 265 265 + // Initialize the client 266 266 + const client = new AtprotoClient({ 267 267 + baseUrl: "https://api.slices.network", 268 268 + sliceUri: Deno.env.get("SLICE_URI")!, 269 269 + }); 270 270 + 271 271 + // List albums with filtering and sorting 272 272 + const albums = await client.com.recordcollector.album.getRecords({ 273 273 + where: { 274 274 + genre: { contains: "jazz" }, 275 275 + }, 276 276 + sortBy: [{ field: "releaseDate", direction: "desc" }], 277 277 + limit: 20, 278 278 + }); 279 279 + 280 280 + // Add a new album 91 281 const newAlbum = await client.com.recordcollector.album.createRecord({ 92 92 - title: "Nevermind", 93 93 - artist: "Nirvana", 94 94 - releaseDate: "1991-09-24", 95 95 - genre: ["grunge", "alternative rock"], 282 282 + title: "Kind of Blue", 283 283 + artist: "Miles Davis", 284 284 + releaseDate: "1959-08-17T00:00:00Z", 285 285 + genre: ["jazz", "modal jazz"], 96 286 condition: "Near Mint", 97 97 - notes: "Original pressing, includes poster", 287 287 + notes: "Original Columbia pressing", 98 288 }); 99 289 100 290 // Get a specific album 101 291 const album = await client.com.recordcollector.album.getRecord({ 102 292 uri: newAlbum.uri, 293 293 + }); 294 294 + 295 295 + // Update an album 296 296 + await client.com.recordcollector.album.updateRecord({ 297 297 + uri: album.uri, 298 298 + record: { 299 299 + ...album.value, 300 300 + notes: "Verified as first pressing!", 301 301 + }, 302 302 + }); 303 303 + 304 304 + // Delete an album 305 305 + await client.com.recordcollector.album.deleteRecord({ 306 306 + uri: album.uri, 103 307 }); 104 308 ``` 105 309 106 106 - ## Syncing External Data 310 310 + ### External Collections 107 311 108 108 - To import data from other AT Protocol repositories: 312 312 + Since the init command included Bluesky profile lexicons, your SDK has methods 313 313 + for querying them: 109 314 110 110 - ### 1. Navigate to Sync 315 315 + ```typescript 316 316 + // Query users by display name (from included Bluesky lexicons) 317 317 + const profiles = await client.app.bsky.actor.profile.getRecords({ 318 318 + where: { 319 319 + displayName: { contains: "vinyl collector" }, 320 320 + }, 321 321 + }); 322 322 + ``` 111 323 112 112 - Go to your slice and click the Sync tab. 324 324 + Any record-type lexicon you add to your slice will generate corresponding SDK 325 325 + methods when you run `slices codegen`. 113 326 114 114 - ### 2. Configure Sync 327 327 + ## Syncing Data 115 328 116 116 - Choose collections to sync: 329 329 + Once your app is running, you can sync data from the AT Protocol network. 117 330 118 118 - - **Primary Collections**: Your slice's lexicons 119 119 - - **External Collections**: Bluesky or other AT Protocol collections 331 331 + ### User Authentication Sync 120 332 121 121 - ### 3. Start Sync 333 333 + When users log in via OAuth, you can sync their data using the 334 334 + `syncUserCollections` method. This discovers and imports their external 335 335 + collections (like Bluesky profiles and posts). 122 336 123 123 - Specify repositories (DIDs) to sync from, or leave empty to sync all available 124 124 - data. 337 337 + ```typescript 338 338 + // After user logs in 339 339 + await client.network.slices.slice.syncUserCollections(); 340 340 + ``` 125 341 126 126 - ### 4. Monitor Progress 342 342 + ### Manual Bulk Sync 127 343 128 128 - The sync will run in the background. Check the status in the UI or via API. 344 344 + Use the web interface at https://slices.network to start a bulk sync job. 345 345 + Navigate to your slice's Sync tab to configure which collections and 346 346 + repositories to sync. 347 347 + 348 348 + **Note:** If you created new lexicons, you'll be the only one with records 349 349 + initially. As more users adopt your app and write records to their own PDSs, you 350 350 + can sync from their repositories to grow your network. 351 351 + 352 352 + ### Real-time Updates 353 353 + 354 354 + Jetstream automatically tracks creates, updates, and deletes across the network: 355 355 + 356 356 + ```bash 357 357 + # Monitor real-time sync 358 358 + slices logs --slice $SLICE_URI 359 359 + ``` 360 360 + 361 361 + ## Deployment 362 362 + 363 363 + Your app is ready for production deployment. 364 364 + 365 365 + ### Deno Deploy 366 366 + 367 367 + Create a free account at [deno.com/deploy](https://deno.com/deploy). Push your code to GitHub, then connect your repository through the Deno Deploy dashboard to deploy your app. 368 368 + 369 369 + For production use with Deno Deploy, switch from SQLite to Deno KV for OAuth and session storage: 370 370 + 371 371 + ```typescript 372 372 + import { DenoKVOAuthStorage } from "@slices/oauth"; 373 373 + import { DenoKVAdapter } from "@slices/session"; 374 374 + 375 375 + // Configure OAuth with Deno KV storage 376 376 + const oauthClient = new OAuthClient({ 377 377 + clientId: Deno.env.get("OAUTH_CLIENT_ID")!, 378 378 + clientSecret: Deno.env.get("OAUTH_CLIENT_SECRET")!, 379 379 + authBaseUrl: Deno.env.get("OAUTH_AIP_BASE_URL")!, 380 380 + redirectUri: Deno.env.get("OAUTH_REDIRECT_URI")!, 381 381 + storage: new DenoKVOAuthStorage(), // Uses Deno KV 382 382 + }); 383 383 + 384 384 + // Configure sessions with Deno KV adapter 385 385 + const sessionStore = new SessionStore({ 386 386 + adapter: new DenoKVAdapter(), // Uses Deno KV 387 387 + cookieOptions: { 388 388 + secure: true, 389 389 + httpOnly: true, 390 390 + }, 391 391 + }); 392 392 + ``` 393 393 + 394 394 + Deno KV provides serverless-compatible storage that scales automatically with your deployment. 395 395 + 396 396 + ## Manual Setup (Advanced) 397 397 + 398 398 + If you prefer to set things up manually or need custom configuration: 399 399 + 400 400 + ### 1. Create a Slice via Web UI 401 401 + 402 402 + Visit https://slices.network and: 403 403 + 404 404 + 1. Log in with your AT Protocol account 405 405 + 2. Click "Create Slice" 406 406 + 3. Choose your namespace (e.g., `com.recordcollector`) 407 407 + 408 408 + ### 2. Create OAuth Credentials 409 409 + 410 410 + In your slice dashboard: 411 411 + 412 412 + 1. Go to Settings → OAuth Clients 413 413 + 2. Create a new client 414 414 + 3. Set redirect URI: `http://localhost:8080/oauth/callback` 415 415 + 4. Copy the Client ID and Secret 416 416 + 417 417 + ### 3. Set Up Your Project 418 418 + 419 419 + Use any framework you prefer. You can use the generated TypeScript SDK (works 420 420 + with any JavaScript/TypeScript environment) or call the XRPC endpoints directly 421 421 + from any language: 422 422 + 423 423 + ```bash 424 424 + # Configure environment 425 425 + cp .env.example .env 426 426 + # Edit .env with your credentials 427 427 + 428 428 + # Start your project 429 429 + # (commands depend on your framework choice) 430 430 + ``` 129 431 130 432 ## Next Steps 131 433 132 132 - - [Core Concepts](./concepts.md) - Understand slices, lexicons, and collections 133 133 - - [API Reference](./api-reference.md) - Explore available endpoints 134 134 - - [SDK Usage](./sdk-usage.md) - Advanced SDK patterns 135 135 - - [Examples](./examples/) - Sample applications 434 434 + - [Core Concepts](./concepts.md): Understand slices, lexicons, sync, and code 435 435 + generation 436 436 + - [API Reference](./api-reference.md): Detailed endpoint documentation 437 437 + - [SDK Usage](./sdk-usage.md): Advanced SDK patterns and examples 438 438 + - [Examples](./examples/): Sample applications and use cases 439 439 + 440 440 + ## Need Help? 441 441 + 442 442 + - Join our [Discord community](https://discord.gg/slices) 443 443 + - Check out [example apps](https://github.com/slices/examples) 444 444 + - Read the [AT Protocol docs](https://atproto.com/) 445 445 + - Report issues on [Tangled](https://tangled.sh/slices.network/slices/issues)

+98 -64

docs/intro.md

··· 1 1 - # Introduction to Slices 1 1 + # Introduction 2 2 3 3 - Slices is an AT Protocol appview platform that enables developers to create 4 4 - custom data slices (appviews) with their own lexicons, sync AT Protocol data, 5 5 - and generate type-safe SDKs. 3 3 + Slices is an open source platform for building structured data applications on 4 4 + the AT Protocol network. 6 5 7 7 - ## What is a Slice? 6 6 + ## What is Slices? 8 7 9 9 - A slice is a custom appview within the AT Protocol ecosystem. Each slice: 8 8 + Slices lets you define custom data schemas and build applications that store, 9 9 + query, and sync structured records across the decentralized AT Protocol network. 10 10 + Think of it as a schema-first backend that automatically handles data 11 11 + validation, indexing, and cross-network synchronization. 10 12 11 11 - - Has its own domain namespace (e.g., `social.grain`, `xyz.statusphere`) 12 12 - - Defines custom lexicons (schemas) for record types 13 13 - - Can sync both internal and external AT Protocol collections 14 14 - - Provides automatically generated type-safe SDKs 13 13 + ## How Slices Works on AT Protocol 15 14 16 16 - ## Why Slices? 15 15 + ```mermaid 16 16 + flowchart LR 17 17 + Users[Users<br/>Create/Update Records] --> PDS[PDS Nodes<br/>Store user data] 18 18 + PDS --> Firehose[Firehose<br/>Stream of all events] 19 19 + Firehose --> SlicesNetwork[Slices Network - AppView<br/>• Monitors all AT Protocol data<br/>• Routes to relevant slices] 20 20 + SlicesNetwork --> SliceA[Slice A<br/>• Blog lexicons<br/>• Post records<br/>• Comment queries] 21 21 + SlicesNetwork --> SliceB[Slice B<br/>• Music lexicons<br/>• Album records<br/>• Playlist queries] 22 22 + SliceA --> ClientA[Application<br/>Client<br/>• Read/write data] 23 23 + SliceB --> ClientB[Application<br/>Client<br/>• Read/write data] 24 24 + ``` 17 25 18 18 - Building AT Protocol applications typically requires: 26 26 + **Flow:** 19 27 20 20 - - Setting up infrastructure to index and query AT Protocol data 21 21 - - Managing OAuth authentication flows 22 22 - - Implementing XRPC handlers for CRUD operations 23 23 - - Creating client libraries for frontend integration 28 28 + 1. Users create records on their Personal Data Server (PDS) 29 29 + 2. The Firehose streams all network events in real-time 30 30 + 3. The Slices Network monitors the firehose and routes data to relevant slices 31 31 + 4. Each slice indexes only records matching its specific lexicons 32 32 + 5. Application clients connect to specific slices to read/write data 24 33 25 25 - Slices provides all of this infrastructure out of the box, letting you focus on 26 26 - your application logic. 34 34 + ## Quick Start 27 35 28 28 - ## Key Features 36 36 + Get started in under a minute: 29 37 30 30 - ### Dynamic API Generation 38 38 + ```bash 39 39 + # Install the CLI globally 40 40 + deno install -g jsr:@slices/cli --name slices 31 41 32 32 - Define a lexicon, and Slices automatically creates REST endpoints for: 42 42 + # Initialize a new slice project 43 43 + slices init my-app 33 44 34 34 - - Listing records with filtering and sorting 35 35 - - Getting individual records by URI 36 36 - - Creating new records with OAuth authentication 37 37 - - Updating existing records 38 38 - - Deleting records 39 39 - - Searching within collections 45 45 + # Start developing 46 46 + cd my-app 47 47 + deno task dev 48 48 + ``` 40 49 41 41 - ### Data Synchronization 50 50 + The `slices init` command creates a full-stack Deno app with OAuth authentication, automatically creates your slice on the network, and generates a type-safe TypeScript SDK. 42 51 43 43 - Sync AT Protocol data into your slice: 52 52 + ## Simple Example 44 53 45 45 - - Import external collections (e.g., Bluesky profiles, posts) 46 46 - - Filter data by repository or collection 47 47 - - Maintain slice-specific data isolation 48 48 - - Real-time sync via Jetstream (coming soon) 54 54 + Define a schema for vinyl albums: 49 55 50 50 - ### Type-Safe SDK Generation 56 56 + ```json lexicons/com/recordcollector/album.json 57 57 + { 58 58 + "lexicon": 1, 59 59 + "id": "com.recordcollector.album", 60 60 + "defs": { 61 61 + "main": { 62 62 + "type": "record", 63 63 + "record": { 64 64 + "type": "object", 65 65 + "required": ["title", "artist", "releaseDate"], 66 66 + "properties": { 67 67 + "title": { "type": "string" }, 68 68 + "artist": { "type": "string" }, 69 69 + "releaseDate": { "type": "string", "format": "datetime" }, 70 70 + "genre": { "type": "array", "items": { "type": "string" } }, 71 71 + "condition": { "type": "string" } 72 72 + } 73 73 + } 74 74 + } 75 75 + } 76 76 + } 77 77 + ``` 51 78 52 52 - Automatically generate TypeScript clients with: 79 79 + Push your lexicon and regenerate the SDK: 53 80 54 54 - - Full type safety for all record types 55 55 - - OAuth authentication integration 56 56 - - Nested API structure matching your lexicons 57 57 - - Blob/CDN URL utilities for media handling 81 81 + ```bash 82 82 + # Push your lexicon to the slice 83 83 + slices lexicon push 84 84 + 85 85 + # Regenerate TypeScript SDK 86 86 + slices codegen 87 87 + ``` 58 88 59 59 - ### Multi-Tenant Architecture 89 89 + Use the auto-generated, type-safe client: 60 90 61 61 - Each slice operates independently: 91 91 + ```typescript 92 92 + import { AtprotoClient } from "./generated_client.ts"; 62 93 63 63 - - Isolated data storage 64 64 - - Custom lexicon definitions 65 65 - - Separate sync configurations 66 66 - - Per-slice statistics and analytics 94 94 + const client = new AtprotoClient({ 95 95 + baseUrl: "https://api.slices.network", 96 96 + sliceUri: "at://your-slice-uri", 97 97 + }); 67 98 68 68 - ## Use Cases 99 99 + // Get all grunge albums 100 100 + const albums = await client.com.recordcollector.album.getRecords({ 101 101 + where: { genre: { contains: "grunge" } }, 102 102 + sortBy: [{ field: "releaseDate", direction: "desc" }], 103 103 + }); 104 104 + ``` 69 105 70 70 - - **Custom Social Apps**: Build specialized social networks with custom record 71 71 - types 72 72 - - **Data Aggregators**: Collect and organize AT Protocol data for analysis 73 73 - - **Specialized Feeds**: Create curated views of AT Protocol content 74 74 - - **Research Tools**: Index and query specific subsets of AT Protocol data 75 75 - - **Creative Applications**: Blogs, galleries, portfolios built on AT Protocol 106 106 + ## Key Features 76 107 77 77 - ## Architecture Overview 108 108 + - **Schema Validation**: Define lexicons that enforce data structure and constraints 109 109 + - **Auto-generated APIs**: REST endpoints created automatically from your schemas 110 110 + - **TypeScript SDKs**: Type-safe clients generated from your lexicons 111 111 + - **Real-time Sync**: Automatic synchronization across the AT Protocol network 112 112 + - **Advanced Querying**: Filter, sort, and paginate records with a powerful query API 113 113 + - **OAuth Built-in**: Authentication with any AT Protocol account 78 114 79 79 - Slices consists of two main components: 115 115 + ## When to Use Slices 80 116 81 81 - - **API Server** (Rust): Handles AT Protocol integration, database operations, 82 82 - and API endpoints 83 83 - - **Frontend** (Deno): Provides web UI for slice management and user 84 84 - authentication 117 117 + Slices is ideal for: 85 118 86 86 - Both components work together to provide a complete AT Protocol appview 87 87 - platform. The API server can also be run standalone as a headless service and 88 88 - interactive with via it's XRPC apis. 119 119 + - **Social Applications**: Build specialized communities, forums, or social features 120 120 + - **Content Platforms**: Create blogs, documentation sites, or media libraries 121 121 + - **SaaS Products**: Develop collaborative tools with structured data needs 122 122 + - **Web APIs**: Design REST APIs with automatic validation and documentation 123 123 + - **Decentralized Apps**: Build on AT Protocol without managing infrastructure 89 124 90 125 ## Next Steps 91 126 92 127 - [Getting Started](./getting-started.md) - Set up your first slice 93 93 - - [Core Concepts](./concepts.md) - Understand the key concepts 94 94 - - [API Reference](./api-reference.md) - Explore the API endpoints 95 95 - - [SDK Usage](./sdk-usage.md) - Learn to use generated clients 128 128 + - [Core Concepts](./concepts.md) - Understand lexicons and collections 129 129 + - [API Reference](./api-reference.md) - Explore the full API

-118

docs/introduction.md

··· 1 1 - # Introduction 2 2 - 3 3 - Slices is an open source platform for building structured data applications on 4 4 - the AT Protocol network. 5 5 - 6 6 - ## What is Slices? 7 7 - 8 8 - Slices lets you define custom data schemas and build applications that store, 9 9 - query, and sync structured records across the decentralized AT Protocol network. 10 10 - Think of it as a schema-first backend that automatically handles data 11 11 - validation, indexing, and cross-network synchronization. 12 12 - 13 13 - ## How Slices Works on AT Protocol 14 14 - 15 15 - ```mermaid 16 16 - flowchart LR 17 17 - Users[Users<br/>Create/Update Records] --> PDS[PDS Nodes<br/>Store user data] 18 18 - PDS --> Firehose[Firehose<br/>Stream of all events] 19 19 - Firehose --> SlicesNetwork[Slices Network - AppView<br/>• Monitors all AT Protocol data<br/>• Routes to relevant slices] 20 20 - SlicesNetwork --> SliceA[Slice A<br/>• Blog lexicons<br/>• Post records<br/>• Comment queries] 21 21 - SlicesNetwork --> SliceB[Slice B<br/>• Music lexicons<br/>• Album records<br/>• Playlist queries] 22 22 - SliceA --> ClientA[Application<br/>Client<br/>• Read/write data] 23 23 - SliceB --> ClientB[Application<br/>Client<br/>• Read/write data] 24 24 - ``` 25 25 - 26 26 - **Flow:** 27 27 - 28 28 - 1. Users create records on their Personal Data Server (PDS) 29 29 - 2. The Firehose streams all network events in real-time 30 30 - 3. The Slices Network monitors the firehose and routes data to relevant slices 31 31 - 4. Each slice indexes only records matching its specific lexicons 32 32 - 5. Application clients connect to specific slices to read/write data 33 33 - 34 34 - ## Quick Start 35 35 - 36 36 - Get started in under a minute: 37 37 - 38 38 - ```bash 39 39 - # Initialize a new slice project 40 40 - deno run -A jsr:@slices/cli init 41 41 - 42 42 - # Follow the prompts to: 43 43 - # 1. Name your slice 44 44 - # 2. Define your first lexicon 45 45 - # 3. Deploy to the network 46 46 - ``` 47 47 - 48 48 - ## Simple Example 49 49 - 50 50 - Define a schema for a blog post: 51 51 - 52 52 - ```json lexicons/com/myblog/post.json 53 53 - { 54 54 - "lexicon": 1, 55 55 - "id": "com.myblog.post", 56 56 - "defs": { 57 57 - "main": { 58 58 - "type": "record", 59 59 - "record": { 60 60 - "type": "object", 61 61 - "required": ["title", "content", "createdAt"], 62 62 - "properties": { 63 63 - "title": { "type": "string", "maxLength": 200 }, 64 64 - "content": { "type": "string", "maxLength": 10000 }, 65 65 - "tags": { "type": "array", "items": { "type": "string" } }, 66 66 - "createdAt": { "type": "string", "format": "datetime" } 67 67 - } 68 68 - } 69 69 - } 70 70 - } 71 71 - } 72 72 - ``` 73 73 - 74 74 - Deploy your lexicon and generate the TypeScript client: 75 75 - 76 76 - ```bash 77 77 - # Push your lexicon to the slice 78 78 - deno run -A jsr:@slices/cli lexicon push 79 79 - 80 80 - # Generate TypeScript SDK from your lexicons 81 81 - deno run -A jsr:@slices/cli codegen 82 82 - ``` 83 83 - 84 84 - Query your data using the auto-generated API: 85 85 - 86 86 - ```typescript 87 87 - // Get all posts with a specific tag 88 88 - const posts = await client.com.myblog.post.getRecords({ 89 89 - slice: "at://your-slice-uri", 90 90 - where: { tags: { contains: "javascript" } }, 91 91 - sortBy: [{ field: "createdAt", direction: "desc" }], 92 92 - }); 93 93 - ``` 94 94 - 95 95 - ## Key Features 96 96 - 97 97 - - **Schema Validation**: Define lexicons that enforce data structure and constraints 98 98 - - **Auto-generated APIs**: REST endpoints created automatically from your schemas 99 99 - - **TypeScript SDKs**: Type-safe clients generated from your lexicons 100 100 - - **Real-time Sync**: Automatic synchronization across the AT Protocol network 101 101 - - **Advanced Querying**: Filter, sort, and paginate records with a powerful query API 102 102 - - **OAuth Built-in**: Authentication with any AT Protocol account 103 103 - 104 104 - ## When to Use Slices 105 105 - 106 106 - Slices is ideal for: 107 107 - 108 108 - - **Social Applications**: Build specialized communities, forums, or social features 109 109 - - **Content Platforms**: Create blogs, documentation sites, or media libraries 110 110 - - **SaaS Products**: Develop collaborative tools with structured data needs 111 111 - - **Web APIs**: Design REST APIs with automatic validation and documentation 112 112 - - **Decentralized Apps**: Build on AT Protocol without managing infrastructure 113 113 - 114 114 - ## Next Steps 115 115 - 116 116 - - [Getting Started](./getting-started.md) - Set up your first slice 117 117 - - [Core Concepts](./concepts.md) - Understand lexicons and collections 118 118 - - [API Reference](./api-reference.md) - Explore the full API

+81 -95

docs/sdk-usage.md

··· 7 7 After generating your TypeScript client, you can use it directly in your 8 8 project: 9 9 10 10 - ```typescript Code 11 11 - import { AtProtoClient } from "./generated_client.ts"; 10 10 + ```typescript 11 11 + import { AtprotoClient } from "./generated_client.ts"; 12 12 import { OAuthClient } from "@slices/oauth"; 13 13 ``` 14 14 ··· 16 16 17 17 ### Without Authentication (Read-Only) 18 18 19 19 - ```typescript Code 20 20 - const client = new AtProtoClient( 21 21 - "https://api.your-domain.com", 22 22 - "at://did:plc:abc/network.slices.slice/your-slice-rkey", 23 23 - ); 19 19 + ```typescript 20 20 + const client = new AtprotoClient({ 21 21 + baseUrl: "https://api.slices.network", 22 22 + sliceUri: "at://did:plc:abc/network.slices.slice/your-slice-rkey", 23 23 + }); 24 24 25 25 // Read operations work without auth 26 26 const albums = await client.com.recordcollector.album.getRecords(); ··· 28 28 29 29 ### With Authentication (Full Access) 30 30 31 31 - ```typescript Code 31 31 + ```typescript 32 32 import { OAuthClient } from "@slices/oauth"; 33 33 34 34 // Set up OAuth client ··· 41 41 }); 42 42 43 43 // Initialize API client with OAuth 44 44 - const client = new AtProtoClient( 45 45 - "https://api.your-domain.com", 46 46 - "at://did:plc:abc/network.slices.slice/your-slice-rekey", 47 47 - oauthClient, 48 48 - ); 44 44 + const client = new AtprotoClient({ 45 45 + baseUrl: "https://api.slices.network", 46 46 + sliceUri: "at://did:plc:abc/network.slices.slice/your-slice-rkey", 47 47 + auth: oauthClient, 48 48 + }); 49 49 ``` 50 50 51 51 ## CRUD Operations ··· 54 54 55 55 The SDK uses `getRecords` for retrieving records: 56 56 57 57 - ```typescript Code 57 57 + ```typescript 58 58 // Get all vinyl records 59 59 const albums = await client.com.recordcollector.album.getRecords(); 60 60 ··· 142 142 The `countRecords` method allows you to count records without fetching them, 143 143 using the same filtering parameters as `getRecords`: 144 144 145 145 - ```typescript Code 145 145 + ```typescript 146 146 // Count all records 147 147 const total = await client.com.recordcollector.album.countRecords(); 148 148 console.log(`Total albums: ${total.count}`); ··· 186 186 187 187 ### Getting a Single Record 188 188 189 189 - ```typescript Code 189 189 + ```typescript 190 190 const album = await client.com.recordcollector.album.getRecord({ 191 191 uri: "at://did:plc:abc/com.recordcollector.album/3jklmno456", 192 192 }); ··· 197 197 198 198 ### Creating Records 199 199 200 200 - ```typescript Code 200 200 + ```typescript 201 201 // Create with auto-generated key 202 202 const newAlbum = await client.com.recordcollector.album.createRecord({ 203 203 title: "In Utero", ··· 221 221 222 222 ### Updating Records 223 223 224 224 - ```typescript Code 224 224 + ```typescript 225 225 // Get the record key from the URI 226 226 const uri = "at://did:plc:abc/com.recordcollector.album/3jklmno456"; 227 227 const rkey = uri.split("/").pop(); // '3jklmno456' ··· 241 241 242 242 ### Deleting Records 243 243 244 244 - ```typescript Code 244 244 + ```typescript 245 245 const rkey = "3jklmno456"; 246 246 await client.com.recordcollector.album.deleteRecord(rkey); 247 247 ``` ··· 250 250 251 251 Access synced external collections like Bluesky profiles: 252 252 253 253 - ```typescript Code 253 253 + ```typescript 254 254 // Get Bluesky profiles in your slice 255 255 const profiles = await client.app.bsky.actor.profile.getRecords(); 256 256 ··· 268 268 269 269 ### Uploading Blobs 270 270 271 271 - ```typescript Code 271 271 + ```typescript 272 272 // Read file as ArrayBuffer 273 273 const file = await Deno.readFile("./nevermind-cover.jpg"); 274 274 ··· 291 291 292 292 ### Converting Blobs to CDN URLs 293 293 294 294 - ```typescript Code 294 294 + ```typescript 295 295 import { recordBlobToCdnUrl } from "./generated-client.ts"; 296 296 297 297 // Get a record with a blob ··· 318 318 319 319 ## Slice Operations 320 320 321 321 - ### Get Slice Statistics 322 322 - 323 323 - ```typescript Code 324 324 - const stats = await client.network.slices.slice.stats({ 325 325 - slice: "at://your-slice-uri", 326 326 - }); 327 327 - 328 328 - console.log(`Total records: ${stats.totalRecords}`); 329 329 - console.log(`Total actors: ${stats.totalActors}`); 330 330 - 331 331 - stats.collectionStats.forEach((stat) => { 332 332 - console.log(`${stat.collection}: ${stat.recordCount} records`); 333 333 - }); 334 334 - ``` 335 335 - 336 321 ### Get Actors 337 322 338 323 The `getActors` method retrieves actors (users) within a slice with powerful 339 324 filtering and sorting capabilities: 340 325 341 341 - ```typescript Code 326 326 + ```typescript 342 327 // Get all actors in the slice 343 343 - const actors = await client.network.slices.slice.getActors(); 328 328 + const actors = await client.getActors(); 344 329 345 330 // With pagination 346 346 - const page1 = await client.network.slices.slice.getActors({ 331 331 + const page1 = await client.getActors({ 347 332 limit: 20, 348 333 }); 349 349 - const page2 = await client.network.slices.slice.getActors({ 334 334 + const page2 = await client.getActors({ 350 335 limit: 20, 351 336 cursor: page1.cursor, 352 337 }); 353 338 354 339 // Filter by specific DIDs 355 355 - const specificActors = await client.network.slices.slice.getActors({ 340 340 + const specificActors = await client.getActors({ 356 341 where: { 357 342 did: { in: ["did:plc:user1", "did:plc:user2"] }, 358 343 }, 359 344 }); 360 345 361 346 // Search by handle 362 362 - const searchByHandle = await client.network.slices.slice.getActors({ 347 347 + const searchByHandle = await client.getActors({ 363 348 where: { 364 349 handle: { contains: "alice" }, 365 350 }, 366 351 }); 367 352 368 353 // Filter by exact handle 369 369 - const exactHandle = await client.network.slices.slice.getActors({ 354 354 + const exactHandle = await client.getActors({ 370 355 where: { 371 356 handle: { eq: "user.bsky.social" }, 372 357 }, ··· 380 365 381 366 ### Browse Slice Records 382 367 383 383 - #### Get Records from Multiple Collections 384 384 - 385 385 - The `getSliceRecords` method uses the same `where` clause approach: 368 368 + The `getSliceRecords` method retrieves records across multiple collections: 386 369 387 387 - ```typescript Code 370 370 + ```typescript 388 371 // Get records from specific collections 389 389 - const records = await client.network.slices.slice.getSliceRecords({ 372 372 + const records = await client.getSliceRecords({ 390 373 where: { 391 391 - collection: { eq: "com.example.post" }, 374 374 + collection: { eq: "com.recordcollector.album" }, 392 375 did: { eq: "did:plc:specific-author" }, // optional 393 376 }, 394 377 limit: 50, ··· 399 382 }); 400 383 401 384 // Search across collections using specific fields 402 402 - const searchResults = await client.network.slices.slice.getSliceRecords({ 385 385 + const searchResults = await client.getSliceRecords({ 403 386 where: { 404 404 - collection: { eq: "com.example.post" }, 405 405 - title: { contains: "hello world" }, 387 387 + collection: { eq: "com.recordcollector.album" }, 388 388 + title: { contains: "nevermind" }, 406 389 did: { eq: "did:plc:specific-author" }, // optional 407 390 }, 408 391 limit: 50, 409 392 }); 410 393 411 394 // Global search across ALL fields in records 412 412 - const globalSearchResults = await client.network.slices.slice.getSliceRecords({ 395 395 + const globalSearchResults = await client.getSliceRecords({ 413 396 where: { 414 414 - collection: { eq: "com.example.post" }, 415 415 - json: { contains: "hello world" }, // Searches entire record content 397 397 + collection: { eq: "com.recordcollector.album" }, 398 398 + json: { contains: "grunge" }, // Searches entire record content 416 399 did: { eq: "did:plc:specific-author" }, // optional 417 400 }, 418 401 limit: 50, ··· 423 406 }); 424 407 425 408 // Get records from any collection with global text search 426 426 - const allCollectionSearch = await client.network.slices.slice.getSliceRecords({ 409 409 + const allCollectionSearch = await client.getSliceRecords({ 427 410 where: { 428 428 - json: { contains: "important content" }, // Searches ALL fields in ALL collections 411 411 + json: { contains: "seattle" }, // Searches ALL fields in ALL collections 429 412 }, 430 413 limit: 20, 431 414 }); ··· 439 422 440 423 Search within specific fields of your records: 441 424 442 442 - ```typescript Code 425 425 + ```typescript 443 426 // Search in title field only 444 427 const titleSearch = await client.com.recordcollector.album.getRecords({ 445 428 where: { ··· 459 442 460 443 Use the special `json` field to search across **all fields** in a record: 461 444 462 462 - ```typescript Code 445 445 + ```typescript 463 446 // Finds records containing "grunge" anywhere in their data 464 447 const globalSearch = await client.com.recordcollector.album.getRecords({ 465 448 where: { ··· 479 462 480 463 When using `getSliceRecords`, you can search across multiple collections: 481 464 482 482 - ```typescript Code 465 465 + ```typescript 483 466 // Search for "seattle" across all collections 484 484 - const crossCollectionSearch = await client.network.slices.slice.getSliceRecords( 485 485 - { 486 486 - where: { 487 487 - json: { contains: "seattle" }, 488 488 - }, 467 467 + const crossCollectionSearch = await client.getSliceRecords({ 468 468 + where: { 469 469 + json: { contains: "seattle" }, 489 470 }, 490 490 - ); 471 471 + }); 491 472 492 473 // Limit to specific collections 493 493 - const specificSearch = await client.network.slices.slice.getSliceRecords({ 474 474 + const specificSearch = await client.getSliceRecords({ 494 475 where: { 495 476 collection: { 496 477 in: ["com.recordcollector.album", "com.recordcollector.review"], ··· 506 487 using the separate `orWhere` parameter. This provides clean type safety and 507 488 autocomplete for field names: 508 489 509 509 - ```typescript Code 490 490 + ```typescript 510 491 // Find albums by either Nirvana OR Alice in Chains 511 492 const albums = await client.com.recordcollector.album.getRecords({ 512 493 orWhere: { ··· 535 516 // SQL: WHERE release_date = '1991-09-24' AND (artist LIKE '%nirvana%' OR genre LIKE '%grunge%') 536 517 537 518 // OR queries work with cross-collection searches too 538 538 - const crossCollectionOrSearch = await client.network.slices.slice 539 539 - .getSliceRecords({ 540 540 - where: { 541 541 - collection: { eq: "com.recordcollector.album" }, 542 542 - }, 543 543 - orWhere: { 544 544 - artist: { contains: "pearl jam" }, 545 545 - genre: { contains: "alternative rock" }, 546 546 - }, 547 547 - }); 519 519 + const crossCollectionOrSearch = await client.getSliceRecords({ 520 520 + where: { 521 521 + collection: { eq: "com.recordcollector.album" }, 522 522 + }, 523 523 + orWhere: { 524 524 + artist: { contains: "pearl jam" }, 525 525 + genre: { contains: "alternative rock" }, 526 526 + }, 527 527 + }); 548 528 549 529 // You get full autocomplete and type safety for field names in both where and orWhere 550 530 const typedSearch = await client.com.recordcollector.album.getRecords({ ··· 562 542 563 543 ### Sync User Collections 564 544 565 565 - ```typescript Code 545 545 + ```typescript 566 546 // Sync current user's data (requires auth) 567 567 - const syncResult = await client.network.slices.slice.syncUserCollections({ 568 568 - timeoutSeconds: 30, 569 569 - }); 547 547 + const syncResult = await client.syncUserCollections(); 570 548 571 549 console.log(`Synced ${syncResult.recordsSynced} records`); 572 550 ``` 573 551 574 552 ## Error Handling 575 553 576 576 - ```typescript Code 554 554 + ```typescript 577 555 try { 578 556 const post = await client.com.example.post.getRecord({ 579 557 uri: "at://invalid-uri", ··· 593 571 594 572 ### 1. Initialize OAuth 595 573 596 596 - ```typescript Code 574 574 + ```typescript 597 575 const oauthClient = new OAuthClient({ 598 576 clientId: process.env.OAUTH_CLIENT_ID, 599 577 clientSecret: process.env.OAUTH_CLIENT_SECRET, ··· 604 582 605 583 ### 2. Start Authorization 606 584 607 607 - ```typescript Code 585 585 + ```typescript 608 586 const authResult = await oauthClient.authorize({ 609 587 loginHint: "user.bsky.social", 610 588 }); ··· 615 593 616 594 ### 3. Handle Callback 617 595 618 618 - ```typescript Code 596 596 + ```typescript 619 597 // In your callback handler 620 598 const urlParams = new URLSearchParams(window.location.search); 621 599 const code = urlParams.get("code"); ··· 626 604 627 605 ### 4. Use Authenticated Client 628 606 629 629 - ```typescript Code 630 630 - const client = new AtProtoClient(apiUrl, sliceUri, oauthClient); 607 607 + ```typescript 608 608 + const client = new AtprotoClient({ 609 609 + baseUrl: "https://api.slices.network", 610 610 + sliceUri: "at://did:plc:abc/network.slices.slice/your-slice-rkey", 611 611 + auth: oauthClient, 612 612 + }); 631 613 632 614 // OAuth tokens are automatically managed 633 633 - const profile = await client.network.slices.actor.profile.createRecord({ 634 634 - displayName: "New User", 635 635 - description: "My profile", 636 636 - }, true); // useSelfRkey for profile 615 615 + const album = await client.com.recordcollector.album.createRecord({ 616 616 + title: "Ten", 617 617 + artist: "Pearl Jam", 618 618 + releaseDate: "1991-08-27", 619 619 + genre: ["grunge", "alternative rock"], 620 620 + condition: "Mint", 621 621 + }); 637 622 ``` 638 623 639 624 ## Type Safety 640 625 641 626 The generated SDK provides full TypeScript type safety: 642 627 643 643 - ```typescript Code 628 628 + ```typescript 644 629 // TypeScript knows the shape of your records 645 630 const album = await client.com.recordcollector.album.getRecord({ uri }); 646 631 ··· 669 654 670 655 ### Batch Operations 671 656 672 672 - ```typescript Code 657 657 + ```typescript 673 658 // Process records in batches 659 659 + 674 660 async function* getAllAlbums() { 675 661 let cursor: string | undefined; 676 662

+1 -84

docs/self-hosting.md

··· 2 2 3 3 This guide covers how to set up and run your own Slices instance. 4 4 5 5 - ## Prerequisites 6 6 - 7 7 - - Docker and Docker Compose 8 8 - - PostgreSQL (or use Docker) 9 9 - - Deno (for frontend) 10 10 - - Rust and Cargo (for API development) 11 11 - - An AT Protocol account (for OAuth) 12 12 - 13 13 - ## Initial Setup 14 14 - 15 15 - ### 1. Clone the Repository 16 16 - 17 17 - ```bash 18 18 - git clone https://tangled.sh/@slices.network/slices 19 19 - cd slice 20 20 - ``` 21 21 - 22 22 - ### 2. Set Up the Database 23 23 - 24 24 - Start PostgreSQL using Docker: 25 25 - 26 26 - ```bash 27 27 - docker-compose up -d postgres 28 28 - ``` 29 29 - 30 30 - Or use an existing PostgreSQL instance and create a database: 31 31 - 32 32 - ```sql 33 33 - CREATE DATABASE slices; 34 34 - ``` 35 35 - 36 36 - ### 3. Configure Environment Variables 37 37 - 38 38 - Create `.env` files for both API and frontend: 39 39 - 40 40 - **API (`/api/.env`)**: 41 41 - 42 42 - ```bash 43 43 - DATABASE_URL=postgres://user:password@localhost:5432/slices 44 44 - AUTH_BASE_URL=https://aip.your-domain.com 45 45 - PORT=3000 46 46 - ``` 47 47 - 48 48 - **Frontend (`/frontend/.env`)**: 49 49 - 50 50 - ```bash 51 51 - OAUTH_CLIENT_ID=your-client-id 52 52 - OAUTH_CLIENT_SECRET=your-client-secret 53 53 - OAUTH_REDIRECT_URI=http://localhost:8000/oauth/callback 54 54 - OAUTH_AIP_BASE_URL=https://aip.your-domain.com 55 55 - SESSION_ENCRYPTION_KEY=your-32-char-key 56 56 - API_URL=http://localhost:3000 57 57 - SLICE_URI=at://did:plc:your-did/network.slices.slice/your-slice-id 58 58 - DATABASE_URL=slices.db 59 59 - ``` 60 60 - 61 61 - ### 4. Register OAuth Client 62 62 - 63 63 - Register your application with the AIP server: 64 64 - 65 65 - ```bash 66 66 - cd frontend 67 67 - ./scripts/register-oauth-client.sh 68 68 - ``` 69 69 - 70 70 - Save the client ID and secret to your `.env` file. 71 71 - 72 72 - ### 5. Start the Services 73 73 - 74 74 - Start the API server: 75 75 - 76 76 - ```bash 77 77 - cd api 78 78 - cargo run 79 79 - ``` 80 80 - 81 81 - Start the frontend: 82 82 - 83 83 - ```bash 84 84 - cd frontend 85 85 - deno task dev 86 86 - ``` 87 87 - 88 88 - Visit `http://localhost:8000` to access the web interface. 5 5 + WIP - coming soon

+40 -1

frontend/src/features/docs/handlers.tsx

··· 16 16 category: "Getting Started", 17 17 docs: [ 18 18 { 19 19 - slug: "introduction", 19 19 + slug: "intro", 20 20 title: "Introduction", 21 21 description: "Overview of Slices platform", 22 22 }, ··· 238 238 // Custom code block renderer - return content as-is since marked-highlight handles it 239 239 renderer.code = function (token: Tokens.Code) { 240 240 return token.text; // marked-highlight has already processed this 241 241 + }; 242 242 + 243 243 + // Custom table renderer 244 244 + renderer.table = function (token: Tokens.Table) { 245 245 + const header = token.header 246 246 + .map((cell: Tokens.TableCell, i: number) => { 247 247 + const text = this.parser.parseInline(cell.tokens); 248 248 + const align = token.align[i] 249 249 + ? ` style="text-align: ${token.align[i]}"` 250 250 + : ""; 251 251 + return `<th class="px-4 py-2 text-left text-sm font-semibold text-zinc-900 dark:text-white border-b-2 border-zinc-200 dark:border-zinc-700"${align}>${text}</th>`; 252 252 + }) 253 253 + .join(""); 254 254 + 255 255 + const rows = token.rows 256 256 + .map((row: Tokens.TableCell[]) => { 257 257 + const cells = row 258 258 + .map((cell: Tokens.TableCell, i: number) => { 259 259 + const text = this.parser.parseInline(cell.tokens); 260 260 + const align = token.align[i] 261 261 + ? ` style="text-align: ${token.align[i]}"` 262 262 + : ""; 263 263 + return `<td class="px-4 py-2 text-sm text-zinc-700 dark:text-zinc-300 border-b border-zinc-100 dark:border-zinc-800"${align}>${text}</td>`; 264 264 + }) 265 265 + .join(""); 266 266 + return `<tr class="hover:bg-zinc-50 dark:hover:bg-zinc-800/50">${cells}</tr>`; 267 267 + }) 268 268 + .join(""); 269 269 + 270 270 + return `<div class="overflow-x-auto my-6"> 271 271 + <table class="min-w-full divide-y divide-zinc-200 dark:divide-zinc-700"> 272 272 + <thead class="bg-zinc-50 dark:bg-zinc-800/50"> 273 273 + <tr>${header}</tr> 274 274 + </thead> 275 275 + <tbody class="bg-white dark:bg-zinc-900 divide-y divide-zinc-100 dark:divide-zinc-800"> 276 276 + ${rows} 277 277 + </tbody> 278 278 + </table> 279 279 + </div>`; 241 280 }; 242 281 243 282 // Set options and use the custom renderer