···31313232### Lexicon Structure
33333434-```json
3434+```json Code
3535{
3636 "lexicon": 1,
3737 "id": "com.recordcollector.album",
···7878### Primary Collections
79798080Collections that match your slice's domain namespace. For example, if your slice
8181-domain is `com.recordcollector`, then `com.recordcollector.album` would be a primary collection.
8181+domain is `com.recordcollector`, then `com.recordcollector.album` would be a
8282+primary collection.
82838384### External Collections
8485···251252252253### Using Generated SDKs
253254254254-```typescript
255255+```typescript Code
255256// Initialize client
256257const client = new AtProtoClient(apiUrl, sliceUri, oauthClient);
257258···283284284285### Blob Structure
285286286286-```json
287287+```json Code
287288{
288289 "$type": "blob",
289290 "ref": { "$link": "bafkreig5bcb..." },
···296297297298Convert blob references to CDN URLs using Bluesky's CDN:
298299299299-```typescript
300300+```typescript Code
300301recordBlobToCdnUrl(record, blobRef, "avatar");
301302// -> https://cdn.bsky.app/img/avatar/plain/did:plc:abc/bafkrei...@jpeg
302303```
+2-113
docs/getting-started.md
···2233This guide will help you set up Slices and create your first slice.
4455-## Prerequisites
66-77-- Docker and Docker Compose
88-- PostgreSQL (or use Docker)
99-- Deno (for frontend)
1010-- Rust and Cargo (for API development)
1111-- An AT Protocol account (for OAuth)
1212-1313-## Initial Setup
1414-1515-### 1. Clone the Repository
1616-1717-```bash
1818-git clone https://tangled.sh/@slices.network/slices
1919-cd slice
2020-```
2121-2222-### 2. Set Up the Database
2323-2424-Start PostgreSQL using Docker:
2525-2626-```bash
2727-docker-compose up -d postgres
2828-```
2929-3030-Or use an existing PostgreSQL instance and create a database:
3131-3232-```sql
3333-CREATE DATABASE slices;
3434-```
3535-3636-### 3. Configure Environment Variables
3737-3838-Create `.env` files for both API and frontend:
3939-4040-**API (`/api/.env`)**:
4141-4242-```bash
4343-DATABASE_URL=postgres://user:password@localhost:5432/slices
4444-AUTH_BASE_URL=https://aip.your-domain.com
4545-PORT=3000
4646-```
4747-4848-**Frontend (`/frontend/.env`)**:
4949-5050-```bash
5151-OAUTH_CLIENT_ID=your-client-id
5252-OAUTH_CLIENT_SECRET=your-client-secret
5353-OAUTH_REDIRECT_URI=http://localhost:8000/oauth/callback
5454-OAUTH_AIP_BASE_URL=https://aip.your-domain.com
5555-SESSION_ENCRYPTION_KEY=your-32-char-key
5656-API_URL=http://localhost:3000
5757-SLICE_URI=at://did:plc:your-did/network.slices.slice/your-slice-id
5858-DATABASE_URL=slices.db
5959-```
6060-6161-### 4. Register OAuth Client
6262-6363-Register your application with the AIP server:
6464-6565-```bash
6666-cd frontend
6767-./scripts/register-oauth-client.sh
6868-```
6969-7070-Save the client ID and secret to your `.env` file.
7171-7272-### 5. Start the Services
7373-7474-Start the API server:
7575-7676-```bash
7777-cd api
7878-cargo run
7979-```
8080-8181-Start the frontend:
8282-8383-```bash
8484-cd frontend
8585-deno task dev
8686-```
8787-8888-Visit `http://localhost:8000` to access the web interface.
8989-905## Creating Your First Slice
916927### 1. Log In
···10520Navigate to your slice and go to the Lexicon tab. Create a lexicon for your
10621first record type:
10722108108-```json
2323+```json Code
10924{
11025 "lexicon": 1,
11126 "id": "com.recordcollector.album",
···1617616277In your application:
16378164164-```typescript
7979+```typescript Code
16580import { AtProtoClient } from "./generated-client.ts";
1668116782const client = new AtProtoClient(
···218133- [API Reference](./api-reference.md) - Explore available endpoints
219134- [SDK Usage](./sdk-usage.md) - Advanced SDK patterns
220135- [Examples](./examples/) - Sample applications
221221-222222-## Troubleshooting
223223-224224-### Database Connection Issues
225225-226226-- Verify PostgreSQL is running: `docker ps`
227227-- Check DATABASE_URL format
228228-- Ensure database exists
229229-230230-### OAuth Errors
231231-232232-- Verify client ID and secret
233233-- Check redirect URI matches configuration
234234-- Ensure AIP server is accessible
235235-236236-### Sync Not Working
237237-238238-- Check user has necessary permissions
239239-- Verify lexicons are valid
240240-- Check API server logs for errors
241241-242242-### Generated Client Issues
243243-244244-- Regenerate client after lexicon changes
245245-- Ensure API server is running
246246-- Check for TypeScript compilation errors
+118
docs/introduction.md
···11+# Introduction
22+33+Slices is an open source platform for building structured data applications on
44+the AT Protocol network.
55+66+## What is Slices?
77+88+Slices lets you define custom data schemas and build applications that store,
99+query, and sync structured records across the decentralized AT Protocol network.
1010+Think of it as a schema-first backend that automatically handles data
1111+validation, indexing, and cross-network synchronization.
1212+1313+## How Slices Works on AT Protocol
1414+1515+```mermaid
1616+flowchart LR
1717+ Users[Users<br/>Create/Update Records] --> PDS[PDS Nodes<br/>Store user data]
1818+ PDS --> Firehose[Firehose<br/>Stream of all events]
1919+ Firehose --> SlicesNetwork[Slices Network - AppView<br/>• Monitors all AT Protocol data<br/>• Routes to relevant slices]
2020+ SlicesNetwork --> SliceA[Slice A<br/>• Blog lexicons<br/>• Post records<br/>• Comment queries]
2121+ SlicesNetwork --> SliceB[Slice B<br/>• Music lexicons<br/>• Album records<br/>• Playlist queries]
2222+ SliceA --> ClientA[Application<br/>Client<br/>• Read/write data]
2323+ SliceB --> ClientB[Application<br/>Client<br/>• Read/write data]
2424+```
2525+2626+**Flow:**
2727+2828+1. Users create records on their Personal Data Server (PDS)
2929+2. The Firehose streams all network events in real-time
3030+3. The Slices Network monitors the firehose and routes data to relevant slices
3131+4. Each slice indexes only records matching its specific lexicons
3232+5. Application clients connect to specific slices to read/write data
3333+3434+## Quick Start
3535+3636+Get started in under a minute:
3737+3838+```bash
3939+# Initialize a new slice project
4040+deno run -A jsr:@slices/cli init
4141+4242+# Follow the prompts to:
4343+# 1. Name your slice
4444+# 2. Define your first lexicon
4545+# 3. Deploy to the network
4646+```
4747+4848+## Simple Example
4949+5050+Define a schema for a blog post:
5151+5252+```json lexicons/com/myblog/post.json
5353+{
5454+ "lexicon": 1,
5555+ "id": "com.myblog.post",
5656+ "defs": {
5757+ "main": {
5858+ "type": "record",
5959+ "record": {
6060+ "type": "object",
6161+ "required": ["title", "content", "createdAt"],
6262+ "properties": {
6363+ "title": { "type": "string", "maxLength": 200 },
6464+ "content": { "type": "string", "maxLength": 10000 },
6565+ "tags": { "type": "array", "items": { "type": "string" } },
6666+ "createdAt": { "type": "string", "format": "datetime" }
6767+ }
6868+ }
6969+ }
7070+ }
7171+}
7272+```
7373+7474+Deploy your lexicon and generate the TypeScript client:
7575+7676+```bash
7777+# Push your lexicon to the slice
7878+deno run -A jsr:@slices/cli lexicon push
7979+8080+# Generate TypeScript SDK from your lexicons
8181+deno run -A jsr:@slices/cli codegen
8282+```
8383+8484+Query your data using the auto-generated API:
8585+8686+```typescript
8787+// Get all posts with a specific tag
8888+const posts = await client.com.myblog.post.getRecords({
8989+ slice: "at://your-slice-uri",
9090+ where: { tags: { contains: "javascript" } },
9191+ sortBy: [{ field: "createdAt", direction: "desc" }],
9292+});
9393+```
9494+9595+## Key Features
9696+9797+- **Schema Validation**: Define lexicons that enforce data structure and constraints
9898+- **Auto-generated APIs**: REST endpoints created automatically from your schemas
9999+- **TypeScript SDKs**: Type-safe clients generated from your lexicons
100100+- **Real-time Sync**: Automatic synchronization across the AT Protocol network
101101+- **Advanced Querying**: Filter, sort, and paginate records with a powerful query API
102102+- **OAuth Built-in**: Authentication with any AT Protocol account
103103+104104+## When to Use Slices
105105+106106+Slices is ideal for:
107107+108108+- **Social Applications**: Build specialized communities, forums, or social features
109109+- **Content Platforms**: Create blogs, documentation sites, or media libraries
110110+- **SaaS Products**: Develop collaborative tools with structured data needs
111111+- **Web APIs**: Design REST APIs with automatic validation and documentation
112112+- **Decentralized Apps**: Build on AT Protocol without managing infrastructure
113113+114114+## Next Steps
115115+116116+- [Getting Started](./getting-started.md) - Set up your first slice
117117+- [Core Concepts](./concepts.md) - Understand lexicons and collections
118118+- [API Reference](./api-reference.md) - Explore the full API
+104
docs/plugins.md
···11+# Plugins
22+33+Enhance your Slices development workflow with plugins.
44+55+## Lexicon IntelliSense
66+77+A VS Code extension that provides real-time validation and IntelliSense support
88+for AT Protocol lexicon files.
99+1010+[View on VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=SlicesNetwork.lexicon-intellisense)
1111+1212+### Features
1313+1414+- **Real-time validation**: Validates lexicon files as you type using the
1515+ `@slices/lexicon` validator
1616+- **JSON Schema support**: Provides autocomplete and validation for lexicon
1717+ structure
1818+- **Cross-lexicon validation**: Validates references between lexicon files in
1919+ your workspace
2020+- **Error diagnostics**: Shows validation errors directly in the editor
2121+2222+### Installation
2323+2424+Install from the VS Code marketplace:
2525+2626+1. Open VS Code Extensions (Cmd+Shift+X on Mac, Ctrl+Shift+X on Windows/Linux)
2727+2. Search for "Lexicon IntelliSense" by SlicesNetwork
2828+3. Click Install
2929+3030+### Configuration
3131+3232+Configure the extension in VS Code settings:
3333+3434+```json .vscode/settings.json
3535+{
3636+ "lexiconIntelliSense.enableValidation": true,
3737+ "lexiconIntelliSense.lexiconDirectory": "lexicons"
3838+}
3939+```
4040+4141+**Settings:**
4242+4343+- `lexiconIntelliSense.enableValidation`: Enable/disable validation (default:
4444+ `true`)
4545+- `lexiconIntelliSense.lexiconDirectory`: Directory containing lexicon files
4646+ relative to workspace root (default: `"lexicons"`)
4747+4848+### Commands
4949+5050+Access these commands via the Command Palette (Cmd+Shift+P):
5151+5252+- **Lexicon: Validate Current File** - Validates the currently open lexicon file
5353+- **Lexicon: Validate Workspace** - Validates all lexicon files in the workspace
5454+5555+### Usage
5656+5757+The extension automatically activates for:
5858+5959+- JSON files in directories containing "lexicons" in the path
6060+- JSON files with lexicon structure (containing `id` and `defs` fields)
6161+6262+It validates:
6363+6464+- **Structure**: Required `id` and `defs` fields
6565+- **Types**: Correct definition types (record, query, procedure, subscription)
6666+- **References**: Resolvable references to other lexicons
6767+- **Formats**: String formats like datetime, uri, nsid, at-uri, did
6868+6969+### Example
7070+7171+When editing a lexicon file like `lexicons/com/example/post.json`:
7272+7373+```json lexicons/com/example/post.json
7474+{
7575+ "lexicon": 1,
7676+ "id": "com.example.post",
7777+ "defs": {
7878+ "main": {
7979+ "type": "record",
8080+ "description": "A blog post",
8181+ "record": {
8282+ "type": "object",
8383+ "required": ["title", "content"],
8484+ "properties": {
8585+ "title": {
8686+ "type": "string",
8787+ "maxLength": 200
8888+ },
8989+ "content": {
9090+ "type": "string"
9191+ }
9292+ }
9393+ }
9494+ }
9595+ }
9696+}
9797+```
9898+9999+The extension provides:
100100+101101+- Autocomplete for property types and fields
102102+- Validation errors for missing required fields
103103+- Warnings for invalid references
104104+- IntelliSense for lexicon-specific properties
+27-27
docs/sdk-usage.md
···77After generating your TypeScript client, you can use it directly in your
88project:
991010-```typescript
1010+```typescript Code
1111import { AtProtoClient } from "./generated_client.ts";
1212import { OAuthClient } from "@slices/oauth";
1313```
···16161717### Without Authentication (Read-Only)
18181919-```typescript
1919+```typescript Code
2020const client = new AtProtoClient(
2121 "https://api.your-domain.com",
2222 "at://did:plc:abc/network.slices.slice/your-slice-rkey",
···28282929### With Authentication (Full Access)
30303131-```typescript
3131+```typescript Code
3232import { OAuthClient } from "@slices/oauth";
33333434// Set up OAuth client
···54545555The SDK uses `getRecords` for retrieving records:
56565757-```typescript
5757+```typescript Code
5858// Get all vinyl records
5959const albums = await client.com.recordcollector.album.getRecords();
6060···142142The `countRecords` method allows you to count records without fetching them,
143143using the same filtering parameters as `getRecords`:
144144145145-```typescript
145145+```typescript Code
146146// Count all records
147147const total = await client.com.recordcollector.album.countRecords();
148148console.log(`Total albums: ${total.count}`);
···186186187187### Getting a Single Record
188188189189-```typescript
189189+```typescript Code
190190const album = await client.com.recordcollector.album.getRecord({
191191 uri: "at://did:plc:abc/com.recordcollector.album/3jklmno456",
192192});
···197197198198### Creating Records
199199200200-```typescript
200200+```typescript Code
201201// Create with auto-generated key
202202const newAlbum = await client.com.recordcollector.album.createRecord({
203203 title: "In Utero",
···221221222222### Updating Records
223223224224-```typescript
224224+```typescript Code
225225// Get the record key from the URI
226226const uri = "at://did:plc:abc/com.recordcollector.album/3jklmno456";
227227const rkey = uri.split("/").pop(); // '3jklmno456'
···241241242242### Deleting Records
243243244244-```typescript
244244+```typescript Code
245245const rkey = "3jklmno456";
246246await client.com.recordcollector.album.deleteRecord(rkey);
247247```
···250250251251Access synced external collections like Bluesky profiles:
252252253253-```typescript
253253+```typescript Code
254254// Get Bluesky profiles in your slice
255255const profiles = await client.app.bsky.actor.profile.getRecords();
256256···268268269269### Uploading Blobs
270270271271-```typescript
271271+```typescript Code
272272// Read file as ArrayBuffer
273273const file = await Deno.readFile("./nevermind-cover.jpg");
274274···291291292292### Converting Blobs to CDN URLs
293293294294-```typescript
294294+```typescript Code
295295import { recordBlobToCdnUrl } from "./generated-client.ts";
296296297297// Get a record with a blob
···320320321321### Get Slice Statistics
322322323323-```typescript
323323+```typescript Code
324324const stats = await client.network.slices.slice.stats({
325325 slice: "at://your-slice-uri",
326326});
···338338The `getActors` method retrieves actors (users) within a slice with powerful
339339filtering and sorting capabilities:
340340341341-```typescript
341341+```typescript Code
342342// Get all actors in the slice
343343const actors = await client.network.slices.slice.getActors();
344344···384384385385The `getSliceRecords` method uses the same `where` clause approach:
386386387387-```typescript
387387+```typescript Code
388388// Get records from specific collections
389389const records = await client.network.slices.slice.getSliceRecords({
390390 where: {
···439439440440Search within specific fields of your records:
441441442442-```typescript
442442+```typescript Code
443443// Search in title field only
444444const titleSearch = await client.com.recordcollector.album.getRecords({
445445 where: {
···459459460460Use the special `json` field to search across **all fields** in a record:
461461462462-```typescript
462462+```typescript Code
463463// Finds records containing "grunge" anywhere in their data
464464const globalSearch = await client.com.recordcollector.album.getRecords({
465465 where: {
···479479480480When using `getSliceRecords`, you can search across multiple collections:
481481482482-```typescript
482482+```typescript Code
483483// Search for "seattle" across all collections
484484const crossCollectionSearch = await client.network.slices.slice.getSliceRecords(
485485 {
···506506using the separate `orWhere` parameter. This provides clean type safety and
507507autocomplete for field names:
508508509509-```typescript
509509+```typescript Code
510510// Find albums by either Nirvana OR Alice in Chains
511511const albums = await client.com.recordcollector.album.getRecords({
512512 orWhere: {
···562562563563### Sync User Collections
564564565565-```typescript
565565+```typescript Code
566566// Sync current user's data (requires auth)
567567const syncResult = await client.network.slices.slice.syncUserCollections({
568568 timeoutSeconds: 30,
···573573574574## Error Handling
575575576576-```typescript
576576+```typescript Code
577577try {
578578 const post = await client.com.example.post.getRecord({
579579 uri: "at://invalid-uri",
···593593594594### 1. Initialize OAuth
595595596596-```typescript
596596+```typescript Code
597597const oauthClient = new OAuthClient({
598598 clientId: process.env.OAUTH_CLIENT_ID,
599599 clientSecret: process.env.OAUTH_CLIENT_SECRET,
···604604605605### 2. Start Authorization
606606607607-```typescript
607607+```typescript Code
608608const authResult = await oauthClient.authorize({
609609 loginHint: "user.bsky.social",
610610});
···615615616616### 3. Handle Callback
617617618618-```typescript
618618+```typescript Code
619619// In your callback handler
620620const urlParams = new URLSearchParams(window.location.search);
621621const code = urlParams.get("code");
···626626627627### 4. Use Authenticated Client
628628629629-```typescript
629629+```typescript Code
630630const client = new AtProtoClient(apiUrl, sliceUri, oauthClient);
631631632632// OAuth tokens are automatically managed
···640640641641The generated SDK provides full TypeScript type safety:
642642643643-```typescript
643643+```typescript Code
644644// TypeScript knows the shape of your records
645645const album = await client.com.recordcollector.album.getRecord({ uri });
646646···669669670670### Batch Operations
671671672672-```typescript
672672+```typescript Code
673673// Process records in batches
674674async function* getAllAlbums() {
675675 let cursor: string | undefined;
+88
docs/self-hosting.md
···11+# Self-Hosting Slices
22+33+This guide covers how to set up and run your own Slices instance.
44+55+## Prerequisites
66+77+- Docker and Docker Compose
88+- PostgreSQL (or use Docker)
99+- Deno (for frontend)
1010+- Rust and Cargo (for API development)
1111+- An AT Protocol account (for OAuth)
1212+1313+## Initial Setup
1414+1515+### 1. Clone the Repository
1616+1717+```bash
1818+git clone https://tangled.sh/@slices.network/slices
1919+cd slice
2020+```
2121+2222+### 2. Set Up the Database
2323+2424+Start PostgreSQL using Docker:
2525+2626+```bash
2727+docker-compose up -d postgres
2828+```
2929+3030+Or use an existing PostgreSQL instance and create a database:
3131+3232+```sql
3333+CREATE DATABASE slices;
3434+```
3535+3636+### 3. Configure Environment Variables
3737+3838+Create `.env` files for both API and frontend:
3939+4040+**API (`/api/.env`)**:
4141+4242+```bash
4343+DATABASE_URL=postgres://user:password@localhost:5432/slices
4444+AUTH_BASE_URL=https://aip.your-domain.com
4545+PORT=3000
4646+```
4747+4848+**Frontend (`/frontend/.env`)**:
4949+5050+```bash
5151+OAUTH_CLIENT_ID=your-client-id
5252+OAUTH_CLIENT_SECRET=your-client-secret
5353+OAUTH_REDIRECT_URI=http://localhost:8000/oauth/callback
5454+OAUTH_AIP_BASE_URL=https://aip.your-domain.com
5555+SESSION_ENCRYPTION_KEY=your-32-char-key
5656+API_URL=http://localhost:3000
5757+SLICE_URI=at://did:plc:your-did/network.slices.slice/your-slice-id
5858+DATABASE_URL=slices.db
5959+```
6060+6161+### 4. Register OAuth Client
6262+6363+Register your application with the AIP server:
6464+6565+```bash
6666+cd frontend
6767+./scripts/register-oauth-client.sh
6868+```
6969+7070+Save the client ID and secret to your `.env` file.
7171+7272+### 5. Start the Services
7373+7474+Start the API server:
7575+7676+```bash
7777+cd api
7878+cargo run
7979+```
8080+8181+Start the frontend:
8282+8383+```bash
8484+cd frontend
8585+deno task dev
8686+```
8787+8888+Visit `http://localhost:8000` to access the web interface.