vicwalker.dev.br
Mirror from bluesky-social/pds
1# PDS
2
3Welcome to the repository for the official Bluesky PDS (Personal Data Server). This repository includes container images and documentation designed to assist technical people with self-hosting a Bluesky PDS.
4
5## Table of Contents
6
7* [FAQ](#faq)
8 * [What is Bluesky?](#what-is-bluesky)
9 * [What is AT Protocol?](#what-is-at-protocol)
10 * [How can developers get invite codes?](#how-can-developers-get-invite-codes)
11 * [Where is the code?](#where-is-the-code)
12 * [What is the current status of federation?](#what-is-the-current-status-of-federation)
13 * [What should I know about running a PDS in the developer sandbox?](#what-should-i-know-about-running-a-pds-in-the-developer-sandbox)
14* [Self\-hosting PDS](#self-hosting-pds)
15 * [Preparation for self\-hosting PDS](#preparation-for-self-hosting-pds)
16 * [Open your cloud firewall for HTTP and HTTPS](#open-your-cloud-firewall-for-http-and-https)
17 * [Configure DNS for your domain](#configure-dns-for-your-domain)
18 * [Check that DNS is working as expected](#check-that-dns-is-working-as-expected)
19 * [Automatic install on Ubuntu 20\.04/22\.04 or Debian 11/12](#automatic-install-on-ubuntu-20042204-or-debian-1112)
20 * [Installing manually on Ubuntu 22\.04](#installing-manually-on-ubuntu-2204)
21 * [Open ports on your Linux firewall](#open-ports-on-your-linux-firewall)
22 * [Install Docker](#install-docker)
23 * [Uninstall old versions](#uninstall-old-versions)
24 * [Set up the repository](#set-up-the-repository)
25 * [Install Docker Engine](#install-docker-engine)
26 * [Verify Docker Engine installation](#verify-docker-engine-installation)
27 * [Set up the PDS directory](#set-up-the-pds-directory)
28 * [Create the Caddyfile](#create-the-caddyfile)
29 * [Create the PDS env configuration file](#create-the-pds-env-configuration-file)
30 * [Start the PDS containers](#start-the-pds-containers)
31 * [Download the Docker compose file](#download-the-docker-compose-file)
32 * [Create the systemd service](#create-the-systemd-service)
33 * [Start the service](#start-the-service)
34 * [Verify your PDS is online](#verify-your-pds-is-online)
35 * [Obtain your PDS admin password](#obtain-your-pds-admin-password)
36 * [Generate an invite code for your PDS](#generate-an-invite-code-for-your-pds)
37 * [Connecting to your server](#connecting-to-your-server)
38 * [Manually updating your PDS](#manually-updating-your-pds)
39* [PDS environment variables](#pds-environment-variables)
40
41
42## FAQ
43
44### What is Bluesky?
45
46Bluesky is a social media application built on AT Protocol.
47
48Please visit the [Bluesky website](https://bsky.app/) for more information.
49
50### What is AT Protocol?
51
52The Authenticated Transfer Protocol, aka atproto, is a protocol for large-scale distributed social applications.
53
54Please visit the [AT Protocol docs](https://atproto.com/guides/overview) for additional information.
55
56### How can developers get invite codes?
57
58There is no invite required to join the sandbox network. Simply set up your own PDS and generate your own invite codes to create accounts. If you desire an account on the production network (on the official Bluesky PDS) please check out the [Bluesky Developer Waitlist](https://docs.google.com/forms/d/e/1FAIpQLSfCuguykw3HaPxIZMJQKRu8_-vsRew90NALVTDOjCSPDmsGNg/viewform) which prioritizes access for developers wanting to build software on atproto.
59
60### Where is the code?
61
62* [Canonical TypeScript code](https://github.com/bluesky-social/atproto)
63* [Experimental Go code](https://github.com/bluesky-social/indigo)
64
65### What is the current status of federation?
66
67We do not currently support PDS federation on the production network but it is now possible to federate in the developer sandbox.
68
69### What should I know about running a PDS in the developer sandbox?
70
71Read the [SANDBOX.md](https://github.com/bluesky-social/pds/blob/main/SANDBOX.md) for an overview of the sandbox network.
72
73## Self-hosting PDS
74
75Self-hosting a Bluesky PDS means running your own Personal Data Server that is capable of federating with the wider Bluesky social network.
76
77### Preparation for self-hosting PDS
78
79Launch a server on any cloud provider, [Digital Ocean](https://digitalocean.com/) and [Vultr](https://vultr.com/) are two popular choices.
80
81Ensure that you can ssh to your server and have root access.
82
83**Server Requirements**
84* Public IPv4 address
85* Public DNS name
86* Public inbound internet access permitted on port 80/tcp and 443/tcp
87
88**Server Recommendations**
89| | |
90| ---------------- | ------------ |
91| Operating System | Ubuntu 22.04 |
92| Memory (RAM) | 2+ GB |
93| CPU Cores | 2+ |
94| Storage | 40+ GB SSD |
95| Architectures | amd64, arm64 |
96
97**Note:** It is a good security practice to restrict inbound ssh access (port 22/tcp) to your own computer's public IP address. You can check your current public IP address using [ifconfig.me](https://ifconfig.me/).
98
99### Open your cloud firewall for HTTP and HTTPS
100
101One of the most common sources of misconfiguration is not opening firewall ports correctly. Please be sure to double check this step.
102
103In your cloud provider's console, the following ports should be open to inbound access from the public internet.
104
105* 80/tcp (Used only for TLS certification verification)
106* 443/tcp (Used for all application requests)
107
108**Note:** there is no need to set up TLS or redirect requests from port 80 to 443 because the Caddy web server, included in the Docker compose file, will handle this for you.
109
110### Configure DNS for your domain
111
112From your DNS provider's control panel, set up a domain with records pointing to your server.
113
114| Name | Type | Value | TTL |
115| --------------- | ---- | ------------- | --- |
116| `example.com` | `A` | `12.34.56.78` | 600 |
117| `*.example.com` | `A` | `12.34.56.78` | 600 |
118
119**Note:**
120* Replace `example.com` with your domain name.
121* Replace `12.34.56.78` with your server's IP address.
122* Some providers may use the `@` symbol to represent the root of your domain.
123* The wildcard record is required when allowing users to create new accounts on your PDS.
124* The TTL can be anything but 600 (10 minutes) is reasonable
125
126### Check that DNS is working as expected
127
128Use a service like [DNS Checker](https://dnschecker.org/) to verify that you can resolve domain names.
129
130Examples to check (record type `A`):
131* `example.com`
132* `random.example.com`
133* `test123.example.com`
134
135These should all return your server's public IP.
136
137### Automatic install on Ubuntu 20.04/22.04 or Debian 11/12
138
139On your server via ssh, run the installer script:
140
141```bash
142wget https://raw.githubusercontent.com/bluesky-social/pds/main/installer.sh
143```
144
145```bash
146sudo bash installer.sh
147```
148
149### Installing manually on Ubuntu 22.04
150
151#### Open ports on your Linux firewall
152
153If your server is running a Linux firewall managed with `ufw`, you will need to open these ports:
154
155```bash
156$ sudo ufw allow 80/tcp
157$ sudo ufw allow 443/tcp
158```
159
160#### Install Docker
161
162On your server, install Docker CE (Community Edition), using the the following instructions. For other operating systems you may reference the [official Docker install guides](https://docs.docker.com/engine/install/).
163
164**Note:** All of the following commands should be run on your server via ssh.
165
166##### Uninstall old versions
167
168```bash
169sudo apt-get remove docker docker-engine docker.io containerd runc
170```
171
172##### Set up the repository
173
174```bash
175sudo apt-get update
176sudo apt-get install \
177 ca-certificates \
178 curl \
179 gnupg
180```
181
182```bash
183sudo install -m 0755 -d /etc/apt/keyrings
184curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
185sudo chmod a+r /etc/apt/keyrings/docker.gpg
186```
187
188```bash
189echo \
190 "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
191 "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
192 sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
193```
194
195##### Install Docker Engine
196
197```bash
198sudo apt-get update
199```
200
201```bash
202sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
203```
204
205##### Verify Docker Engine installation
206
207```bash
208sudo docker run hello-world
209```
210
211#### Set up the PDS directory
212
213```bash
214sudo mkdir /pds
215sudo mkdir --parents /pds/caddy/data
216sudo mkdir --parents /pds/caddy/etc/caddy
217```
218
219#### Create the Caddyfile
220
221Be sure to replace `example.com` with your own domain.
222
223```bash
224cat <<CADDYFILE | sudo tee /pds/caddy/etc/caddy/Caddyfile
225{
226 email you@example.com
227 on_demand_tls {
228 ask http://localhost:3000
229 }
230}
231
232*.example.com, example.com {
233 tls {
234 on_demand
235 }
236 reverse_proxy http://localhost:3000
237}
238CADDYFILE
239```
240
241#### Create the PDS env configuration file
242
243You should fill in the first 5 values, but leave the rest untouched unless you have good reason to change it.
244
245See the PDS environment variables section at the end of this README for explanations of each value
246
247Your PDS will need two secp256k1 private keys provided as hex strings. You can securely generate these keys using `openssl` with the following command:
248
249**Note:**
250* Replace `example.com` with your domain name.
251
252```bash
253PDS_HOSTNAME="example.com"
254PDS_JWT_SECRET="$(openssl rand --hex 16)"
255PDS_ADMIN_PASSWORD="$(openssl rand --hex 16)"
256PDS_REPO_SIGNING_KEY_K256_PRIVATE_KEY_HEX="$(openssl ecparam --name secp256k1 --genkey --noout --outform DER | tail --bytes=+8 | head --bytes=32 | xxd --plain --cols 32)"
257PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX="$(openssl ecparam --name secp256k1 --genkey --noout --outform DER | tail --bytes=+8 | head --bytes=32 | xxd --plain --cols 32)"
258
259cat <<PDS_CONFIG | sudo tee /pds/pds.env
260PDS_HOSTNAME=${PDS_HOSTNAME}
261PDS_JWT_SECRET=${PDS_JWT_SECRET}
262PDS_ADMIN_PASSWORD=${PDS_ADMIN_PASSWORD}
263PDS_REPO_SIGNING_KEY_K256_PRIVATE_KEY_HEX=${PDS_REPO_SIGNING_KEY_K256_PRIVATE_KEY_HEX}
264PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX=${PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX}
265PDS_DB_SQLITE_LOCATION=/pds/pds.sqlite
266PDS_BLOBSTORE_DISK_LOCATION=/pds/blocks
267PDS_DID_PLC_URL=https://plc.bsky-sandbox.dev
268PDS_BSKY_APP_VIEW_URL=https://api.bsky-sandbox.dev
269PDS_BSKY_APP_VIEW_DID=did:web:api.bsky-sandbox.dev
270PDS_CRAWLERS=https://bgs.bsky-sandbox.dev
271PDS_CONFIG
272```
273
274#### Start the PDS containers
275
276##### Download the Docker compose file
277
278Download the `compose.yaml` to run your PDS, which includes the following containers:
279
280* `pds` Node PDS server running on http://localhost:3000
281* `caddy` HTTP reverse proxy handling TLS and proxying requests to the PDS server
282* `watchtower` Daemon responsible for auto-updating containers to keep the server secure and federating
283
284```bash
285curl https://raw.githubusercontent.com/bluesky-social/pds/main/compose.yaml | sudo tee /pds/compose.yaml
286```
287
288##### Create the systemd service
289
290```bash
291cat <<SYSTEMD_UNIT_FILE | sudo tee /etc/systemd/system/pds.service
292[Unit]
293Description=Bluesky PDS Service
294Documentation=https://github.com/bluesky-social/pds
295Requires=docker.service
296After=docker.service
297
298[Service]
299Type=oneshot
300RemainAfterExit=yes
301WorkingDirectory=/pds
302ExecStart=/usr/bin/docker compose --file /pds/compose.yaml up --detach
303ExecStop=/usr/bin/docker compose --file /pds/compose.yaml down
304
305[Install]
306WantedBy=default.target
307SYSTEMD_UNIT_FILE
308```
309
310##### Start the service
311
312**Reload the systemd daemon to create the new service:**
313```bash
314sudo systemctl daemon-reload
315```
316
317**Enable the systemd service:**
318```bash
319sudo systemctl enable pds
320```
321
322**Start the pds systemd service:**
323```bash
324sudo systemctl start pds
325```
326
327**Ensure that containers are running**
328
329There should be a caddy, pds, and watchtower container running.
330
331```bash
332sudo systemctl status pds
333```
334
335```bash
336sudo docker ps
337```
338
339### Verify your PDS is online
340
341You can check if your server is online and healthy by requesting the healthcheck endpoint.
342
343```bash
344curl https://example.com/xrpc/_health
345{"version":"0.2.2-beta.2"}
346```
347
348### Obtain your PDS admin password
349
350Your PDS admin password should be in your `pds.env` file if you used the installer script.
351
352**For example:**
353
354```bash
355$ source /pds/pds.env
356$ echo $PDS_ADMIN_PASSWORD
357a7b5970b6a5077bb41fc68a26d30adda
358```
359### Generate an invite code for your PDS
360
361By default, your PDS will require an invite code to create an account.
362
363You can generate a new invite code with the following command:
364
365```bash
366PDS_HOSTNAME="example.com"
367PDS_ADMIN_PASSWORD="<YOUR PDS ADMIN PASSWORD>"
368
369curl --silent \
370 --show-error \
371 --request POST \
372 --user "admin:${PDS_ADMIN_PASSWORD}" \
373 --header "Content-Type: application/json" \
374 --data '{"useCount": 1}' \
375 https://${PDS_HOSTNAME}/xrpc/com.atproto.server.createInviteCode
376```
377
378**Note:** the `useCount` field specifies how many times an invite code can be used
379
380### Connecting to your server
381
382You can use the Bluesky app to connect to your server to create an account.
383
3841. Get the Bluesky app
385 * [Bluesky for Web (sandbox)](https://app.bsky-sandbox.dev/)
386 * [Bluesky for iPhone](https://apps.apple.com/us/app/bluesky-social/id6444370199)
387 * [Bluesky for Android](https://play.google.com/store/apps/details?id=xyz.blueskyweb.app)
3881. Enter the URL of your PDS (e.g. `https://example.com/`)
3891. Create an account using the generated invite code
3901. Create a post
391
392_Note: because we use on-the-fly TLS certs, it may take 10-30s for your handle to be accessible. If you aren't seeing your first post/profile, wait 30s and try to make another post._
393
394Checkout [SANDBOX.md](./SANDBOX.md) for an overview of participating in the sandbox network.
395
396### Manually updating your PDS
397
398If you use use Docker `compose.yaml` file in this repo, your PDS will automatically update nightly. To manually update to the latest version use the following commands.
399
400**Pull the latest PDS container image:**
401```bash
402sudo docker pull ghcr.io/bluesky-social/pds:latest
403```
404
405**Restart PDS with the new container image:**
406```bash
407sudo systemctl restart pds
408```
409
410## PDS environment variables
411
412You will need to customize various settings configured through the PDS environment variables. See the below table to find the variables you'll need to set.
413
414| Environment Variable | Value | Should update? | Notes |
415| ----------------------------------------- | ---------------------------- | -------------- | --------------------------------------------------------------------------- |
416| PDS_HOSTNAME | example.com | ✅ | Public domain you intend to deploy your service at |
417| PDS_JWT_SECRET | jwt-secret | ✅ | Use a secure high-entropy string that is 32 characters in length |
418| PDS_ADMIN_PASSWORD | admin-pass | ✅ | Use a secure high-entropy string that is 32 characters in length |
419| PDS_REPO_SIGNING_KEY_K256_PRIVATE_KEY_HEX | 3ee68... | ✅ | See above Generate Keys section - once set, do not change |
420| PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX | e049f... | ✅ | See above Generate Keys section - once set, do not change |
421| PDS_DB_SQLITE_LOCATION | /pds/pds.sqlite | ❌ | Or use `PDS_DB_POSTGRES_URL` depending on which database you intend to use |
422| PDS_BLOBSTORE_DISK_LOCATION | /pds/blocks | ❌ | Only update if you update the mounted volume for your docker image as well |
423| PDS_DID_PLC_URL | https://plc.bsky-sandbox.dev | ❌ | Do not adjust if you intend to federate with the Bluesky federation sandbox |
424| PDS_BSKY_APP_VIEW_URL | https://api.bsky-sandbox.dev | ❌ | Do not adjust if you intend to federate with the Bluesky federation sandbox |
425| PDS_BSKY_APP_VIEW_DID | did:web:api.bsky-sandbox.dev | ❌ | Do not adjust if you intend to federate with the Bluesky federation sandbox |
426| PDS_CRAWLERS | https://bgs.bsky-sandbox.dev | ❌ | Do not adjust if you intend to federate with the Bluesky federation sandbox |
427
428There are additional environment variables that can be tweaked depending on how you're running your service. For instance, storing blobs in AWS S3, keys in AWS KMS, or setting up an email service.
429
430Feel free to explore those [Here](https://github.com/bluesky-social/atproto/blob/main/packages/pds/src/config/env.ts). However, we will not be providing support for more advanced configurations.