evan.jarrett.net
A container registry that uses the AT Protocol for manifest storage and S3 for blob storage.
atcr.io
docker
container
atproto
go
1# ATCR UpCloud Deployment Guide
2
3This guide walks you through deploying ATCR on UpCloud with Rocky Linux.
4
5## Architecture
6
7- **AppView** (atcr.io) - OCI registry API + web UI
8- **Hold Service** (hold01.atcr.io) - Presigned URL generator for blob storage
9- **Caddy** - Reverse proxy with automatic HTTPS
10- **UpCloud Object Storage** (blobs.atcr.io) - S3-compatible blob storage
11
12## Prerequisites
13
14### 1. UpCloud Account
15- Active UpCloud account
16- Object Storage enabled
17- Billing configured
18
19### 2. Domain Names
20You need three DNS records:
21- `atcr.io` (or your domain) - AppView
22- `hold01.atcr.io` - Hold service
23- `blobs.atcr.io` - S3 storage (CNAME)
24
25### 3. ATProto Account
26- Bluesky/ATProto account
27- Your DID (get from: `https://bsky.social/xrpc/com.atproto.identity.resolveHandle?handle=yourhandle.bsky.social`)
28
29### 4. UpCloud Object Storage Bucket
30Create an S3 bucket in UpCloud Object Storage:
311. Go to UpCloud Console → Storage → Object Storage
322. Create new bucket (e.g., `atcr-blobs`)
333. Note the region (e.g., `us-chi1`)
344. Generate access credentials (Access Key ID + Secret)
355. Note the endpoint (e.g., `s3.us-chi1.upcloudobjects.com`)
36
37## Deployment Steps
38
39### Step 1: Configure DNS
40
41Set up DNS records (using Cloudflare or your DNS provider):
42
43```
44Type Name Value Proxy
45────────────────────────────────────────────────────────────────────────────
46A atcr.io [your-upcloud-ip] ☁️ DISABLED
47A hold01.atcr.io [your-upcloud-ip] ☁️ DISABLED
48CNAME blobs.atcr.io atcr-blobs.us-chi1.upcloudobjects.com ☁️ DISABLED
49```
50
51**IMPORTANT:**
52- **DISABLE Cloudflare proxy** (gray cloud, not orange) for all three domains
53- Proxied connections break Docker registry protocol and presigned URLs
54- You'll still get HTTPS via Caddy's Let's Encrypt integration
55
56Wait for DNS propagation (5-30 minutes). Verify with:
57```bash
58dig atcr.io
59dig hold01.atcr.io
60dig blobs.atcr.io
61```
62
63### Step 2: Create UpCloud Server
64
651. Go to UpCloud Console → Servers → Deploy a new server
662. Select location (match your S3 region if possible)
673. Select **Rocky Linux 9** operating system
684. Choose plan (minimum: 2 GB RAM, 1 CPU)
695. Configure hostname: `atcr`
706. Enable IPv4 public networking
717. **Optional:** Enable IPv6
728. **User data:** Paste contents of `deploy/init-upcloud.sh`
73 - Update `ATCR_REPO` variable with your git repository URL
74 - Or leave empty and manually copy files later
759. Create SSH key or use password authentication
7610. Click **Deploy**
77
78### Step 3: Wait for Initialization
79
80The init script will:
81- Update system packages (~2-5 minutes)
82- Install Docker and Docker Compose
83- Configure firewall
84- Clone repository (if ATCR_REPO configured)
85- Create systemd service
86- Create helper scripts
87
88Monitor progress:
89```bash
90# SSH into server
91ssh root@[your-upcloud-ip]
92
93# Check cloud-init logs
94tail -f /var/log/cloud-init-output.log
95```
96
97Wait for the completion message in the logs.
98
99### Step 4: Configure Environment
100
101Edit the environment configuration:
102
103```bash
104# SSH into server
105ssh root@[your-upcloud-ip]
106
107# Edit environment file
108cd /opt/atcr
109nano .env
110```
111
112**Required configuration:**
113
114```bash
115# Domains
116APPVIEW_DOMAIN=atcr.io
117HOLD_DOMAIN=hold01.atcr.io
118
119# Your ATProto DID
120HOLD_OWNER=did:plc:your-did-here
121
122# UpCloud S3 credentials
123AWS_ACCESS_KEY_ID=your-access-key-id
124AWS_SECRET_ACCESS_KEY=your-secret-access-key
125AWS_REGION=us-chi1
126S3_BUCKET=atcr-blobs
127
128# S3 endpoint (choose one):
129# Option 1: Custom domain (recommended)
130S3_ENDPOINT=https://blobs.atcr.io
131# Option 2: Direct UpCloud endpoint
132# S3_ENDPOINT=https://s3.us-chi1.upcloudobjects.com
133
134# Public access (optional)
135HOLD_PUBLIC=false # Set to true to allow anonymous pulls
136```
137
138Save and exit (Ctrl+X, Y, Enter).
139
140### Step 5: Start ATCR
141
142```bash
143# Start services
144systemctl start atcr
145
146# Check status
147systemctl status atcr
148
149# Verify containers are running
150docker ps
151```
152
153You should see three containers:
154- `atcr-caddy`
155- `atcr-appview`
156- `atcr-hold`
157
158### Step 6: Complete Hold OAuth Registration
159
160The hold service needs to register itself with your PDS:
161
162```bash
163# Get OAuth URL from logs
164/opt/atcr/get-hold-oauth.sh
165```
166
167Look for output like:
168```
169Visit this URL to authorize: https://bsky.social/oauth/authorize?...
170```
171
1721. Copy the URL and open in your browser
1732. Log in with your ATProto account
1743. Authorize the hold service
1754. Return to terminal
176
177The hold service will create records in your PDS:
178- `io.atcr.hold` - Hold definition
179- `io.atcr.hold.crew` - Your membership as captain
180
181Verify registration:
182```bash
183docker logs atcr-hold | grep -i "success\|registered\|created"
184```
185
186### Step 7: Test the Registry
187
188#### Test 1: Check endpoints
189
190```bash
191# AppView (should return {})
192curl https://atcr.io/v2/
193
194# Hold service (should return {"status":"ok"})
195curl https://hold01.atcr.io/health
196```
197
198#### Test 2: Configure Docker client
199
200On your local machine:
201
202```bash
203# Install credential helper
204# (Build from source or download release)
205go install atcr.io/cmd/docker-credential-atcr@latest
206
207# Configure Docker to use the credential helper
208# Add to ~/.docker/config.json:
209{
210 "credHelpers": {
211 "atcr.io": "atcr"
212 }
213}
214```
215
216#### Test 3: Push a test image
217
218```bash
219# Tag an image
220docker tag alpine:latest atcr.io/yourhandle/test:latest
221
222# Push to ATCR
223docker push atcr.io/yourhandle/test:latest
224
225# Pull from ATCR
226docker pull atcr.io/yourhandle/test:latest
227```
228
229### Step 8: Monitor and Maintain
230
231#### View logs
232
233```bash
234# All services
235/opt/atcr/logs.sh
236
237# Specific service
238/opt/atcr/logs.sh atcr-appview
239/opt/atcr/logs.sh atcr-hold
240/opt/atcr/logs.sh atcr-caddy
241
242# Or use docker directly
243docker logs -f atcr-appview
244```
245
246#### Restart services
247
248```bash
249# Restart all
250systemctl restart atcr
251
252# Or use docker-compose
253cd /opt/atcr
254docker compose -f deploy/docker-compose.prod.yml restart
255```
256
257#### Rebuild after code changes
258
259```bash
260/opt/atcr/rebuild.sh
261```
262
263#### Update configuration
264
265```bash
266# Edit environment
267nano /opt/atcr/.env
268
269# Restart services
270systemctl restart atcr
271```
272
273## Architecture Details
274
275### Service Communication
276
277```
278Internet
279 ↓
280Caddy (443) ───────────┐
281 ├─→ atcr-appview:5000 (Registry API + Web UI)
282 └─→ atcr-hold:8080 (Presigned URL generator)
283 ↓
284 UpCloud S3 (blobs.atcr.io)
285```
286
287### Data Flow: Push
288
289```
2901. docker push atcr.io/user/image:tag
2912. AppView ← Docker client (manifest + blob metadata)
2923. AppView → ATProto PDS (store manifest record)
2934. Hold ← Docker client (request presigned URL)
2945. Hold → UpCloud S3 API (generate presigned URL)
2956. Hold → Docker client (return presigned URL)
2967. UpCloud S3 ← Docker client (upload blob directly)
297```
298
299### Data Flow: Pull
300
301```
3021. docker pull atcr.io/user/image:tag
3032. AppView ← Docker client (get manifest)
3043. AppView → ATProto PDS (fetch manifest record)
3054. AppView → Docker client (return manifest with holdEndpoint)
3065. Hold ← Docker client (request presigned URL)
3076. Hold → UpCloud S3 API (generate presigned URL)
3087. Hold → Docker client (return presigned URL)
3098. UpCloud S3 ← Docker client (download blob directly)
310```
311
312**Key insight:** The hold service only generates presigned URLs. Actual data transfer happens directly between Docker clients and S3, minimizing bandwidth costs.
313
314## Troubleshooting
315
316### Issue: "Cannot connect to registry"
317
318**Check DNS:**
319```bash
320dig atcr.io
321dig hold01.atcr.io
322```
323
324**Check Caddy logs:**
325```bash
326docker logs atcr-caddy
327```
328
329**Check firewall:**
330```bash
331firewall-cmd --list-all
332```
333
334### Issue: "Certificate errors"
335
336**Verify DNS is propagated:**
337```bash
338curl -I https://atcr.io
339```
340
341**Check Caddy is obtaining certificates:**
342```bash
343docker logs atcr-caddy | grep -i certificate
344```
345
346**Common causes:**
347- DNS not propagated (wait 30 minutes)
348- Cloudflare proxy enabled (must be disabled)
349- Port 80/443 blocked by firewall
350
351### Issue: "Presigned URLs fail"
352
353**Check S3 endpoint configuration:**
354```bash
355docker exec atcr-hold env | grep S3
356```
357
358**Verify custom domain CNAME:**
359```bash
360dig blobs.atcr.io CNAME
361```
362
363**Test S3 connectivity:**
364```bash
365docker exec atcr-hold wget -O- https://blobs.atcr.io/
366```
367
368**Common causes:**
369- Cloudflare proxy enabled on blobs.atcr.io
370- S3_ENDPOINT misconfigured
371- AWS credentials invalid
372
373### Issue: "Hold registration fails"
374
375**Check hold owner DID:**
376```bash
377docker exec atcr-hold env | grep HOLD_OWNER
378```
379
380**Verify OAuth flow:**
381```bash
382/opt/atcr/get-hold-oauth.sh
383```
384
385**Manual registration:**
386```bash
387# Get fresh OAuth URL
388docker restart atcr-hold
389docker logs -f atcr-hold
390```
391
392### Issue: "High bandwidth usage"
393
394Presigned URLs should eliminate hold bandwidth. If seeing high usage:
395
396**Verify presigned URLs are enabled:**
397```bash
398docker logs atcr-hold | grep -i presigned
399```
400
401**Check S3 driver:**
402```bash
403docker exec atcr-hold env | grep STORAGE_DRIVER
404# Should be: s3 (not filesystem)
405```
406
407**Verify direct S3 access:**
408```bash
409# Push should show 307 redirects in logs
410docker logs -f atcr-hold
411# Then push an image
412```
413
414### Automatic Updates
415
416```bash
417# Install automatic updates
418dnf install -y dnf-automatic
419
420# Enable timer
421systemctl enable --now dnf-automatic.timer
422```
423
424### Monitoring
425
426```bash
427# Install monitoring tools
428dnf install -y htop iotop nethogs
429
430# Monitor resources
431htop
432
433# Monitor Docker
434docker stats
435```
436
437### Backups
438
439Critical data to backup:
440- `/opt/atcr/.env` - Configuration
441- Docker volumes:
442 - `atcr-appview-data` - Auth keys, UI database, OAuth tokens
443 - `caddy_data` - TLS certificates
444
445```bash
446# Backup volumes
447docker run --rm \
448 -v atcr-appview-data:/data \
449 -v /backup:/backup \
450 alpine tar czf /backup/atcr-appview-data.tar.gz /data
451```
452
453## Scaling Considerations
454
455### Single Server (Current Setup)
456- Suitable for: 100-1000 users
457- Bottleneck: AppView CPU (manifest queries)
458- Storage: Unlimited (S3)
459
460### Multi-Server (Future)
461- Multiple AppView instances behind load balancer
462- Shared Redis for hold cache (replace in-memory cache)
463- PostgreSQL for UI database (replace SQLite)
464- Multiple hold services (geo-distributed)
465
466## Support
467
468- Documentation: https://tangled.org/@evan.jarrett.net/at-container-registry
469- Issues: https://tangled.org/@evan.jarrett.net/at-container-registry/issues
470- Bluesky: @evan.jarrett.net