TODO _headers file place.wisp.settings lexicon as a lexiconal way of configuring this
+1
-31
README.md
+1
-31
README.md
···
50
cargo build
51
```
52
53
-
## Features
54
-
55
-
### URL Redirects and Rewrites
56
-
57
-
The hosting service supports Netlify-style `_redirects` files for managing URLs. Place a `_redirects` file in your site root to enable:
58
-
59
-
- **301/302 Redirects**: Permanent and temporary URL redirects
60
-
- **200 Rewrites**: Serve different content without changing the URL
61
-
- **404 Custom Pages**: Custom error pages for specific paths
62
-
- **Splats & Placeholders**: Dynamic path matching (`/blog/:year/:month/:day`, `/news/*`)
63
-
- **Query Parameter Matching**: Redirect based on URL parameters
64
-
- **Conditional Redirects**: Route by country, language, or cookie presence
65
-
- **Force Redirects**: Override existing files with redirects
66
-
67
-
Example `_redirects`:
68
-
```
69
-
# Single-page app routing (React, Vue, etc.)
70
-
/* /index.html 200
71
-
72
-
# Simple redirects
73
-
/home /
74
-
/old-blog/* /blog/:splat
75
-
76
-
# API proxy
77
-
/api/* https://api.example.com/:splat 200
78
-
79
-
# Country-based routing
80
-
/ /us/ 302 Country=us
81
-
/ /uk/ 302 Country=gb
82
-
```
83
-
84
## Limits
85
86
- Max file size: 100MB (PDS limit)
87
- Max files: 2000
88
89
## Tech Stack
+123
hosting-service/EXAMPLE.md
+123
hosting-service/EXAMPLE.md
···
···
1
+
# HTML Path Rewriting Example
2
+
3
+
This document demonstrates how HTML path rewriting works when serving sites via the `/s/:identifier/:site/*` route.
4
+
5
+
## Problem
6
+
7
+
When you create a static site with absolute paths like `/style.css` or `/images/logo.png`, these paths work fine when served from the root domain. However, when served from a subdirectory like `/s/alice.bsky.social/mysite/`, these absolute paths break because they resolve to the server root instead of the site root.
8
+
9
+
## Solution
10
+
11
+
The hosting service automatically rewrites absolute paths in HTML files to work correctly in the subdirectory context.
12
+
13
+
## Example
14
+
15
+
**Original HTML file (index.html):**
16
+
```html
17
+
<!DOCTYPE html>
18
+
<html>
19
+
<head>
20
+
<meta charset="UTF-8">
21
+
<title>My Site</title>
22
+
<link rel="stylesheet" href="/style.css">
23
+
<link rel="icon" href="/favicon.ico">
24
+
<script src="/app.js"></script>
25
+
</head>
26
+
<body>
27
+
<header>
28
+
<img src="/images/logo.png" alt="Logo">
29
+
<nav>
30
+
<a href="/">Home</a>
31
+
<a href="/about">About</a>
32
+
<a href="/contact">Contact</a>
33
+
</nav>
34
+
</header>
35
+
36
+
<main>
37
+
<h1>Welcome</h1>
38
+
<img src="/images/hero.jpg"
39
+
srcset="/images/hero.jpg 1x, /images/hero@2x.jpg 2x"
40
+
alt="Hero">
41
+
42
+
<form action="/submit" method="post">
43
+
<input type="text" name="email">
44
+
<button>Submit</button>
45
+
</form>
46
+
</main>
47
+
48
+
<footer>
49
+
<a href="https://example.com">External Link</a>
50
+
<a href="#top">Back to Top</a>
51
+
</footer>
52
+
</body>
53
+
</html>
54
+
```
55
+
56
+
**When accessed via `/s/alice.bsky.social/mysite/`, the HTML is rewritten to:**
57
+
```html
58
+
<!DOCTYPE html>
59
+
<html>
60
+
<head>
61
+
<meta charset="UTF-8">
62
+
<title>My Site</title>
63
+
<link rel="stylesheet" href="/s/alice.bsky.social/mysite/style.css">
64
+
<link rel="icon" href="/s/alice.bsky.social/mysite/favicon.ico">
65
+
<script src="/s/alice.bsky.social/mysite/app.js"></script>
66
+
</head>
67
+
<body>
68
+
<header>
69
+
<img src="/s/alice.bsky.social/mysite/images/logo.png" alt="Logo">
70
+
<nav>
71
+
<a href="/s/alice.bsky.social/mysite/">Home</a>
72
+
<a href="/s/alice.bsky.social/mysite/about">About</a>
73
+
<a href="/s/alice.bsky.social/mysite/contact">Contact</a>
74
+
</nav>
75
+
</header>
76
+
77
+
<main>
78
+
<h1>Welcome</h1>
79
+
<img src="/s/alice.bsky.social/mysite/images/hero.jpg"
80
+
srcset="/s/alice.bsky.social/mysite/images/hero.jpg 1x, /s/alice.bsky.social/mysite/images/hero@2x.jpg 2x"
81
+
alt="Hero">
82
+
83
+
<form action="/s/alice.bsky.social/mysite/submit" method="post">
84
+
<input type="text" name="email">
85
+
<button>Submit</button>
86
+
</form>
87
+
</main>
88
+
89
+
<footer>
90
+
<a href="https://example.com">External Link</a>
91
+
<a href="#top">Back to Top</a>
92
+
</footer>
93
+
</body>
94
+
</html>
95
+
```
96
+
97
+
## What's Preserved
98
+
99
+
Notice that:
100
+
- ✅ Absolute paths are rewritten: `/style.css` → `/s/alice.bsky.social/mysite/style.css`
101
+
- ✅ External URLs are preserved: `https://example.com` stays the same
102
+
- ✅ Anchors are preserved: `#top` stays the same
103
+
- ✅ The rewriting is safe and won't break your site
104
+
105
+
## Supported Attributes
106
+
107
+
The rewriter handles these HTML attributes:
108
+
- `src` - images, scripts, iframes, videos, audio
109
+
- `href` - links, stylesheets
110
+
- `action` - forms
111
+
- `data` - objects
112
+
- `poster` - video posters
113
+
- `srcset` - responsive images
114
+
115
+
## Testing Your Site
116
+
117
+
To test if your site works with path rewriting:
118
+
119
+
1. Upload your site to your PDS as a `place.wisp.fs` record
120
+
2. Access it via: `https://hosting.wisp.place/s/YOUR_HANDLE/SITE_NAME/`
121
+
3. Check that all resources load correctly
122
+
123
+
If you're using relative paths already (like `./style.css` or `../images/logo.png`), they'll work without any rewriting.
-134
hosting-service/example-_redirects
-134
hosting-service/example-_redirects
···
1
-
# Example _redirects file for Wisp hosting
2
-
# Place this file in the root directory of your site as "_redirects"
3
-
# Lines starting with # are comments
4
-
5
-
# ===================================
6
-
# SIMPLE REDIRECTS
7
-
# ===================================
8
-
9
-
# Redirect home page
10
-
# /home /
11
-
12
-
# Redirect old URLs to new ones
13
-
# /old-blog /blog
14
-
# /about-us /about
15
-
16
-
# ===================================
17
-
# SPLAT REDIRECTS (WILDCARDS)
18
-
# ===================================
19
-
20
-
# Redirect entire directories
21
-
# /news/* /blog/:splat
22
-
# /old-site/* /new-site/:splat
23
-
24
-
# ===================================
25
-
# PLACEHOLDER REDIRECTS
26
-
# ===================================
27
-
28
-
# Restructure blog URLs
29
-
# /blog/:year/:month/:day/:slug /posts/:year-:month-:day/:slug
30
-
31
-
# Capture multiple parameters
32
-
# /products/:category/:id /shop/:category/item/:id
33
-
34
-
# ===================================
35
-
# STATUS CODES
36
-
# ===================================
37
-
38
-
# Permanent redirect (301) - default if not specified
39
-
# /permanent-move /new-location 301
40
-
41
-
# Temporary redirect (302)
42
-
# /temp-redirect /temp-location 302
43
-
44
-
# Rewrite (200) - serves different content, URL stays the same
45
-
# /api/* /functions/:splat 200
46
-
47
-
# Custom 404 page
48
-
# /shop/* /shop-closed.html 404
49
-
50
-
# ===================================
51
-
# FORCE REDIRECTS
52
-
# ===================================
53
-
54
-
# Force redirect even if file exists (note the ! after status code)
55
-
# /override-file /other-file.html 200!
56
-
57
-
# ===================================
58
-
# CONDITIONAL REDIRECTS
59
-
# ===================================
60
-
61
-
# Country-based redirects (ISO 3166-1 alpha-2 codes)
62
-
# / /us/ 302 Country=us
63
-
# / /uk/ 302 Country=gb
64
-
# / /anz/ 302 Country=au,nz
65
-
66
-
# Language-based redirects
67
-
# /products /en/products 301 Language=en
68
-
# /products /de/products 301 Language=de
69
-
# /products /fr/products 301 Language=fr
70
-
71
-
# Cookie-based redirects (checks if cookie exists)
72
-
# /* /legacy/:splat 200 Cookie=is_legacy
73
-
74
-
# ===================================
75
-
# QUERY PARAMETERS
76
-
# ===================================
77
-
78
-
# Match specific query parameters
79
-
# /store id=:id /blog/:id 301
80
-
81
-
# Multiple parameters
82
-
# /search q=:query category=:cat /find/:cat/:query 301
83
-
84
-
# ===================================
85
-
# DOMAIN-LEVEL REDIRECTS
86
-
# ===================================
87
-
88
-
# Redirect to different domain (must include protocol)
89
-
# /external https://example.com/path
90
-
91
-
# Redirect entire subdomain
92
-
# http://blog.example.com/* https://example.com/blog/:splat 301!
93
-
# https://blog.example.com/* https://example.com/blog/:splat 301!
94
-
95
-
# ===================================
96
-
# COMMON PATTERNS
97
-
# ===================================
98
-
99
-
# Remove .html extensions
100
-
# /page.html /page
101
-
102
-
# Add trailing slash
103
-
# /about /about/
104
-
105
-
# Single-page app fallback (serve index.html for all paths)
106
-
# /* /index.html 200
107
-
108
-
# API proxy
109
-
# /api/* https://api.example.com/:splat 200
110
-
111
-
# ===================================
112
-
# CUSTOM ERROR PAGES
113
-
# ===================================
114
-
115
-
# Language-specific 404 pages
116
-
# /en/* /en/404.html 404
117
-
# /de/* /de/404.html 404
118
-
119
-
# Section-specific 404 pages
120
-
# /shop/* /shop/not-found.html 404
121
-
# /blog/* /blog/404.html 404
122
-
123
-
# ===================================
124
-
# NOTES
125
-
# ===================================
126
-
#
127
-
# - Rules are processed in order (first match wins)
128
-
# - More specific rules should come before general ones
129
-
# - Splats (*) can only be used at the end of a path
130
-
# - Query parameters are automatically preserved for 200, 301, 302
131
-
# - Trailing slashes are normalized (/ and no / are treated the same)
132
-
# - Default status code is 301 if not specified
133
-
#
134
-
···
-215
hosting-service/src/lib/redirects.test.ts
-215
hosting-service/src/lib/redirects.test.ts
···
1
-
import { describe, it, expect } from 'bun:test'
2
-
import { parseRedirectsFile, matchRedirectRule } from './redirects';
3
-
4
-
describe('parseRedirectsFile', () => {
5
-
it('should parse simple redirects', () => {
6
-
const content = `
7
-
# Comment line
8
-
/old-path /new-path
9
-
/home / 301
10
-
`;
11
-
const rules = parseRedirectsFile(content);
12
-
expect(rules).toHaveLength(2);
13
-
expect(rules[0]).toMatchObject({
14
-
from: '/old-path',
15
-
to: '/new-path',
16
-
status: 301,
17
-
force: false,
18
-
});
19
-
expect(rules[1]).toMatchObject({
20
-
from: '/home',
21
-
to: '/',
22
-
status: 301,
23
-
force: false,
24
-
});
25
-
});
26
-
27
-
it('should parse redirects with different status codes', () => {
28
-
const content = `
29
-
/temp-redirect /target 302
30
-
/rewrite /content 200
31
-
/not-found /404 404
32
-
`;
33
-
const rules = parseRedirectsFile(content);
34
-
expect(rules).toHaveLength(3);
35
-
expect(rules[0]?.status).toBe(302);
36
-
expect(rules[1]?.status).toBe(200);
37
-
expect(rules[2]?.status).toBe(404);
38
-
});
39
-
40
-
it('should parse force redirects', () => {
41
-
const content = `/force-path /target 301!`;
42
-
const rules = parseRedirectsFile(content);
43
-
expect(rules[0]?.force).toBe(true);
44
-
expect(rules[0]?.status).toBe(301);
45
-
});
46
-
47
-
it('should parse splat redirects', () => {
48
-
const content = `/news/* /blog/:splat`;
49
-
const rules = parseRedirectsFile(content);
50
-
expect(rules[0]?.from).toBe('/news/*');
51
-
expect(rules[0]?.to).toBe('/blog/:splat');
52
-
});
53
-
54
-
it('should parse placeholder redirects', () => {
55
-
const content = `/blog/:year/:month/:day /posts/:year-:month-:day`;
56
-
const rules = parseRedirectsFile(content);
57
-
expect(rules[0]?.from).toBe('/blog/:year/:month/:day');
58
-
expect(rules[0]?.to).toBe('/posts/:year-:month-:day');
59
-
});
60
-
61
-
it('should parse country-based redirects', () => {
62
-
const content = `/ /anz 302 Country=au,nz`;
63
-
const rules = parseRedirectsFile(content);
64
-
expect(rules[0]?.conditions?.country).toEqual(['au', 'nz']);
65
-
});
66
-
67
-
it('should parse language-based redirects', () => {
68
-
const content = `/products /en/products 301 Language=en`;
69
-
const rules = parseRedirectsFile(content);
70
-
expect(rules[0]?.conditions?.language).toEqual(['en']);
71
-
});
72
-
73
-
it('should parse cookie-based redirects', () => {
74
-
const content = `/* /legacy/:splat 200 Cookie=is_legacy,my_cookie`;
75
-
const rules = parseRedirectsFile(content);
76
-
expect(rules[0]?.conditions?.cookie).toEqual(['is_legacy', 'my_cookie']);
77
-
});
78
-
});
79
-
80
-
describe('matchRedirectRule', () => {
81
-
it('should match exact paths', () => {
82
-
const rules = parseRedirectsFile('/old-path /new-path');
83
-
const match = matchRedirectRule('/old-path', rules);
84
-
expect(match).toBeTruthy();
85
-
expect(match?.targetPath).toBe('/new-path');
86
-
expect(match?.status).toBe(301);
87
-
});
88
-
89
-
it('should match paths with trailing slash', () => {
90
-
const rules = parseRedirectsFile('/old-path /new-path');
91
-
const match = matchRedirectRule('/old-path/', rules);
92
-
expect(match).toBeTruthy();
93
-
expect(match?.targetPath).toBe('/new-path');
94
-
});
95
-
96
-
it('should match splat patterns', () => {
97
-
const rules = parseRedirectsFile('/news/* /blog/:splat');
98
-
const match = matchRedirectRule('/news/2024/01/15/my-post', rules);
99
-
expect(match).toBeTruthy();
100
-
expect(match?.targetPath).toBe('/blog/2024/01/15/my-post');
101
-
});
102
-
103
-
it('should match placeholder patterns', () => {
104
-
const rules = parseRedirectsFile('/blog/:year/:month/:day /posts/:year-:month-:day');
105
-
const match = matchRedirectRule('/blog/2024/01/15', rules);
106
-
expect(match).toBeTruthy();
107
-
expect(match?.targetPath).toBe('/posts/2024-01-15');
108
-
});
109
-
110
-
it('should preserve query strings for 301/302 redirects', () => {
111
-
const rules = parseRedirectsFile('/old /new 301');
112
-
const match = matchRedirectRule('/old', rules, {
113
-
queryParams: { foo: 'bar', baz: 'qux' },
114
-
});
115
-
expect(match?.targetPath).toContain('?');
116
-
expect(match?.targetPath).toContain('foo=bar');
117
-
expect(match?.targetPath).toContain('baz=qux');
118
-
});
119
-
120
-
it('should match based on query parameters', () => {
121
-
const rules = parseRedirectsFile('/store id=:id /blog/:id 301');
122
-
const match = matchRedirectRule('/store', rules, {
123
-
queryParams: { id: 'my-post' },
124
-
});
125
-
expect(match).toBeTruthy();
126
-
expect(match?.targetPath).toContain('/blog/my-post');
127
-
});
128
-
129
-
it('should not match when query params are missing', () => {
130
-
const rules = parseRedirectsFile('/store id=:id /blog/:id 301');
131
-
const match = matchRedirectRule('/store', rules, {
132
-
queryParams: {},
133
-
});
134
-
expect(match).toBeNull();
135
-
});
136
-
137
-
it('should match based on country header', () => {
138
-
const rules = parseRedirectsFile('/ /aus 302 Country=au');
139
-
const match = matchRedirectRule('/', rules, {
140
-
headers: { 'cf-ipcountry': 'AU' },
141
-
});
142
-
expect(match).toBeTruthy();
143
-
expect(match?.targetPath).toBe('/aus');
144
-
});
145
-
146
-
it('should not match wrong country', () => {
147
-
const rules = parseRedirectsFile('/ /aus 302 Country=au');
148
-
const match = matchRedirectRule('/', rules, {
149
-
headers: { 'cf-ipcountry': 'US' },
150
-
});
151
-
expect(match).toBeNull();
152
-
});
153
-
154
-
it('should match based on language header', () => {
155
-
const rules = parseRedirectsFile('/products /en/products 301 Language=en');
156
-
const match = matchRedirectRule('/products', rules, {
157
-
headers: { 'accept-language': 'en-US,en;q=0.9' },
158
-
});
159
-
expect(match).toBeTruthy();
160
-
expect(match?.targetPath).toBe('/en/products');
161
-
});
162
-
163
-
it('should match based on cookie presence', () => {
164
-
const rules = parseRedirectsFile('/* /legacy/:splat 200 Cookie=is_legacy');
165
-
const match = matchRedirectRule('/some-path', rules, {
166
-
cookies: { is_legacy: 'true' },
167
-
});
168
-
expect(match).toBeTruthy();
169
-
expect(match?.targetPath).toBe('/legacy/some-path');
170
-
});
171
-
172
-
it('should return first matching rule', () => {
173
-
const content = `
174
-
/path /first
175
-
/path /second
176
-
`;
177
-
const rules = parseRedirectsFile(content);
178
-
const match = matchRedirectRule('/path', rules);
179
-
expect(match?.targetPath).toBe('/first');
180
-
});
181
-
182
-
it('should match more specific rules before general ones', () => {
183
-
const content = `
184
-
/jobs/customer-ninja /careers/support
185
-
/jobs/* /careers/:splat
186
-
`;
187
-
const rules = parseRedirectsFile(content);
188
-
189
-
const match1 = matchRedirectRule('/jobs/customer-ninja', rules);
190
-
expect(match1?.targetPath).toBe('/careers/support');
191
-
192
-
const match2 = matchRedirectRule('/jobs/developer', rules);
193
-
expect(match2?.targetPath).toBe('/careers/developer');
194
-
});
195
-
196
-
it('should handle SPA routing pattern', () => {
197
-
const rules = parseRedirectsFile('/* /index.html 200');
198
-
199
-
// Should match any path
200
-
const match1 = matchRedirectRule('/about', rules);
201
-
expect(match1).toBeTruthy();
202
-
expect(match1?.targetPath).toBe('/index.html');
203
-
expect(match1?.status).toBe(200);
204
-
205
-
const match2 = matchRedirectRule('/users/123/profile', rules);
206
-
expect(match2).toBeTruthy();
207
-
expect(match2?.targetPath).toBe('/index.html');
208
-
expect(match2?.status).toBe(200);
209
-
210
-
const match3 = matchRedirectRule('/', rules);
211
-
expect(match3).toBeTruthy();
212
-
expect(match3?.targetPath).toBe('/index.html');
213
-
});
214
-
});
215
-
···
-413
hosting-service/src/lib/redirects.ts
-413
hosting-service/src/lib/redirects.ts
···
1
-
import { readFile } from 'fs/promises';
2
-
import { existsSync } from 'fs';
3
-
4
-
export interface RedirectRule {
5
-
from: string;
6
-
to: string;
7
-
status: number;
8
-
force: boolean;
9
-
conditions?: {
10
-
country?: string[];
11
-
language?: string[];
12
-
role?: string[];
13
-
cookie?: string[];
14
-
};
15
-
// For pattern matching
16
-
fromPattern?: RegExp;
17
-
fromParams?: string[]; // Named parameters from the pattern
18
-
queryParams?: Record<string, string>; // Expected query parameters
19
-
}
20
-
21
-
export interface RedirectMatch {
22
-
rule: RedirectRule;
23
-
targetPath: string;
24
-
status: number;
25
-
}
26
-
27
-
/**
28
-
* Parse a _redirects file into an array of redirect rules
29
-
*/
30
-
export function parseRedirectsFile(content: string): RedirectRule[] {
31
-
const lines = content.split('\n');
32
-
const rules: RedirectRule[] = [];
33
-
34
-
for (let lineNum = 0; lineNum < lines.length; lineNum++) {
35
-
const lineRaw = lines[lineNum];
36
-
if (!lineRaw) continue;
37
-
38
-
const line = lineRaw.trim();
39
-
40
-
// Skip empty lines and comments
41
-
if (!line || line.startsWith('#')) {
42
-
continue;
43
-
}
44
-
45
-
try {
46
-
const rule = parseRedirectLine(line);
47
-
if (rule && rule.fromPattern) {
48
-
rules.push(rule);
49
-
}
50
-
} catch (err) {
51
-
console.warn(`Failed to parse redirect rule on line ${lineNum + 1}: ${line}`, err);
52
-
}
53
-
}
54
-
55
-
return rules;
56
-
}
57
-
58
-
/**
59
-
* Parse a single redirect rule line
60
-
* Format: /from [query_params] /to [status] [conditions]
61
-
*/
62
-
function parseRedirectLine(line: string): RedirectRule | null {
63
-
// Split by whitespace, but respect quoted strings (though not commonly used)
64
-
const parts = line.split(/\s+/);
65
-
66
-
if (parts.length < 2) {
67
-
return null;
68
-
}
69
-
70
-
let idx = 0;
71
-
const from = parts[idx++];
72
-
73
-
if (!from) {
74
-
return null;
75
-
}
76
-
77
-
let status = 301; // Default status
78
-
let force = false;
79
-
const conditions: NonNullable<RedirectRule['conditions']> = {};
80
-
const queryParams: Record<string, string> = {};
81
-
82
-
// Parse query parameters that come before the destination path
83
-
// They look like: key=:value (and don't start with /)
84
-
while (idx < parts.length) {
85
-
const part = parts[idx];
86
-
if (!part) {
87
-
idx++;
88
-
continue;
89
-
}
90
-
91
-
// If it starts with / or http, it's the destination path
92
-
if (part.startsWith('/') || part.startsWith('http://') || part.startsWith('https://')) {
93
-
break;
94
-
}
95
-
96
-
// If it contains = and comes before the destination, it's a query param
97
-
if (part.includes('=')) {
98
-
const splitIndex = part.indexOf('=');
99
-
const key = part.slice(0, splitIndex);
100
-
const value = part.slice(splitIndex + 1);
101
-
102
-
if (key && value) {
103
-
queryParams[key] = value;
104
-
}
105
-
idx++;
106
-
} else {
107
-
// Not a query param, must be destination or something else
108
-
break;
109
-
}
110
-
}
111
-
112
-
// Next part should be the destination
113
-
if (idx >= parts.length) {
114
-
return null;
115
-
}
116
-
117
-
const to = parts[idx++];
118
-
if (!to) {
119
-
return null;
120
-
}
121
-
122
-
// Parse remaining parts for status code and conditions
123
-
for (let i = idx; i < parts.length; i++) {
124
-
const part = parts[i];
125
-
126
-
if (!part) continue;
127
-
128
-
// Check for status code (with optional ! for force)
129
-
if (/^\d+!?$/.test(part)) {
130
-
if (part.endsWith('!')) {
131
-
force = true;
132
-
status = parseInt(part.slice(0, -1));
133
-
} else {
134
-
status = parseInt(part);
135
-
}
136
-
continue;
137
-
}
138
-
139
-
// Check for condition parameters (Country=, Language=, Role=, Cookie=)
140
-
if (part.includes('=')) {
141
-
const splitIndex = part.indexOf('=');
142
-
const key = part.slice(0, splitIndex);
143
-
const value = part.slice(splitIndex + 1);
144
-
145
-
if (!key || !value) continue;
146
-
147
-
const keyLower = key.toLowerCase();
148
-
149
-
if (keyLower === 'country') {
150
-
conditions.country = value.split(',').map(v => v.trim().toLowerCase());
151
-
} else if (keyLower === 'language') {
152
-
conditions.language = value.split(',').map(v => v.trim().toLowerCase());
153
-
} else if (keyLower === 'role') {
154
-
conditions.role = value.split(',').map(v => v.trim());
155
-
} else if (keyLower === 'cookie') {
156
-
conditions.cookie = value.split(',').map(v => v.trim().toLowerCase());
157
-
}
158
-
}
159
-
}
160
-
161
-
// Parse the 'from' pattern
162
-
const { pattern, params } = convertPathToRegex(from);
163
-
164
-
return {
165
-
from,
166
-
to,
167
-
status,
168
-
force,
169
-
conditions: Object.keys(conditions).length > 0 ? conditions : undefined,
170
-
queryParams: Object.keys(queryParams).length > 0 ? queryParams : undefined,
171
-
fromPattern: pattern,
172
-
fromParams: params,
173
-
};
174
-
}
175
-
176
-
/**
177
-
* Convert a path pattern with placeholders and splats to a regex
178
-
* Examples:
179
-
* /blog/:year/:month/:day -> captures year, month, day
180
-
* /news/* -> captures splat
181
-
*/
182
-
function convertPathToRegex(pattern: string): { pattern: RegExp; params: string[] } {
183
-
const params: string[] = [];
184
-
let regexStr = '^';
185
-
186
-
// Split by query string if present
187
-
const pathPart = pattern.split('?')[0] || pattern;
188
-
189
-
// Escape special regex characters except * and :
190
-
let escaped = pathPart.replace(/[.+^${}()|[\]\\]/g, '\\$&');
191
-
192
-
// Replace :param with named capture groups
193
-
escaped = escaped.replace(/:([a-zA-Z_][a-zA-Z0-9_]*)/g, (match, paramName) => {
194
-
params.push(paramName);
195
-
// Match path segment (everything except / and ?)
196
-
return '([^/?]+)';
197
-
});
198
-
199
-
// Replace * with splat capture (matches everything including /)
200
-
if (escaped.includes('*')) {
201
-
escaped = escaped.replace(/\*/g, '(.*)');
202
-
params.push('splat');
203
-
}
204
-
205
-
regexStr += escaped;
206
-
207
-
// Make trailing slash optional
208
-
if (!regexStr.endsWith('.*')) {
209
-
regexStr += '/?';
210
-
}
211
-
212
-
regexStr += '$';
213
-
214
-
return {
215
-
pattern: new RegExp(regexStr),
216
-
params,
217
-
};
218
-
}
219
-
220
-
/**
221
-
* Match a request path against redirect rules
222
-
*/
223
-
export function matchRedirectRule(
224
-
requestPath: string,
225
-
rules: RedirectRule[],
226
-
context?: {
227
-
queryParams?: Record<string, string>;
228
-
headers?: Record<string, string>;
229
-
cookies?: Record<string, string>;
230
-
}
231
-
): RedirectMatch | null {
232
-
// Normalize path: ensure leading slash, remove trailing slash (except for root)
233
-
let normalizedPath = requestPath.startsWith('/') ? requestPath : `/${requestPath}`;
234
-
235
-
for (const rule of rules) {
236
-
// Check query parameter conditions first (if any)
237
-
if (rule.queryParams) {
238
-
// If rule requires query params but none provided, skip this rule
239
-
if (!context?.queryParams) {
240
-
continue;
241
-
}
242
-
243
-
const queryMatches = Object.entries(rule.queryParams).every(([key, value]) => {
244
-
const actualValue = context.queryParams?.[key];
245
-
return actualValue !== undefined;
246
-
});
247
-
248
-
if (!queryMatches) {
249
-
continue;
250
-
}
251
-
}
252
-
253
-
// Check conditional redirects (country, language, role, cookie)
254
-
if (rule.conditions) {
255
-
if (rule.conditions.country && context?.headers) {
256
-
const cfCountry = context.headers['cf-ipcountry'];
257
-
const xCountry = context.headers['x-country'];
258
-
const country = (cfCountry?.toLowerCase() || xCountry?.toLowerCase());
259
-
if (!country || !rule.conditions.country.includes(country)) {
260
-
continue;
261
-
}
262
-
}
263
-
264
-
if (rule.conditions.language && context?.headers) {
265
-
const acceptLang = context.headers['accept-language'];
266
-
if (!acceptLang) {
267
-
continue;
268
-
}
269
-
// Parse accept-language header (simplified)
270
-
const langs = acceptLang.split(',').map(l => {
271
-
const langPart = l.split(';')[0];
272
-
return langPart ? langPart.trim().toLowerCase() : '';
273
-
}).filter(l => l !== '');
274
-
const hasMatch = rule.conditions.language.some(lang =>
275
-
langs.some(l => l === lang || l.startsWith(lang + '-'))
276
-
);
277
-
if (!hasMatch) {
278
-
continue;
279
-
}
280
-
}
281
-
282
-
if (rule.conditions.cookie && context?.cookies) {
283
-
const hasCookie = rule.conditions.cookie.some(cookieName =>
284
-
context.cookies && cookieName in context.cookies
285
-
);
286
-
if (!hasCookie) {
287
-
continue;
288
-
}
289
-
}
290
-
291
-
// Role-based redirects would need JWT verification - skip for now
292
-
if (rule.conditions.role) {
293
-
continue;
294
-
}
295
-
}
296
-
297
-
// Match the path pattern
298
-
const match = rule.fromPattern?.exec(normalizedPath);
299
-
if (!match) {
300
-
continue;
301
-
}
302
-
303
-
// Build the target path by replacing placeholders
304
-
let targetPath = rule.to;
305
-
306
-
// Replace captured parameters
307
-
if (rule.fromParams && match.length > 1) {
308
-
for (let i = 0; i < rule.fromParams.length; i++) {
309
-
const paramName = rule.fromParams[i];
310
-
const paramValue = match[i + 1];
311
-
312
-
if (!paramName || !paramValue) continue;
313
-
314
-
if (paramName === 'splat') {
315
-
targetPath = targetPath.replace(':splat', paramValue);
316
-
} else {
317
-
targetPath = targetPath.replace(`:${paramName}`, paramValue);
318
-
}
319
-
}
320
-
}
321
-
322
-
// Handle query parameter replacements
323
-
if (rule.queryParams && context?.queryParams) {
324
-
for (const [key, placeholder] of Object.entries(rule.queryParams)) {
325
-
const actualValue = context.queryParams[key];
326
-
if (actualValue && placeholder && placeholder.startsWith(':')) {
327
-
const paramName = placeholder.slice(1);
328
-
if (paramName) {
329
-
targetPath = targetPath.replace(`:${paramName}`, actualValue);
330
-
}
331
-
}
332
-
}
333
-
}
334
-
335
-
// Preserve query string for 200, 301, 302 redirects (unless target already has one)
336
-
if ([200, 301, 302].includes(rule.status) && context?.queryParams && !targetPath.includes('?')) {
337
-
const queryString = Object.entries(context.queryParams)
338
-
.map(([k, v]) => `${encodeURIComponent(k)}=${encodeURIComponent(v)}`)
339
-
.join('&');
340
-
if (queryString) {
341
-
targetPath += `?${queryString}`;
342
-
}
343
-
}
344
-
345
-
return {
346
-
rule,
347
-
targetPath,
348
-
status: rule.status,
349
-
};
350
-
}
351
-
352
-
return null;
353
-
}
354
-
355
-
/**
356
-
* Load redirect rules from a cached site
357
-
*/
358
-
export async function loadRedirectRules(did: string, rkey: string): Promise<RedirectRule[]> {
359
-
const CACHE_DIR = process.env.CACHE_DIR || './cache/sites';
360
-
const redirectsPath = `${CACHE_DIR}/${did}/${rkey}/_redirects`;
361
-
362
-
if (!existsSync(redirectsPath)) {
363
-
return [];
364
-
}
365
-
366
-
try {
367
-
const content = await readFile(redirectsPath, 'utf-8');
368
-
return parseRedirectsFile(content);
369
-
} catch (err) {
370
-
console.error('Failed to load _redirects file', err);
371
-
return [];
372
-
}
373
-
}
374
-
375
-
/**
376
-
* Parse cookies from Cookie header
377
-
*/
378
-
export function parseCookies(cookieHeader?: string): Record<string, string> {
379
-
if (!cookieHeader) return {};
380
-
381
-
const cookies: Record<string, string> = {};
382
-
const parts = cookieHeader.split(';');
383
-
384
-
for (const part of parts) {
385
-
const [key, ...valueParts] = part.split('=');
386
-
if (key && valueParts.length > 0) {
387
-
cookies[key.trim()] = valueParts.join('=').trim();
388
-
}
389
-
}
390
-
391
-
return cookies;
392
-
}
393
-
394
-
/**
395
-
* Parse query string into object
396
-
*/
397
-
export function parseQueryString(url: string): Record<string, string> {
398
-
const queryStart = url.indexOf('?');
399
-
if (queryStart === -1) return {};
400
-
401
-
const queryString = url.slice(queryStart + 1);
402
-
const params: Record<string, string> = {};
403
-
404
-
for (const pair of queryString.split('&')) {
405
-
const [key, value] = pair.split('=');
406
-
if (key) {
407
-
params[decodeURIComponent(key)] = value ? decodeURIComponent(value) : '';
408
-
}
409
-
}
410
-
411
-
return params;
412
-
}
413
-
···
+6
-168
hosting-service/src/server.ts
+6
-168
hosting-service/src/server.ts
···
7
import { lookup } from 'mime-types';
8
import { logger, observabilityMiddleware, observabilityErrorHandler, logCollector, errorTracker, metricsCollector } from './lib/observability';
9
import { fileCache, metadataCache, rewrittenHtmlCache, getCacheKey, type FileMetadata } from './lib/cache';
10
-
import { loadRedirectRules, matchRedirectRule, parseCookies, parseQueryString, type RedirectRule } from './lib/redirects';
11
12
const BASE_HOST = process.env.BASE_HOST || 'wisp.place';
13
···
36
}
37
}
38
39
-
// Cache for redirect rules (per site)
40
-
const redirectRulesCache = new Map<string, RedirectRule[]>();
41
-
42
-
/**
43
-
* Clear redirect rules cache for a specific site
44
-
* Should be called when a site is updated/recached
45
-
*/
46
-
export function clearRedirectRulesCache(did: string, rkey: string) {
47
-
const cacheKey = `${did}:${rkey}`;
48
-
redirectRulesCache.delete(cacheKey);
49
-
}
50
-
51
// Helper to serve files from cache
52
-
async function serveFromCache(
53
-
did: string,
54
-
rkey: string,
55
-
filePath: string,
56
-
fullUrl?: string,
57
-
headers?: Record<string, string>
58
-
) {
59
-
// Check for redirect rules first
60
-
const redirectCacheKey = `${did}:${rkey}`;
61
-
let redirectRules = redirectRulesCache.get(redirectCacheKey);
62
-
63
-
if (redirectRules === undefined) {
64
-
// Load rules for the first time
65
-
redirectRules = await loadRedirectRules(did, rkey);
66
-
redirectRulesCache.set(redirectCacheKey, redirectRules);
67
-
}
68
-
69
-
// Apply redirect rules if any exist
70
-
if (redirectRules.length > 0) {
71
-
const requestPath = '/' + (filePath || '');
72
-
const queryParams = fullUrl ? parseQueryString(fullUrl) : {};
73
-
const cookies = parseCookies(headers?.['cookie']);
74
-
75
-
const redirectMatch = matchRedirectRule(requestPath, redirectRules, {
76
-
queryParams,
77
-
headers,
78
-
cookies,
79
-
});
80
-
81
-
if (redirectMatch) {
82
-
const { targetPath, status } = redirectMatch;
83
-
84
-
// Handle different status codes
85
-
if (status === 200) {
86
-
// Rewrite: serve different content but keep URL the same
87
-
// Remove leading slash for internal path resolution
88
-
const rewritePath = targetPath.startsWith('/') ? targetPath.slice(1) : targetPath;
89
-
return serveFileInternal(did, rkey, rewritePath);
90
-
} else if (status === 301 || status === 302) {
91
-
// External redirect: change the URL
92
-
return new Response(null, {
93
-
status,
94
-
headers: {
95
-
'Location': targetPath,
96
-
'Cache-Control': status === 301 ? 'public, max-age=31536000' : 'public, max-age=0',
97
-
},
98
-
});
99
-
} else if (status === 404) {
100
-
// Custom 404 page
101
-
const custom404Path = targetPath.startsWith('/') ? targetPath.slice(1) : targetPath;
102
-
const response = await serveFileInternal(did, rkey, custom404Path);
103
-
// Override status to 404
104
-
return new Response(response.body, {
105
-
status: 404,
106
-
headers: response.headers,
107
-
});
108
-
}
109
-
}
110
-
}
111
-
112
-
// No redirect matched, serve normally
113
-
return serveFileInternal(did, rkey, filePath);
114
-
}
115
-
116
-
// Internal function to serve a file (used by both normal serving and rewrites)
117
-
async function serveFileInternal(did: string, rkey: string, filePath: string) {
118
// Default to index.html if path is empty or ends with /
119
let requestPath = filePath || 'index.html';
120
if (requestPath.endsWith('/')) {
···
216
did: string,
217
rkey: string,
218
filePath: string,
219
-
basePath: string,
220
-
fullUrl?: string,
221
-
headers?: Record<string, string>
222
) {
223
-
// Check for redirect rules first
224
-
const redirectCacheKey = `${did}:${rkey}`;
225
-
let redirectRules = redirectRulesCache.get(redirectCacheKey);
226
-
227
-
if (redirectRules === undefined) {
228
-
// Load rules for the first time
229
-
redirectRules = await loadRedirectRules(did, rkey);
230
-
redirectRulesCache.set(redirectCacheKey, redirectRules);
231
-
}
232
-
233
-
// Apply redirect rules if any exist
234
-
if (redirectRules.length > 0) {
235
-
const requestPath = '/' + (filePath || '');
236
-
const queryParams = fullUrl ? parseQueryString(fullUrl) : {};
237
-
const cookies = parseCookies(headers?.['cookie']);
238
-
239
-
const redirectMatch = matchRedirectRule(requestPath, redirectRules, {
240
-
queryParams,
241
-
headers,
242
-
cookies,
243
-
});
244
-
245
-
if (redirectMatch) {
246
-
const { targetPath, status } = redirectMatch;
247
-
248
-
// Handle different status codes
249
-
if (status === 200) {
250
-
// Rewrite: serve different content but keep URL the same
251
-
const rewritePath = targetPath.startsWith('/') ? targetPath.slice(1) : targetPath;
252
-
return serveFileInternalWithRewrite(did, rkey, rewritePath, basePath);
253
-
} else if (status === 301 || status === 302) {
254
-
// External redirect: change the URL
255
-
// For sites.wisp.place, we need to adjust the target path to include the base path
256
-
// unless it's an absolute URL
257
-
let redirectTarget = targetPath;
258
-
if (!targetPath.startsWith('http://') && !targetPath.startsWith('https://')) {
259
-
redirectTarget = basePath + (targetPath.startsWith('/') ? targetPath.slice(1) : targetPath);
260
-
}
261
-
return new Response(null, {
262
-
status,
263
-
headers: {
264
-
'Location': redirectTarget,
265
-
'Cache-Control': status === 301 ? 'public, max-age=31536000' : 'public, max-age=0',
266
-
},
267
-
});
268
-
} else if (status === 404) {
269
-
// Custom 404 page
270
-
const custom404Path = targetPath.startsWith('/') ? targetPath.slice(1) : targetPath;
271
-
const response = await serveFileInternalWithRewrite(did, rkey, custom404Path, basePath);
272
-
// Override status to 404
273
-
return new Response(response.body, {
274
-
status: 404,
275
-
headers: response.headers,
276
-
});
277
-
}
278
-
}
279
-
}
280
-
281
-
// No redirect matched, serve normally
282
-
return serveFileInternalWithRewrite(did, rkey, filePath, basePath);
283
-
}
284
-
285
-
// Internal function to serve a file with rewriting
286
-
async function serveFileInternalWithRewrite(did: string, rkey: string, filePath: string, basePath: string) {
287
// Default to index.html if path is empty or ends with /
288
let requestPath = filePath || 'index.html';
289
if (requestPath.endsWith('/')) {
···
461
462
try {
463
await downloadAndCacheSite(did, rkey, siteData.record, pdsEndpoint, siteData.cid);
464
-
// Clear redirect rules cache since the site was updated
465
-
clearRedirectRulesCache(did, rkey);
466
logger.info('Site cached successfully', { did, rkey });
467
return true;
468
} catch (err) {
···
530
531
// Serve with HTML path rewriting to handle absolute paths
532
const basePath = `/${identifier}/${site}/`;
533
-
const headers: Record<string, string> = {};
534
-
c.req.raw.headers.forEach((value, key) => {
535
-
headers[key.toLowerCase()] = value;
536
-
});
537
-
return serveFromCacheWithRewrite(did, site, filePath, basePath, c.req.url, headers);
538
}
539
540
// Check if this is a DNS hash subdomain
···
570
return c.text('Site not found', 404);
571
}
572
573
-
const headers: Record<string, string> = {};
574
-
c.req.raw.headers.forEach((value, key) => {
575
-
headers[key.toLowerCase()] = value;
576
-
});
577
-
return serveFromCache(customDomain.did, rkey, path, c.req.url, headers);
578
}
579
580
// Route 2: Registered subdomains - /*.wisp.place/*
···
598
return c.text('Site not found', 404);
599
}
600
601
-
const headers: Record<string, string> = {};
602
-
c.req.raw.headers.forEach((value, key) => {
603
-
headers[key.toLowerCase()] = value;
604
-
});
605
-
return serveFromCache(domainInfo.did, rkey, path, c.req.url, headers);
606
}
607
608
// Route 1: Custom domains - /*
···
625
return c.text('Site not found', 404);
626
}
627
628
-
const headers: Record<string, string> = {};
629
-
c.req.raw.headers.forEach((value, key) => {
630
-
headers[key.toLowerCase()] = value;
631
-
});
632
-
return serveFromCache(customDomain.did, rkey, path, c.req.url, headers);
633
});
634
635
// Internal observability endpoints (for admin panel)
···
7
import { lookup } from 'mime-types';
8
import { logger, observabilityMiddleware, observabilityErrorHandler, logCollector, errorTracker, metricsCollector } from './lib/observability';
9
import { fileCache, metadataCache, rewrittenHtmlCache, getCacheKey, type FileMetadata } from './lib/cache';
10
11
const BASE_HOST = process.env.BASE_HOST || 'wisp.place';
12
···
35
}
36
}
37
38
// Helper to serve files from cache
39
+
async function serveFromCache(did: string, rkey: string, filePath: string) {
40
// Default to index.html if path is empty or ends with /
41
let requestPath = filePath || 'index.html';
42
if (requestPath.endsWith('/')) {
···
138
did: string,
139
rkey: string,
140
filePath: string,
141
+
basePath: string
142
) {
143
// Default to index.html if path is empty or ends with /
144
let requestPath = filePath || 'index.html';
145
if (requestPath.endsWith('/')) {
···
317
318
try {
319
await downloadAndCacheSite(did, rkey, siteData.record, pdsEndpoint, siteData.cid);
320
logger.info('Site cached successfully', { did, rkey });
321
return true;
322
} catch (err) {
···
384
385
// Serve with HTML path rewriting to handle absolute paths
386
const basePath = `/${identifier}/${site}/`;
387
+
return serveFromCacheWithRewrite(did, site, filePath, basePath);
388
}
389
390
// Check if this is a DNS hash subdomain
···
420
return c.text('Site not found', 404);
421
}
422
423
+
return serveFromCache(customDomain.did, rkey, path);
424
}
425
426
// Route 2: Registered subdomains - /*.wisp.place/*
···
444
return c.text('Site not found', 404);
445
}
446
447
+
return serveFromCache(domainInfo.did, rkey, path);
448
}
449
450
// Route 1: Custom domains - /*
···
467
return c.text('Site not found', 404);
468
}
469
470
+
return serveFromCache(customDomain.did, rkey, path);
471
});
472
473
// Internal observability endpoints (for admin panel)
cli/.gitignore
cli/.gitignore
This file has not been changed.
cli/Cargo.lock
cli/Cargo.lock
This file has not been changed.
cli/Cargo.toml
cli/Cargo.toml
This file has not been changed.
+8
-15
cli/src/blob_map.rs
+8
-15
cli/src/blob_map.rs
···
34
let blob_ref = &file_node.blob;
35
let cid_string = blob_ref.blob().r#ref.to_string();
36
37
-
// Store both normalized and full paths
38
-
// Normalize by removing base folder prefix (e.g., "cobblemon/index.html" -> "index.html")
39
-
let normalized_path = normalize_path(&full_path);
40
-
41
blob_map.insert(
42
-
normalized_path.clone(),
43
-
(blob_ref.clone().into_static(), cid_string.clone())
44
);
45
-
46
-
// Also store the full path for matching
47
-
if normalized_path != full_path {
48
-
blob_map.insert(
49
-
full_path,
50
-
(blob_ref.clone().into_static(), cid_string)
51
-
);
52
-
}
53
}
54
EntryNode::Directory(subdir) => {
55
let sub_map = extract_blob_map_recursive(subdir, full_path);
···
67
/// Normalize file path by removing base folder prefix
68
/// Example: "cobblemon/index.html" -> "index.html"
69
///
70
-
/// Mirrors TypeScript implementation at src/routes/wisp.ts line 291
71
pub fn normalize_path(path: &str) -> String {
72
// Remove base folder prefix (everything before first /)
73
if let Some(idx) = path.find('/') {
···
34
let blob_ref = &file_node.blob;
35
let cid_string = blob_ref.blob().r#ref.to_string();
36
37
+
// Store with full path (mirrors TypeScript implementation)
38
blob_map.insert(
39
+
full_path,
40
+
(blob_ref.clone().into_static(), cid_string)
41
);
42
}
43
EntryNode::Directory(subdir) => {
44
let sub_map = extract_blob_map_recursive(subdir, full_path);
···
56
/// Normalize file path by removing base folder prefix
57
/// Example: "cobblemon/index.html" -> "index.html"
58
///
59
+
/// Note: This function is kept for reference but is no longer used in production code.
60
+
/// The TypeScript server has a similar normalization (src/routes/wisp.ts line 291) to handle
61
+
/// uploads that include a base folder prefix, but our CLI doesn't need this since we
62
+
/// track full paths consistently.
63
+
#[allow(dead_code)]
64
pub fn normalize_path(path: &str) -> String {
65
// Remove base folder prefix (everything before first /)
66
if let Some(idx) = path.find('/') {
cli/src/cid.rs
cli/src/cid.rs
This file has not been changed.
+30
-19
cli/src/main.rs
+30
-19
cli/src/main.rs
···
152
};
153
154
// Build directory tree
155
-
let (root_dir, total_files, reused_count) = build_directory(agent, &path, &existing_blob_map).await?;
156
let uploaded_count = total_files - reused_count;
157
158
// Create the Fs record
···
181
Ok(())
182
}
183
184
-
185
fn build_directory<'a>(
186
agent: &'a Agent<impl jacquard::client::AgentSession + IdentityResolver + 'a>,
187
dir_path: &'a Path,
188
existing_blobs: &'a HashMap<String, (jacquard_common::types::blob::BlobRef<'static>, String)>,
189
) -> std::pin::Pin<Box<dyn std::future::Future<Output = miette::Result<(Directory<'static>, usize, usize)>> + 'a>>
190
{
191
Box::pin(async move {
···
211
212
213
214
215
-
216
-
217
-
218
-
219
-
220
-
221
}
222
223
// Process files concurrently with a limit of 5
224
let file_results: Vec<(Entry<'static>, bool)> = stream::iter(file_tasks)
225
-
.map(|(name, path)| async move {
226
-
let (file_node, reused) = process_file(agent, &path, &name, existing_blobs).await?;
227
let entry = Entry::new()
228
.name(CowStr::from(name))
229
.node(EntryNode::File(Box::new(file_node)))
···
251
// Process directories recursively (sequentially to avoid too much nesting)
252
let mut dir_entries = Vec::new();
253
for (name, path) in dir_tasks {
254
-
let (subdir, sub_total, sub_reused) = build_directory(agent, &path, existing_blobs).await?;
255
dir_entries.push(Entry::new()
256
.name(CowStr::from(name))
257
.node(EntryNode::Directory(Box::new(subdir)))
···
275
276
/// Process a single file: gzip -> base64 -> upload blob (or reuse existing)
277
/// Returns (File, reused: bool)
278
async fn process_file(
279
agent: &Agent<impl jacquard::client::AgentSession + IdentityResolver>,
280
file_path: &Path,
281
-
file_name: &str,
282
existing_blobs: &HashMap<String, (jacquard_common::types::blob::BlobRef<'static>, String)>,
283
) -> miette::Result<(File<'static>, bool)>
284
{
···
301
// Compute CID for this file (CRITICAL: on base64-encoded gzipped content)
302
let file_cid = cid::compute_cid(&base64_bytes);
303
304
-
// Normalize the file path for comparison
305
-
let normalized_path = blob_map::normalize_path(file_name);
306
-
307
// Check if we have an existing blob with the same CID
308
-
let existing_blob = existing_blobs.get(&normalized_path)
309
-
.or_else(|| existing_blobs.get(file_name));
310
311
if let Some((existing_blob_ref, existing_cid)) = existing_blob {
312
if existing_cid == &file_cid {
313
// CIDs match - reuse existing blob
314
-
println!(" ✓ Reusing blob for {} (CID: {})", file_name, file_cid);
315
return Ok((
316
File::new()
317
.r#type(CowStr::from("file"))
···
326
}
327
328
// File is new or changed - upload it
329
-
println!(" ↑ Uploading {} ({} bytes, CID: {})", file_name, base64_bytes.len(), file_cid);
330
let blob = agent.upload_blob(
331
base64_bytes,
332
MimeType::new_static("application/octet-stream"),
···
152
};
153
154
// Build directory tree
155
+
let (root_dir, total_files, reused_count) = build_directory(agent, &path, &existing_blob_map, String::new()).await?;
156
let uploaded_count = total_files - reused_count;
157
158
// Create the Fs record
···
181
Ok(())
182
}
183
184
+
/// Recursively build a Directory from a filesystem path
185
+
/// current_path is the path from the root of the site (e.g., "" for root, "config" for config dir)
186
fn build_directory<'a>(
187
agent: &'a Agent<impl jacquard::client::AgentSession + IdentityResolver + 'a>,
188
dir_path: &'a Path,
189
existing_blobs: &'a HashMap<String, (jacquard_common::types::blob::BlobRef<'static>, String)>,
190
+
current_path: String,
191
) -> std::pin::Pin<Box<dyn std::future::Future<Output = miette::Result<(Directory<'static>, usize, usize)>> + 'a>>
192
{
193
Box::pin(async move {
···
213
214
215
216
+
let metadata = entry.metadata().into_diagnostic()?;
217
218
+
if metadata.is_file() {
219
+
// Construct full path for this file (for blob map lookup)
220
+
let full_path = if current_path.is_empty() {
221
+
name_str.clone()
222
+
} else {
223
+
format!("{}/{}", current_path, name_str)
224
+
};
225
+
file_tasks.push((name_str, path, full_path));
226
+
} else if metadata.is_dir() {
227
+
dir_tasks.push((name_str, path));
228
+
}
229
}
230
231
// Process files concurrently with a limit of 5
232
let file_results: Vec<(Entry<'static>, bool)> = stream::iter(file_tasks)
233
+
.map(|(name, path, full_path)| async move {
234
+
let (file_node, reused) = process_file(agent, &path, &full_path, existing_blobs).await?;
235
let entry = Entry::new()
236
.name(CowStr::from(name))
237
.node(EntryNode::File(Box::new(file_node)))
···
259
// Process directories recursively (sequentially to avoid too much nesting)
260
let mut dir_entries = Vec::new();
261
for (name, path) in dir_tasks {
262
+
// Construct full path for subdirectory
263
+
let subdir_path = if current_path.is_empty() {
264
+
name.clone()
265
+
} else {
266
+
format!("{}/{}", current_path, name)
267
+
};
268
+
let (subdir, sub_total, sub_reused) = build_directory(agent, &path, existing_blobs, subdir_path).await?;
269
dir_entries.push(Entry::new()
270
.name(CowStr::from(name))
271
.node(EntryNode::Directory(Box::new(subdir)))
···
289
290
/// Process a single file: gzip -> base64 -> upload blob (or reuse existing)
291
/// Returns (File, reused: bool)
292
+
/// file_path_key is the full path from the site root (e.g., "config/file.json") for blob map lookup
293
async fn process_file(
294
agent: &Agent<impl jacquard::client::AgentSession + IdentityResolver>,
295
file_path: &Path,
296
+
file_path_key: &str,
297
existing_blobs: &HashMap<String, (jacquard_common::types::blob::BlobRef<'static>, String)>,
298
) -> miette::Result<(File<'static>, bool)>
299
{
···
316
// Compute CID for this file (CRITICAL: on base64-encoded gzipped content)
317
let file_cid = cid::compute_cid(&base64_bytes);
318
319
// Check if we have an existing blob with the same CID
320
+
let existing_blob = existing_blobs.get(file_path_key);
321
322
if let Some((existing_blob_ref, existing_cid)) = existing_blob {
323
if existing_cid == &file_cid {
324
// CIDs match - reuse existing blob
325
+
println!(" ✓ Reusing blob for {} (CID: {})", file_path_key, file_cid);
326
return Ok((
327
File::new()
328
.r#type(CowStr::from("file"))
···
337
}
338
339
// File is new or changed - upload it
340
+
println!(" ↑ Uploading {} ({} bytes, CID: {})", file_path_key, base64_bytes.len(), file_cid);
341
let blob = agent.upload_blob(
342
base64_bytes,
343
MimeType::new_static("application/octet-stream"),
History
3 rounds
0 comments
nekomimi.pet
submitted
#2
6 commits
expand
collapse
9a803381
init support for redirects file
f1f70b3b
Add support for existing blob reuse in deployment process
56b1ef45
dont normalize paths when comparing CIDs
38b1c4c6
add pull and serve to cli
436d7a06
remove jacquard submodule
122e18dd
update flake
expand 0 comments
closed without merging
nekomimi.pet
submitted
#1
2 commits
expand
collapse
f1f70b3b
Add support for existing blob reuse in deployment process
56b1ef45
dont normalize paths when comparing CIDs
expand 0 comments
nekomimi.pet
submitted
#0
2 commits
expand
collapse
9a803381
init support for redirects file
f1f70b3b
Add support for existing blob reuse in deployment process