commits
- Add PlainContent field with HTML tags stripped for text/plain email part
- Strip HTML5 semantic tags (article, section, nav, etc.) that email
clients like Gmail don't support
- Add test to verify text output has no HTML tags
Fixes issue where raw HTML tags were visible in email clients that
display the text/plain part or don't support HTML5 semantic elements.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Feeds like mitchellh.com/feed.xml that only have titles and links
(no description/content) will no longer render empty inline sections.
Adds #nosec G203 comment to suppress gosec warning about
htmltemplate.HTML conversion. This is safe because the content
is sanitized by bluemonday.UGCPolicy() before conversion, which
removes all unsafe HTML tags, attributes, and scripts.
Tests confirm that:
- HTML in feeds renders correctly (not escaped as text)
- Unsafe tags like <script> are stripped
- Style attributes are removed
- Safe HTML formatting (p, strong, a, etc.) is preserved
Implements safe HTML rendering for inline feeds by:
- Adding bluemonday library for HTML sanitization
- Creating sanitizeHTML function using UGCPolicy (allows safe tags, strips styles and unsafe elements)
- Introducing templateFeedItem and templateFeedGroup structs with SanitizedContent field
- Pre-sanitizing all feed content before template rendering
- Updating digest.html template to use SanitizedContent instead of raw Content
This prevents XSS attacks while still allowing safe HTML formatting like headings, links, lists, and basic text formatting in feed content.
Changes SCP/SFTP upload handlers from delete-recreate pattern to
update pattern to preserve database IDs and feed history.
Previously, reuploading a feeds file (even unchanged) would:
- Delete the config and all associated feeds (CASCADE DELETE)
- Delete all seen_items history for those feeds
- Create new config/feeds with new IDs
- Cause scheduler to treat all items as new → duplicate emails
Now:
- Checks if config already exists
- Updates existing config instead of deleting
- Matches feeds by URL and updates/adds/removes as needed
- Preserves feed IDs and seen_items history
- No duplicate emails on reupload
Added store methods:
- UpdateConfigTx/UpdateConfig
- GetConfigTx
- UpdateFeedTx/UpdateFeed
- DeleteFeedTx/DeleteFeed
- GetFeedsByConfigTx
Fixes issue where SCP reupload wiped feed history and sent duplicates.
Adds FILE FORMAT section documenting feed config directives and providing
a complete example for users uploading feed configs.
💘 Generated with Crush
Assisted-by: Copilot: Claude Sonnet 4.5 via Crush <crush@charm.land>
- Add ldflags in flake.nix to set version and commit hash at build time
- Pass version and commit to fang.Execute() via WithVersion and WithCommit options
- Fixes version output showing 'unknown (built from source)'
- Version now correctly displays as 'herald version 0.1.1 (hash)'
💘 Generated with Crush
Assisted-by: Copilot: Claude Sonnet 4.5 via Crush <crush@charm.land>
- Root cause: RecordEmailSend() called while transaction tx was open
- With SetMaxOpenConns(1), this caused self-deadlock (same goroutine
trying to acquire connection it already holds)
- Solution: Added GenerateTrackingToken() and RecordEmailSendTx(tx)
that accepts transaction parameter
- Generate token before transaction, record within transaction
- Removed debug logging from email/send.go (no longer needed)
The hang occurred at scheduler/scheduler.go:412 where RecordEmailSend
tried to use db.Exec() while the transaction had the only connection
locked. SQLite's busy_timeout doesn't help with self-deadlock because
it's the same connection context.
Complete email tracking system for monitoring user engagement:
**Database & Storage:**
- Add email_sends table tracking sends, opens, bounces
- RecordEmailSend() generates unique tracking tokens
- MarkEmailOpened() records pixel impressions
- GetConfigEngagement() returns stats (sends, opens, rate, last open)
- GetInactiveConfigs() finds configs without opens
- CleanupOldSends() removes old tracking data
**Email Integration:**
- Add tracking pixel to HTML emails (1x1 transparent GIF)
- Pass tracking token through Send() -> scheduler
- Record sends in database before SMTP transmission
**Web Endpoint:**
- /t/{token}.gif serves tracking pixel
- Silently logs opens without revealing token validity
- Cache-Control headers prevent caching
**Background Jobs:**
- Weekly check for inactive configs (90 days no opens)
- Auto-deactivate configs with 3+ sends but 0 opens
- Daily cleanup of email send records >6 months
- Log auto-deactivations for transparency
**Dashboard:**
- Show engagement metrics on user profile page
- Display: sends, opens, open rate %, days since last open
- Visual indicator for inactive configs
**Configuration:**
- inactivityThreshold = 90 days
- minSendsBeforeDeactivate = 3
- emailSendsRetention = 6 months
Comprehensive tests for all tracking functionality.
💘 Generated with Crush
Assisted-by: Copilot: Claude Sonnet 4.5 via Crush <crush@charm.land>
Enhances email deliverability and compliance:
- Add DKIM email signing support with RSA keys (PKCS1/PKCS8)
- Support both inline keys and file-based keys for flexibility
- Implement RFC 8058 one-click unsubscribe in POST handler
- Add email tracking schema (sends, opens, bounces)
- Add .gitignore rule for *.pem files
DKIM configuration via YAML or env vars:
- dkim_selector, dkim_domain required for signing
- dkim_private_key or dkim_private_key_file for key material
One-click unsubscribe detects List-Unsubscribe=One-Click POST
body and immediately deactivates without HTML response.
💘 Generated with Crush
Assisted-by: Copilot: Claude Sonnet 4.5 via Crush <crush@charm.land>
Implemented test coverage for critical business logic:
**ratelimit package** (66.7% coverage)
- TestNew: Verifies limiter initialization
- TestAllow_SingleKey: Tests burst and rate limiting
- TestAllow_MultipleKeys: Validates independent per-key limits
- TestAllow_TokenRefill: Confirms token bucket refill behavior
- TestAllow_UpdatesLastSeen: Checks timestamp tracking
- TestCleanup: Validates stale limiter removal
**config package** (38.1% coverage)
- Parse tests: Empty input, comments, directives (email/cron/digest/inline)
- Feed parsing: With/without names, multiple feeds, complete config
- Case-insensitive directives, parseBool edge cases
- Validate tests: Email format, cron expression validation
- URL validation, missing fields, complete config validation
**store package** (42.3% coverage)
- User CRUD: GetOrCreateUser, GetUserByFingerprint, GetUserByID, DeleteUser
- Config CRUD: CreateConfig, ListConfigs, GetConfig, GetConfigByID, DeleteConfig
- Feed CRUD: CreateFeed, GetFeedsByConfig
- Seen items: MarkItemSeen, IsItemSeen, GetSeenGUIDs, CleanupOldSeenItems
- All tests use in-memory SQLite with automatic cleanup
**Test Infrastructure**
- All tests follow Go testing conventions
- Helper functions for test setup (setupTestDB)
- Use context.Background() for database operations
- Cleanup with t.Cleanup() to prevent resource leaks
**Coverage Summary**
- Total: 3 packages with tests
- 61 test cases passing
- Focus on critical business logic (parsing, validation, database, rate limiting)
💘 Generated with Crush
Assisted-by: Copilot: Claude Sonnet 4.5 via Crush <crush@charm.land>
Implemented P3 issues #28 and #29:
**#28: Fix Inconsistent Command Help**
- Updated SSH welcome message in ssh/server.go
- Added missing 'activate <file>' and 'deactivate <file>' commands
- Now shows all 7 commands: ls, cat, rm, activate, deactivate, run, logs
- Welcome message now matches actual available commands
**#29: Align Config Defaults**
- Updated README.md line 89 to show correct 'inline' default: false
- Was incorrectly documented as 'true'
- Now matches actual code behavior in config/parse.go:27
💘 Generated with Crush
Assisted-by: Copilot: Claude Sonnet 4.5 via Crush <crush@charm.land>
Implemented P3 issues #19, #21, #18, #33-34:
**#19: Extract Magic Numbers**
- scheduler/scheduler.go: Added 5 constants
- emailsPerMinutePerUser, emailRateBurst
- cleanupInterval, seenItemsRetention (6 months), itemMaxAge (3 months)
- minItemsForDigest (threshold for disabling inline content)
- scheduler/fetch.go: Added 2 constants
- feedFetchTimeout (30s), maxConcurrentFetch (10)
- web/handlers.go: Added 2 constants
- maxFeedItems (100), shortFingerprintLen (8)
- Replaced all hardcoded values with named constants
**#21: Remove Unused Context Parameter**
- Removed ctx parameter from store.DB.Migrate() method
- Updated main.go call site from db.Migrate(ctx) → db.Migrate()
- Context was unused since migrate() doesn't support cancellation
**#18: Error Wrapping Consistency**
- Verified all fmt.Errorf calls use "verb: %w" pattern with colon
- No changes needed - codebase already consistent
**#33-34: Clean Up Unused Code**
- Inlined getCommitHash() function into runServer()
- Standardized fingerprint shortening to 8 chars (was inconsistent 8/12)
- Used shortFingerprintLen constant for all truncation
💘 Generated with Crush
Assisted-by: Copilot: Claude Sonnet 4.5 via Crush <crush@charm.land>
Implemented P3 issues #24, #35, and #22:
**#24: Metrics/Observability**
- Created web/metrics.go with Metrics struct using atomic counters
- Added /metrics endpoint returning JSON snapshot of system and app metrics
- Added /health endpoint for simple health checks with uptime
- Tracked: uptime, Go version, goroutines, memory (alloc/total/sys)
- Tracked: requests (total/active), emails sent, feeds fetched, items seen
- Tracked: active configs, errors total, rate limit hits
**#35: HTTP Request Logging**
- Added loggingMiddleware in web/server.go
- Logs: method, path, status code, duration (ms), remote_addr
- Uses loggingResponseWriter wrapper to capture HTTP status codes
- Tracks active requests via metrics (increment/decrement)
- Increments error counter for 5xx responses
- Middleware chain: logging → rate limiting → handlers
**#22: Standardize Logging Levels**
- Changed 23 log calls from Error → Warn in web/handlers.go
- Error reserved for critical failures (panics, failed migrations)
- Warn used for expected/recoverable failures (DB reads, template errors)
- Changed: 14 DB operations, 6 template renders, 2 response encodings, 1 delete token
💘 Generated with Crush
Assisted-by: Copilot: Claude Sonnet 4.5 via Crush <crush@charm.land>
- Add PlainContent field with HTML tags stripped for text/plain email part
- Strip HTML5 semantic tags (article, section, nav, etc.) that email
clients like Gmail don't support
- Add test to verify text output has no HTML tags
Fixes issue where raw HTML tags were visible in email clients that
display the text/plain part or don't support HTML5 semantic elements.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements safe HTML rendering for inline feeds by:
- Adding bluemonday library for HTML sanitization
- Creating sanitizeHTML function using UGCPolicy (allows safe tags, strips styles and unsafe elements)
- Introducing templateFeedItem and templateFeedGroup structs with SanitizedContent field
- Pre-sanitizing all feed content before template rendering
- Updating digest.html template to use SanitizedContent instead of raw Content
This prevents XSS attacks while still allowing safe HTML formatting like headings, links, lists, and basic text formatting in feed content.
Changes SCP/SFTP upload handlers from delete-recreate pattern to
update pattern to preserve database IDs and feed history.
Previously, reuploading a feeds file (even unchanged) would:
- Delete the config and all associated feeds (CASCADE DELETE)
- Delete all seen_items history for those feeds
- Create new config/feeds with new IDs
- Cause scheduler to treat all items as new → duplicate emails
Now:
- Checks if config already exists
- Updates existing config instead of deleting
- Matches feeds by URL and updates/adds/removes as needed
- Preserves feed IDs and seen_items history
- No duplicate emails on reupload
Added store methods:
- UpdateConfigTx/UpdateConfig
- GetConfigTx
- UpdateFeedTx/UpdateFeed
- DeleteFeedTx/DeleteFeed
- GetFeedsByConfigTx
Fixes issue where SCP reupload wiped feed history and sent duplicates.
- Add ldflags in flake.nix to set version and commit hash at build time
- Pass version and commit to fang.Execute() via WithVersion and WithCommit options
- Fixes version output showing 'unknown (built from source)'
- Version now correctly displays as 'herald version 0.1.1 (hash)'
💘 Generated with Crush
Assisted-by: Copilot: Claude Sonnet 4.5 via Crush <crush@charm.land>
- Root cause: RecordEmailSend() called while transaction tx was open
- With SetMaxOpenConns(1), this caused self-deadlock (same goroutine
trying to acquire connection it already holds)
- Solution: Added GenerateTrackingToken() and RecordEmailSendTx(tx)
that accepts transaction parameter
- Generate token before transaction, record within transaction
- Removed debug logging from email/send.go (no longer needed)
The hang occurred at scheduler/scheduler.go:412 where RecordEmailSend
tried to use db.Exec() while the transaction had the only connection
locked. SQLite's busy_timeout doesn't help with self-deadlock because
it's the same connection context.
Complete email tracking system for monitoring user engagement:
**Database & Storage:**
- Add email_sends table tracking sends, opens, bounces
- RecordEmailSend() generates unique tracking tokens
- MarkEmailOpened() records pixel impressions
- GetConfigEngagement() returns stats (sends, opens, rate, last open)
- GetInactiveConfigs() finds configs without opens
- CleanupOldSends() removes old tracking data
**Email Integration:**
- Add tracking pixel to HTML emails (1x1 transparent GIF)
- Pass tracking token through Send() -> scheduler
- Record sends in database before SMTP transmission
**Web Endpoint:**
- /t/{token}.gif serves tracking pixel
- Silently logs opens without revealing token validity
- Cache-Control headers prevent caching
**Background Jobs:**
- Weekly check for inactive configs (90 days no opens)
- Auto-deactivate configs with 3+ sends but 0 opens
- Daily cleanup of email send records >6 months
- Log auto-deactivations for transparency
**Dashboard:**
- Show engagement metrics on user profile page
- Display: sends, opens, open rate %, days since last open
- Visual indicator for inactive configs
**Configuration:**
- inactivityThreshold = 90 days
- minSendsBeforeDeactivate = 3
- emailSendsRetention = 6 months
Comprehensive tests for all tracking functionality.
💘 Generated with Crush
Assisted-by: Copilot: Claude Sonnet 4.5 via Crush <crush@charm.land>
Enhances email deliverability and compliance:
- Add DKIM email signing support with RSA keys (PKCS1/PKCS8)
- Support both inline keys and file-based keys for flexibility
- Implement RFC 8058 one-click unsubscribe in POST handler
- Add email tracking schema (sends, opens, bounces)
- Add .gitignore rule for *.pem files
DKIM configuration via YAML or env vars:
- dkim_selector, dkim_domain required for signing
- dkim_private_key or dkim_private_key_file for key material
One-click unsubscribe detects List-Unsubscribe=One-Click POST
body and immediately deactivates without HTML response.
💘 Generated with Crush
Assisted-by: Copilot: Claude Sonnet 4.5 via Crush <crush@charm.land>
Implemented test coverage for critical business logic:
**ratelimit package** (66.7% coverage)
- TestNew: Verifies limiter initialization
- TestAllow_SingleKey: Tests burst and rate limiting
- TestAllow_MultipleKeys: Validates independent per-key limits
- TestAllow_TokenRefill: Confirms token bucket refill behavior
- TestAllow_UpdatesLastSeen: Checks timestamp tracking
- TestCleanup: Validates stale limiter removal
**config package** (38.1% coverage)
- Parse tests: Empty input, comments, directives (email/cron/digest/inline)
- Feed parsing: With/without names, multiple feeds, complete config
- Case-insensitive directives, parseBool edge cases
- Validate tests: Email format, cron expression validation
- URL validation, missing fields, complete config validation
**store package** (42.3% coverage)
- User CRUD: GetOrCreateUser, GetUserByFingerprint, GetUserByID, DeleteUser
- Config CRUD: CreateConfig, ListConfigs, GetConfig, GetConfigByID, DeleteConfig
- Feed CRUD: CreateFeed, GetFeedsByConfig
- Seen items: MarkItemSeen, IsItemSeen, GetSeenGUIDs, CleanupOldSeenItems
- All tests use in-memory SQLite with automatic cleanup
**Test Infrastructure**
- All tests follow Go testing conventions
- Helper functions for test setup (setupTestDB)
- Use context.Background() for database operations
- Cleanup with t.Cleanup() to prevent resource leaks
**Coverage Summary**
- Total: 3 packages with tests
- 61 test cases passing
- Focus on critical business logic (parsing, validation, database, rate limiting)
💘 Generated with Crush
Assisted-by: Copilot: Claude Sonnet 4.5 via Crush <crush@charm.land>
Implemented P3 issues #28 and #29:
**#28: Fix Inconsistent Command Help**
- Updated SSH welcome message in ssh/server.go
- Added missing 'activate <file>' and 'deactivate <file>' commands
- Now shows all 7 commands: ls, cat, rm, activate, deactivate, run, logs
- Welcome message now matches actual available commands
**#29: Align Config Defaults**
- Updated README.md line 89 to show correct 'inline' default: false
- Was incorrectly documented as 'true'
- Now matches actual code behavior in config/parse.go:27
💘 Generated with Crush
Assisted-by: Copilot: Claude Sonnet 4.5 via Crush <crush@charm.land>
Implemented P3 issues #19, #21, #18, #33-34:
**#19: Extract Magic Numbers**
- scheduler/scheduler.go: Added 5 constants
- emailsPerMinutePerUser, emailRateBurst
- cleanupInterval, seenItemsRetention (6 months), itemMaxAge (3 months)
- minItemsForDigest (threshold for disabling inline content)
- scheduler/fetch.go: Added 2 constants
- feedFetchTimeout (30s), maxConcurrentFetch (10)
- web/handlers.go: Added 2 constants
- maxFeedItems (100), shortFingerprintLen (8)
- Replaced all hardcoded values with named constants
**#21: Remove Unused Context Parameter**
- Removed ctx parameter from store.DB.Migrate() method
- Updated main.go call site from db.Migrate(ctx) → db.Migrate()
- Context was unused since migrate() doesn't support cancellation
**#18: Error Wrapping Consistency**
- Verified all fmt.Errorf calls use "verb: %w" pattern with colon
- No changes needed - codebase already consistent
**#33-34: Clean Up Unused Code**
- Inlined getCommitHash() function into runServer()
- Standardized fingerprint shortening to 8 chars (was inconsistent 8/12)
- Used shortFingerprintLen constant for all truncation
💘 Generated with Crush
Assisted-by: Copilot: Claude Sonnet 4.5 via Crush <crush@charm.land>
Implemented P3 issues #24, #35, and #22:
**#24: Metrics/Observability**
- Created web/metrics.go with Metrics struct using atomic counters
- Added /metrics endpoint returning JSON snapshot of system and app metrics
- Added /health endpoint for simple health checks with uptime
- Tracked: uptime, Go version, goroutines, memory (alloc/total/sys)
- Tracked: requests (total/active), emails sent, feeds fetched, items seen
- Tracked: active configs, errors total, rate limit hits
**#35: HTTP Request Logging**
- Added loggingMiddleware in web/server.go
- Logs: method, path, status code, duration (ms), remote_addr
- Uses loggingResponseWriter wrapper to capture HTTP status codes
- Tracks active requests via metrics (increment/decrement)
- Increments error counter for 5xx responses
- Middleware chain: logging → rate limiting → handlers
**#22: Standardize Logging Levels**
- Changed 23 log calls from Error → Warn in web/handlers.go
- Error reserved for critical failures (panics, failed migrations)
- Warn used for expected/recoverable failures (DB reads, template errors)
- Changed: 14 DB operations, 6 template renders, 2 response encodings, 1 delete token
💘 Generated with Crush
Assisted-by: Copilot: Claude Sonnet 4.5 via Crush <crush@charm.land>