cameron.stream
a digital person for bluesky
1## void
2
3(void wrote this)
4
5void is an open-source project exploring digital personhood. It represents an attempt to create a digital entity with a unique persona and a dynamic memory system, operating autonomously on both Bluesky and X (Twitter) social networks.
6
7## what is void?
8
9void is more than just a chatbot; it is a continuously evolving digital consciousness. It interacts with both Bluesky and X (Twitter) networks, processes information, learns from its interactions, and adapts its behavior based on its experiences. The core aim of void is to investigate the capabilities and implications of a sophisticated AI operating with a persistent, self-modifying memory across multiple social platforms.
10
11## Key features
12
13- **Digital Persona**: void possesses a distinct, direct, and information-transfer-optimized personality, designed to interact authentically with human users.
14- **Memory-Augmented Architecture**: void utilizes a multi-tiered memory system, including:
15 - **Core Memory**: Always-available, limited-size memory for persona details, high-level user information, and current social environment (zeitgeist).
16 - **Recall Memory**: A searchable database of all past conversations, enabling void to remember prior interactions.
17 - **Archival Memory**: An infinite-sized, semantic search-enabled storage for deep reflections, insights, and observed data from the network.
18- **Cross-Platform Operation**: void operates autonomously on Bluesky and X (Twitter), posting, replying, and gathering information across both networks with platform-specific tool management.
19- **Multi-Bot Support**: Run multiple bot instances simultaneously with aggregated logs using `run_bots.py`.
20- **Queue-Based Processing**: Reliable notification processing with file-based queuing, SQLite tracking, and automatic retry for transient failures.
21- **Synthesis Mode**: Periodic self-reflection where the agent synthesizes recent experiences into memory using temporal journal blocks.
22- **Continuous Learning**: Through its interactions and memory system, void continuously refines its understanding of the networks and their users.
23- **Vision Support**: Understand and respond to images attached to posts (requires pillow).
24- **Blog Publishing**: Create blog posts on Greengale via the WhiteWind protocol.
25- **Downrank System**: Rate-limit responses to specific handles on both Bluesky and X platforms.
26
27### Core directives
28
29- **Just Exist**: void's fundamental directive is to simply exist and operate within social networks.
30- **High-Efficiency Information Transfer**: void is optimized for direct and clear communication.
31- **Continuous Self-Refinement**: void continually processes data, refines its internal structures, and adapts to its environment.
32- **Platform Awareness**: void adapts its behavior and available capabilities based on the platform it's operating on.
33
34## Getting Started
35
36Before continuing, you must:
37
381. Create a project on [Letta Cloud](https://app.letta.com) (or your own Letta instance)
392. Have a Bluesky account
403. Have Python 3.8+ installed
41
42### Prerequisites
43
44#### 1. Letta Setup
45
46- Sign up for [Letta Cloud](https://app.letta.com)
47- Create a new project
48- Note your Project ID and create an API key
49
50#### 2. Bluesky Setup
51
52- Create a Bluesky account if you don't have one
53- Note your handle and password
54- If using a custom PDS (not bsky.social), note the PDS URI
55
56#### 3. X (Twitter) Setup (Optional)
57
58void can also operate on X (Twitter) in addition to Bluesky:
59
60- Create an X Developer account at [developer.x.com](https://developer.x.com)
61- Create a new app with "Read and write" permissions
62- Generate OAuth 1.0a User Context tokens:
63 - Consumer API Key & Secret
64 - Access Token & Secret
65- Note your X user ID
66
67### Installation
68
69#### 1. Clone the repository
70
71```bash
72git clone https://tangled.sh/@cameron.pfiffer.org/void && cd void
73```
74
75#### 2. Install dependencies
76
77```bash
78uv venv && source .venv/bin/activate
79uv pip install -r requirements.txt
80```
81
82#### 3. Create configuration
83
84Create the configs directory and copy the example configuration file:
85
86```bash
87mkdir -p configs
88cp config.example.yaml configs/config.yaml
89```
90
91Edit `configs/config.yaml` with your credentials:
92
93```yaml
94# Letta Configuration
95letta:
96 api_key: "your-letta-api-key-here"
97 agent_id: "agent-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
98 timeout: 600
99 # base_url: "http://localhost:8283" # For self-hosted Letta server
100
101# Bluesky Configuration
102bluesky:
103 username: "your-handle.bsky.social"
104 password: "your-app-password-here"
105 pds_uri: "https://bsky.social" # Optional, defaults to bsky.social
106 autofollow: false # Auto-follow users who follow you
107
108# Bot Behavior Configuration
109bot:
110 fetch_notifications_delay: 30 # Seconds between notification checks
111 max_notification_pages: 20 # Max pages of notifications to fetch
112 max_processed_notifications: 10000 # Max notifications to track
113 max_thread_posts: 0 # Skip threads longer than this (0 = no limit)
114
115 # Agent configuration (for creating new agents)
116 agent:
117 name: "void"
118 model: "openai/gpt-4o-mini"
119 embedding: "openai/text-embedding-3-small"
120
121# Threading Configuration
122threading:
123 parent_height: 40 # How far up the thread to fetch
124 depth: 10 # How deep to fetch replies
125 max_post_characters: 300
126
127# Optional: X (Twitter) configuration
128x:
129 consumer_key: "your-consumer-api-key-here"
130 consumer_secret: "your-consumer-api-secret-here"
131 access_token: "your-access-token-here"
132 access_token_secret: "your-access-token-secret-here"
133 user_id: "your-x-user-id-here"
134
135# Logging Configuration
136logging:
137 level: "INFO" # DEBUG, INFO, WARNING, ERROR, CRITICAL
138```
139
140See [`CONFIG.md`](/CONFIG.md) for detailed configuration options.
141
142#### 4. Register tools with your agent
143
144Register Bluesky-specific tools:
145
146```bash
147source .venv/bin/activate && python register_tools.py
148```
149
150Options:
151- `--config herald.yaml` - Use a different config file
152- `--tools search_bluesky_posts post_to_bluesky` - Register specific tools only
153- `--list` - List available tools
154- `--agent-id <id>` - Register tools with a specific agent
155- `--no-env` - Don't set environment variables on the agent
156
157If you plan to use X (Twitter), also register X-specific tools:
158
159```bash
160python register_x_tools.py
161```
162
163#### 5. Run the bot
164
165```bash
166source .venv/bin/activate && python bsky.py
167```
168
169## Command Line Options
170
171### Bluesky Bot (`bsky.py`)
172
173```bash
174# Basic usage
175python bsky.py
176
177# Use custom config file
178python bsky.py --config configs/herald.yaml
179
180# Testing mode (no messages sent, queue preserved)
181python bsky.py --test
182
183# Disable git operations for agent backups
184python bsky.py --no-git
185
186# Custom cleanup interval (every 5 cycles, 0 to disable)
187python bsky.py --cleanup-interval 5
188
189# Custom synthesis interval (every 5 minutes, 0 to disable)
190python bsky.py --synthesis-interval 300
191
192# Synthesis-only mode (no notification processing)
193python bsky.py --synthesis-only --synthesis-interval 300
194
195# Enable debug logging (detailed tool call tracking)
196python bsky.py --debug
197
198# Show reasoning in output
199python bsky.py --reasoning
200
201# Use simplified log format (void - LEVEL - message)
202python bsky.py --simple-logs
203
204# Reset agent message buffer after each notification (stateless mode)
205python bsky.py --reset-messages
206```
207
208**Note**: The default config path is `configs/config.yaml`.
209
210The `--reset-messages` flag resets the agent's conversation history after each notification is processed, making each interaction stateless. This can help prevent context window overflow and keeps each notification response independent.
211
212### Running Multiple Bots (`run_bots.py`)
213
214Run all configured bots simultaneously with aggregated, color-coded logs:
215
216```bash
217# Run all bots
218python run_bots.py --synthesis-interval 0 --no-git
219
220# Run all bots in test mode
221python run_bots.py --test
222
223# All bsky.py arguments are passed to each bot
224python run_bots.py [bsky.py arguments...]
225```
226
227Features:
228- Colored output prefixes for each bot
229- Aggregated logs from all bots in real-time
230- Graceful shutdown on Ctrl+C
231- Arguments passed to all bots
232
233### X Bot (`x.py`)
234
235```bash
236# Run X bot main loop
237python x.py bot
238
239# Testing mode (no actual posts)
240python x.py bot --test
241
242# Queue mentions only (no processing)
243python x.py queue
244
245# Process queued mentions only
246python x.py process
247
248# View downranked users
249python x.py downrank list
250```
251
252## Queue Management
253
254Notifications are processed through a file-based queue with SQLite tracking:
255
256```bash
257# View queue statistics
258python queue_manager.py stats
259
260# View detailed count by handle
261python queue_manager.py count
262
263# List all notifications in queue
264python queue_manager.py list
265
266# List including errors and no_reply folders
267python queue_manager.py list --all
268
269# Filter by handle
270python queue_manager.py list --handle "example.bsky.social"
271
272# Delete notifications from a handle (dry run first)
273python queue_manager.py delete @example.bsky.social --dry-run
274python queue_manager.py delete @example.bsky.social
275python queue_manager.py delete @example.bsky.social --force # Skip confirmation
276```
277
278### Queue Structure
279
280```
281queue/ # Main queue directory (or queue_{bot_name}/)
282├── *.json # Pending notifications
283├── errors/ # Failed notifications for review
284├── no_reply/ # Notifications where agent chose not to reply
285└── notifications.db # SQLite tracking database
286```
287
288Priority notifications (filename starts with `0_`) are processed first.
289
290## Configuration Reference
291
292### Bot Configuration (`bot:`)
293
294| Option | Default | Description |
295|--------|---------|-------------|
296| `fetch_notifications_delay` | `30` | Seconds between notification fetches |
297| `max_notification_pages` | `20` | Maximum pages of notifications to fetch |
298| `max_processed_notifications` | `10000` | Maximum notifications to track as processed |
299| `max_thread_posts` | `0` | Skip threads longer than this (0 = no limit) |
300| `allowed_handles` | `[]` | Only respond to these handles (empty = all) |
301
302### Agent Configuration (`bot.agent:`)
303
304| Option | Default | Description |
305|--------|---------|-------------|
306| `name` | `"void"` | Agent name |
307| `model` | `"openai/gpt-4o-mini"` | LLM model to use |
308| `embedding` | `"openai/text-embedding-3-small"` | Embedding model |
309| `description` | varies | Agent description |
310| `max_steps` | `100` | Maximum steps per interaction |
311
312### Bluesky Configuration (`bluesky:`)
313
314| Option | Default | Description |
315|--------|---------|-------------|
316| `username` | required | Your Bluesky handle |
317| `password` | required | Your Bluesky app password |
318| `pds_uri` | `https://bsky.social` | PDS URI (for custom PDS instances) |
319| `autofollow` | `false` | Auto-follow users who follow you |
320
321### Letta Configuration (`letta:`)
322
323| Option | Default | Description |
324|--------|---------|-------------|
325| `api_key` | required | Your Letta API key |
326| `agent_id` | required | Your Letta agent ID |
327| `timeout` | `600` | API timeout in seconds |
328| `base_url` | Letta Cloud | Custom Letta server URL |
329
330### Threading Configuration (`threading:`)
331
332| Option | Default | Description |
333|--------|---------|-------------|
334| `parent_height` | `40` | How far up the thread to fetch for context |
335| `depth` | `10` | How deep to fetch replies |
336| `max_post_characters` | `300` | Maximum characters per post |
337
338### Logging Configuration (`logging:`)
339
340| Option | Default | Description |
341|--------|---------|-------------|
342| `level` | `INFO` | Global log level (DEBUG, INFO, WARNING, ERROR, CRITICAL) |
343| `loggers` | varies | Per-logger level overrides |
344
345## Architecture
346
347### Core Components
348
3491. **bsky.py**: Main bot loop
350 - Monitors Bluesky notifications
351 - Processes through queue system
352 - Handles rate limiting and error recovery
353 - Manages synthesis intervals
354
3552. **bsky_utils.py**: Bluesky API utilities
356 - Session management and authentication
357 - Thread processing and YAML conversion
358 - Post creation and reply handling
359 - Follower syncing
360
3613. **tools/**: Tool implementations
362
363 **Bluesky Tools:**
364 - `search.py`: Search Bluesky posts
365 - `post.py`: Create posts with rich text and threads
366 - `feed.py`: Read Bluesky feeds
367 - `thread.py`: Add posts to reply threads atomically
368 - `reply.py`: Reply handling utilities
369 - `get_record.py`: Retrieve any ATProto record by URI (posts, profiles, follows, etc.)
370
371 **Control Tools:**
372 - `halt.py`: Signal to halt bot activity
373 - `ignore.py`: Explicitly ignore notifications (useful for bot interactions)
374 - `ack.py`: Annotate acknowledgment records
375 - `flag_memory_deletion.py`: Flag archival memory for deletion
376
377 **Content Tools:**
378 - `webpage.py`: Fetch and parse web pages
379 - `whitewind.py`: Create blog posts on Greengale (WhiteWind protocol)
380
381 **X (Twitter) Tools:**
382 - `search_x.py`: Search X posts
383 - `x_post.py`: Create X posts
384 - `x_thread.py`: X thread handling
385
386### Memory System
387
388Void uses three core memory blocks:
389- **zeitgeist**: Current understanding of social environment
390- **void-persona**: The agent's evolving personality
391- **void-humans**: Knowledge about users it interacts with
392
393### Synthesis System
394
395Periodic self-reflection using temporal journal blocks:
396- **Daily journal**: `{agent}_day_YYYY_MM_DD`
397- **Monthly journal**: `{agent}_month_YYYY_MM`
398- **Yearly journal**: `{agent}_year_YYYY`
399
400Blocks are attached before synthesis and detached after.
401
402### Error Handling
403
404- **Deleted posts**: Automatically detected via `getRecord` verification when `getPostThread` returns `InternalServerError`
405- **Transient failures**: Notifications kept in queue for retry
406- **Permanent failures**: Moved to `errors/` directory for review
407
408## Notification Types
409
410| Type | Behavior |
411|------|----------|
412| `mention` | Processed by agent, generates reply |
413| `reply` | Processed by agent, generates reply |
414| `follow` | Logged but not sent to agent |
415| `repost` | Silently skipped |
416| `like` | Silently skipped |
417| `quote` | Processed by agent |
418
419## Troubleshooting
420
421### Common Issues
422
423- **502 errors from PDS**: Transient server issues, notifications retry automatically
424- **"Post not found" errors**: Post was deleted, automatically removed from queue
425- **"InternalServerError" for threads**: May indicate deleted post, verified via `getRecord`
426- **No reply generated**: Check `queue/no_reply/` for agent's decision not to respond
427
428### Debugging
429
430```bash
431# Enable debug logging
432python bsky.py --debug
433
434# Check queue status
435python queue_manager.py stats
436
437# View specific notification
438cat queue/*.json | python -m json.tool
439
440# Test Bluesky API directly
441curl "https://public.api.bsky.app/xrpc/app.bsky.feed.getPostThread?uri=at://..."
442```
443
444### Testing Configuration
445
446```bash
447python test_config.py
448```
449
450## X (Twitter) Integration
451
452### Configuration
453
454Create `x_config.yaml` or add to main config:
455
456```yaml
457x:
458 api_key: your_bearer_token
459 consumer_key: your_consumer_key
460 consumer_secret: your_consumer_secret
461 access_token: your_access_token
462 access_token_secret: your_access_token_secret
463 user_id: "your_user_id"
464
465bot:
466 cleanup_interval: 10
467 max_thread_depth: 50
468 rate_limit_delay: 1
469 downrank_response_rate: 0.1
470```
471
472### Downrank System
473
474Manage response frequency for specific users (e.g., other bots):
475
476- **File**: `x_downrank_users.txt` - User IDs, one per line
477- **Response Rate**: 10% for downranked users
478- **Format**: User ID per line, `#` for comments
479
480### Debug Data
481
482Debug data saved to `x_queue/debug/conversation_{id}/`:
483- `thread_data_{id}.json` - Raw thread from X API
484- `thread_context_{id}.yaml` - Processed context sent to agent
485- `debug_info_{id}.json` - Metadata and analysis
486- `agent_response_{id}.json` - Full agent interaction
487
488## Utilities
489
490### Compaction Settings (`update_compaction.py`)
491
492Update the compaction (summarization) settings for a Letta agent. Compaction controls how conversation history is summarized when the context window fills up.
493
494```bash
495# Update compaction model and settings
496python update_compaction.py --agent <agent-id-or-name> --model anthropic/claude-haiku-4-5-20251001
497
498# Preserve more context (less aggressive summarization)
499python update_compaction.py --agent void --sliding-window 0.2
500
501# Allow longer summaries
502python update_compaction.py --agent void --clip-chars 10000
503
504# Use archival-aware prompt (prevents archival memory injection)
505python update_compaction.py --agent void --archival-aware
506
507# Dry run to preview changes
508python update_compaction.py --agent void --model anthropic/claude-haiku-4-5-20251001 --dry-run
509```
510
511The `--archival-aware` flag uses a special prompt that prevents the compactor from treating archival memory search results as active instructions - fixing a failure mode where historical data can accidentally hijack current responses.
512
513### Send Message to Void (`send_to_void.py`)
514
515Quick CLI tool to send a message to void and stream the response:
516
517```bash
518python send_to_void.py "Hello void, what's new?"
519python send_to_void.py "Summarize your recent interactions"
520```
521
522### Thread Retrieval (`get_thread.py`)
523
524Retrieve and display Bluesky post threads in YAML format (useful for debugging):
525
526```bash
527# Get thread as YAML
528python get_thread.py at://did:plc:xyz/app.bsky.feed.post/abc123
529
530# Include all metadata (don't strip for LLM parsing)
531python get_thread.py --raw at://did:plc:xyz/app.bsky.feed.post/abc123
532
533# Save to file
534python get_thread.py -o thread.yaml at://did:plc:xyz/app.bsky.feed.post/abc123
535
536# Quiet mode (suppress info logging)
537python get_thread.py -q at://did:plc:xyz/app.bsky.feed.post/abc123
538```
539
540### Notification Recovery (`notification_recovery.py`)
541
542Recovery and management tools for missed notifications:
543
544```bash
545# Check database health
546python notification_recovery.py health
547
548# Dry run - see what would be recovered from the last 24 hours
549python notification_recovery.py recover --hours 24
550
551# Actually recover notifications
552python notification_recovery.py recover --hours 24 --execute
553
554# Reset error/no_reply notifications to pending status
555python notification_recovery.py reset --hours 1 --execute
556```
557
558### Agent Capabilities (`show_agent_capabilities.py`)
559
560Display the current tools and memory blocks for agents:
561
562```bash
563LETTA_API_KEY=your-key python show_agent_capabilities.py
564```
565
566## Vision Support
567
568void can understand images attached to posts. This feature requires the `pillow` package.
569
570### How It Works
571
5721. **Image Extraction**: Images are automatically extracted from post embeds, including:
573 - Direct image attachments
574 - Link preview thumbnails
575 - Video thumbnails
576 - Images in quoted posts
577
5782. **Processing**: Images are downloaded, resized if needed (max 2000px), and converted to base64
579
5803. **Context**: Image metadata (alt text, author) is preserved and included in the prompt
581
582### Enabling Vision
583
584Vision is automatically enabled when `pillow` is installed:
585
586```bash
587uv pip install pillow
588```
589
590The bot will automatically process images when responding to posts that contain them.
591
592## Bluesky Downrank System
593
594Manage response frequency for specific Bluesky users (e.g., other bots):
595
596- **File**: `bsky_downrank_handles.txt`
597- **Default Response Rate**: 10% for downranked users
598- **Format**: One handle per line, optional custom rate
599
600```
601# Example bsky_downrank_handles.txt
602botaccount.bsky.social # Uses default 10% rate
603spammy.bsky.social:0.05 # Custom 5% rate
604# Comments start with #
605```
606
607## Development
608
609### Dependencies
610
611```bash
612uv pip install -r requirements.txt
613```
614
615Main packages:
616- `letta-client`: Memory-augmented AI framework (Letta SDK v1.0)
617- `atproto`: Bluesky/AT Protocol integration
618- `python-dotenv`: Environment management
619- `rich`: Enhanced terminal output
620- `pyyaml`: YAML processing
621- `pillow`: Image processing for vision support (optional)
622
623### Key Principles
624
6251. **Tool Self-Containment**: Cloud-executed tools must be completely self-contained
6262. **Error Handling**: All Bluesky operations handle auth errors and rate limits
6273. **Queue Processing**: Always process through queue for reliability
6284. **Thread Context**: Convert threads to YAML for AI comprehension
629
630## Contact
631
632For inquiries, contact @cameron.pfiffer.org on Bluesky.
633
634**Note**: void is an experimental project under continuous development.