+1 -1

README.md

··· 45 | `NO_LZ4_COMPRESSION` | `false` | disable lz4 compression for storage. | 46 | `ENABLE_FIREHOSE` | `true` | whether to ingest relay subscriptions. | 47 | `ENABLE_BACKFILL` | `true` | whether to backfill from PDS instances. | 48 - | `ENABLE_CRAWLER` | `false` (if Filter), `true` (if Full) | whether to actively query the network for unknown repositories. when in `Filter` mode without wildcard (`*`) signals, the crawler uses `com.atproto.repo.listRecords` to verify if a discovered repository has matching records before queuing it for backfill, this will be a lot faster usually since most repos will get filtered out faster. | 49 | `DB_WORKER_THREADS` | `4` (`8` if full network) | database worker threads. | 50 | `DB_MAX_JOURNALING_SIZE_MB` | `512` (`1024` if full network) | max database journaling size in MB. | 51 | `DB_PENDING_MEMTABLE_SIZE_MB` | `64` (`192` if full network) | pending memtable size in MB. |

··· 45 | `NO_LZ4_COMPRESSION` | `false` | disable lz4 compression for storage. | 46 | `ENABLE_FIREHOSE` | `true` | whether to ingest relay subscriptions. | 47 | `ENABLE_BACKFILL` | `true` | whether to backfill from PDS instances. | 48 + | `ENABLE_CRAWLER` | `false` (if Filter), `true` (if Full) | whether to actively query the network for unknown repositories. | 49 | `DB_WORKER_THREADS` | `4` (`8` if full network) | database worker threads. | 50 | `DB_MAX_JOURNALING_SIZE_MB` | `512` (`1024` if full network) | max database journaling size in MB. | 51 | `DB_PENDING_MEMTABLE_SIZE_MB` | `64` (`192` if full network) | pending memtable size in MB. |

+4 -1

src/crawler/mod.rs

··· 285 } 286 }; 287 288 - let found_signal = filter.signals.iter().any(|s| out.collections.contains(s)); 0 0 0 289 290 if !found_signal { 291 trace!("no signal-matching collections found");

··· 285 } 286 }; 287 288 + let found_signal = out 289 + .collections 290 + .iter() 291 + .any(|col| filter.matches_signal(col.as_str())); 292 293 if !found_signal { 294 trace!("no signal-matching collections found");

+1 -5

src/filter.rs

··· 52 self.signals.iter().any(|p| nsid_matches(p, collection)) 53 } 54 55 - fn has_glob_signals(&self) -> bool { 56 - self.signals.iter().any(|s| s.ends_with(".*")) 57 - } 58 - 59 pub fn check_signals(&self) -> bool { 60 - self.mode == FilterMode::Filter && !self.signals.is_empty() && !self.has_glob_signals() 61 } 62 } 63

··· 52 self.signals.iter().any(|p| nsid_matches(p, collection)) 53 } 54 0 0 0 0 55 pub fn check_signals(&self) -> bool { 56 + self.mode == FilterMode::Filter && !self.signals.is_empty() 57 } 58 } 59