Our Personal Data Server from scratch! tranquil.farm
oauth atproto pds rust postgresql objectstorage fun

feat: nix module #16

merged opened by lewis.moe targeting main from feat/nix-module

Added a module and a test file to test it out! Tested locally on qemu

Labels

None yet.

assignee

None yet.

Participants 2
Referenced by
AT URI
at://did:plc:3fwecdnvtcscjnrx2p4n7alz/sh.tangled.repo.pull/3me4wjqzoiv22
+507 -147
Interdiff #0 โ†’ #1
+1 -1
flake.nix
··· 30 default = pkgs.callPackage ./shell.nix {}; 31 }); 32 33 - nixosModules.default = import ./module.nix; 34 35 checks.x86_64-linux.integration = import ./test.nix { 36 pkgs = nixpkgs.legacyPackages.x86_64-linux;
··· 30 default = pkgs.callPackage ./shell.nix {}; 31 }); 32 33 + nixosModules.default = import ./module.nix self; 34 35 checks.x86_64-linux.integration = import ./test.nix { 36 pkgs = nixpkgs.legacyPackages.x86_64-linux;
+285 -118
module.nix
··· 1 - { 2 config, 3 lib, 4 pkgs, ··· 8 9 optionalStr = lib.types.nullOr lib.types.str; 10 optionalInt = lib.types.nullOr lib.types.int; 11 - optionalBool = lib.types.nullOr lib.types.bool; 12 optionalPath = lib.types.nullOr lib.types.str; 13 - optionalPort = lib.types.nullOr lib.types.port; 14 15 filterNulls = lib.filterAttrs (_: v: v != null); 16 17 boolToStr = b: 18 - if b == true 19 then "true" 20 - else if b == false 21 - then "false" 22 - else null; 23 24 settingsToEnv = settings: let 25 raw = { 26 SERVER_HOST = settings.server.host; ··· 38 AWS_REGION = settings.storage.awsRegion; 39 S3_BUCKET = settings.storage.s3Bucket; 40 41 - BACKUP_ENABLED = boolToStr settings.backup.enabled; 42 BACKUP_STORAGE_BACKEND = settings.backup.backend; 43 BACKUP_STORAGE_PATH = settings.backup.path; 44 BACKUP_S3_BUCKET = settings.backup.s3Bucket; ··· 94 PDS_AGE_ASSURANCE_OVERRIDE = boolToStr settings.misc.ageAssuranceOverride; 95 ALLOW_HTTP_PROXY = boolToStr settings.misc.allowHttpProxy; 96 97 - SSO_GITHUB_ENABLED = boolToStr settings.sso.github.enabled; 98 SSO_GITHUB_CLIENT_ID = settings.sso.github.clientId; 99 100 - SSO_DISCORD_ENABLED = boolToStr settings.sso.discord.enabled; 101 SSO_DISCORD_CLIENT_ID = settings.sso.discord.clientId; 102 103 - SSO_GOOGLE_ENABLED = boolToStr settings.sso.google.enabled; 104 SSO_GOOGLE_CLIENT_ID = settings.sso.google.clientId; 105 106 - SSO_GITLAB_ENABLED = boolToStr settings.sso.gitlab.enabled; 107 SSO_GITLAB_CLIENT_ID = settings.sso.gitlab.clientId; 108 SSO_GITLAB_ISSUER = settings.sso.gitlab.issuer; 109 110 - SSO_OIDC_ENABLED = boolToStr settings.sso.oidc.enabled; 111 SSO_OIDC_CLIENT_ID = settings.sso.oidc.clientId; 112 SSO_OIDC_ISSUER = settings.sso.oidc.issuer; 113 SSO_OIDC_NAME = settings.sso.oidc.name; 114 115 - SSO_APPLE_ENABLED = boolToStr settings.sso.apple.enabled; 116 SSO_APPLE_CLIENT_ID = settings.sso.apple.clientId; 117 SSO_APPLE_TEAM_ID = settings.sso.apple.teamId; 118 SSO_APPLE_KEY_ID = settings.sso.apple.keyId; ··· 123 options.services.tranquil-pds = { 124 enable = lib.mkEnableOption "tranquil-pds AT Protocol personal data server"; 125 126 - package = lib.mkPackageOption pkgs "tranquil-pds" {}; 127 128 user = lib.mkOption { 129 type = lib.types.str; ··· 155 ''; 156 }; 157 158 settings = { 159 server = { 160 host = lib.mkOption { ··· 182 }; 183 184 maxConnections = lib.mkOption { 185 - type = optionalInt; 186 - default = null; 187 description = "Maximum database connections"; 188 }; 189 190 minConnections = lib.mkOption { 191 - type = optionalInt; 192 - default = null; 193 description = "Minimum database connections"; 194 }; 195 196 acquireTimeoutSecs = lib.mkOption { 197 - type = optionalInt; 198 - default = null; 199 description = "Connection acquire timeout in seconds"; 200 }; 201 }; ··· 208 }; 209 210 blobPath = lib.mkOption { 211 - type = optionalPath; 212 - default = null; 213 description = "Path for filesystem blob storage"; 214 }; 215 ··· 233 }; 234 235 backup = { 236 - enabled = lib.mkOption { 237 - type = optionalBool; 238 - default = null; 239 - description = "Enable automatic repo backups"; 240 - }; 241 242 backend = lib.mkOption { 243 type = lib.types.enum ["filesystem" "s3"]; ··· 246 }; 247 248 path = lib.mkOption { 249 - type = optionalPath; 250 - default = null; 251 description = "Path for filesystem backup storage"; 252 }; 253 ··· 258 }; 259 260 retentionCount = lib.mkOption { 261 - type = optionalInt; 262 - default = null; 263 description = "Number of backups to retain"; 264 }; 265 266 intervalSecs = lib.mkOption { 267 - type = optionalInt; 268 - default = null; 269 description = "Backup interval in seconds"; 270 }; 271 }; ··· 280 281 security = { 282 allowInsecureSecrets = lib.mkOption { 283 - type = optionalBool; 284 - default = null; 285 description = "Allow default/weak secrets (development only, NEVER in production ofc)"; 286 }; 287 }; 288 289 plc = { 290 directoryUrl = lib.mkOption { 291 - type = optionalStr; 292 - default = null; 293 description = "PLC directory URL"; 294 }; 295 296 timeoutSecs = lib.mkOption { 297 - type = optionalInt; 298 - default = null; 299 description = "PLC request timeout in seconds"; 300 }; 301 302 connectTimeoutSecs = lib.mkOption { 303 - type = optionalInt; 304 - default = null; 305 description = "PLC connection timeout in seconds"; 306 }; 307 ··· 314 315 did = { 316 cacheTtlSecs = lib.mkOption { 317 - type = optionalInt; 318 - default = null; 319 description = "DID document cache TTL in seconds"; 320 }; 321 }; ··· 330 331 firehose = { 332 bufferSize = lib.mkOption { 333 - type = optionalInt; 334 - default = null; 335 description = "Firehose broadcast channel buffer size"; 336 }; 337 ··· 344 345 notifications = { 346 batchSize = lib.mkOption { 347 - type = optionalInt; 348 - default = null; 349 description = "Notification queue batch size"; 350 }; 351 352 pollIntervalMs = lib.mkOption { 353 - type = optionalInt; 354 - default = null; 355 description = "Notification queue poll interval in ms"; 356 }; 357 ··· 388 389 limits = { 390 maxBlobSize = lib.mkOption { 391 - type = optionalInt; 392 - default = null; 393 description = "Maximum blob size in bytes"; 394 }; 395 }; 396 397 import = { 398 accepting = lib.mkOption { 399 - type = optionalBool; 400 - default = null; 401 description = "Accept repository imports"; 402 }; 403 404 maxSize = lib.mkOption { 405 - type = optionalInt; 406 - default = null; 407 description = "Maximum import size in bytes"; 408 }; 409 410 maxBlocks = lib.mkOption { 411 - type = optionalInt; 412 - default = null; 413 description = "Maximum blocks per import"; 414 }; 415 416 skipVerification = lib.mkOption { 417 - type = optionalBool; 418 - default = null; 419 description = "Skip verification during import (testing only)"; 420 }; 421 }; 422 423 registration = { 424 inviteCodeRequired = lib.mkOption { 425 - type = optionalBool; 426 - default = null; 427 description = "Require invite codes for registration"; 428 }; 429 ··· 434 }; 435 436 enableSelfHostedDidWeb = lib.mkOption { 437 - type = optionalBool; 438 - default = null; 439 description = "Enable self-hosted did:web identities"; 440 }; 441 }; ··· 462 463 rateLimiting = { 464 disable = lib.mkOption { 465 - type = optionalBool; 466 - default = null; 467 description = "Disable rate limiting (testing only, NEVER in production you naughty!)"; 468 }; 469 }; 470 471 scheduling = { 472 deleteCheckIntervalSecs = lib.mkOption { 473 - type = optionalInt; 474 - default = null; 475 description = "Scheduled deletion check interval in seconds"; 476 }; 477 }; ··· 492 493 misc = { 494 ageAssuranceOverride = lib.mkOption { 495 - type = optionalBool; 496 - default = null; 497 description = "Override age assurance checks"; 498 }; 499 500 allowHttpProxy = lib.mkOption { 501 - type = optionalBool; 502 - default = null; 503 description = "Allow HTTP for proxy requests (development only)"; 504 }; 505 }; 506 507 sso = { 508 github = { 509 - enabled = lib.mkOption { 510 - type = optionalBool; 511 - default = null; 512 description = "Enable GitHub SSO"; 513 }; 514 ··· 520 }; 521 522 discord = { 523 - enabled = lib.mkOption { 524 - type = optionalBool; 525 - default = null; 526 description = "Enable Discord SSO"; 527 }; 528 ··· 534 }; 535 536 google = { 537 - enabled = lib.mkOption { 538 - type = optionalBool; 539 - default = null; 540 description = "Enable Google SSO"; 541 }; 542 ··· 548 }; 549 550 gitlab = { 551 - enabled = lib.mkOption { 552 - type = optionalBool; 553 - default = null; 554 description = "Enable GitLab SSO"; 555 }; 556 ··· 568 }; 569 570 oidc = { 571 - enabled = lib.mkOption { 572 - type = optionalBool; 573 - default = null; 574 description = "Enable generic OIDC SSO"; 575 }; 576 ··· 594 }; 595 596 apple = { 597 - enabled = lib.mkOption { 598 - type = optionalBool; 599 - default = null; 600 description = "Enable Apple Sign-in"; 601 }; 602 ··· 622 }; 623 }; 624 625 - config = let 626 - effectiveBlobPath = 627 - if cfg.settings.storage.blobPath != null 628 - then cfg.settings.storage.blobPath 629 - else "${cfg.dataDir}/blobs"; 630 - effectiveBackupPath = 631 - if cfg.settings.backup.path != null 632 - then cfg.settings.backup.path 633 - else "${cfg.dataDir}/backups"; 634 - envVars = 635 - (settingsToEnv cfg.settings) 636 - // { 637 - BLOB_STORAGE_PATH = effectiveBlobPath; 638 - BACKUP_STORAGE_PATH = effectiveBackupPath; 639 }; 640 - in 641 - lib.mkIf cfg.enable { 642 - assertions = [ 643 - { 644 - assertion = config.systemd.enable or true; 645 - message = "services.tranquil-pds requires systemd"; 646 - } 647 - ]; 648 649 users.users.${cfg.user} = { 650 isSystemUser = true; 651 inherit (cfg) group; ··· 656 657 systemd.tmpfiles.rules = [ 658 "d ${cfg.dataDir} 0750 ${cfg.user} ${cfg.group} -" 659 - "d ${effectiveBlobPath} 0750 ${cfg.user} ${cfg.group} -" 660 - "d ${effectiveBackupPath} 0750 ${cfg.user} ${cfg.group} -" 661 ]; 662 663 systemd.services.tranquil-pds = { ··· 666 wants = ["network.target"]; 667 wantedBy = ["multi-user.target"]; 668 669 - environment = envVars; 670 671 serviceConfig = { 672 Type = "exec"; ··· 698 RemoveIPC = true; 699 700 ReadWritePaths = [ 701 - effectiveBlobPath 702 - effectiveBackupPath 703 ]; 704 }; 705 }; 706 - }; 707 }
··· 1 + self: { 2 config, 3 lib, 4 pkgs, ··· 8 9 optionalStr = lib.types.nullOr lib.types.str; 10 optionalInt = lib.types.nullOr lib.types.int; 11 optionalPath = lib.types.nullOr lib.types.str; 12 13 filterNulls = lib.filterAttrs (_: v: v != null); 14 15 boolToStr = b: 16 + if b 17 then "true" 18 + else "false"; 19 20 + backendUrl = "http://127.0.0.1:${toString cfg.settings.server.port}"; 21 + 22 + useACME = cfg.nginx.enableACME && cfg.nginx.useACMEHost == null; 23 + hasSSL = useACME || cfg.nginx.useACMEHost != null; 24 + 25 settingsToEnv = settings: let 26 raw = { 27 SERVER_HOST = settings.server.host; ··· 39 AWS_REGION = settings.storage.awsRegion; 40 S3_BUCKET = settings.storage.s3Bucket; 41 42 + BACKUP_ENABLED = boolToStr settings.backup.enable; 43 BACKUP_STORAGE_BACKEND = settings.backup.backend; 44 BACKUP_STORAGE_PATH = settings.backup.path; 45 BACKUP_S3_BUCKET = settings.backup.s3Bucket; ··· 95 PDS_AGE_ASSURANCE_OVERRIDE = boolToStr settings.misc.ageAssuranceOverride; 96 ALLOW_HTTP_PROXY = boolToStr settings.misc.allowHttpProxy; 97 98 + SSO_GITHUB_ENABLED = boolToStr settings.sso.github.enable; 99 SSO_GITHUB_CLIENT_ID = settings.sso.github.clientId; 100 101 + SSO_DISCORD_ENABLED = boolToStr settings.sso.discord.enable; 102 SSO_DISCORD_CLIENT_ID = settings.sso.discord.clientId; 103 104 + SSO_GOOGLE_ENABLED = boolToStr settings.sso.google.enable; 105 SSO_GOOGLE_CLIENT_ID = settings.sso.google.clientId; 106 107 + SSO_GITLAB_ENABLED = boolToStr settings.sso.gitlab.enable; 108 SSO_GITLAB_CLIENT_ID = settings.sso.gitlab.clientId; 109 SSO_GITLAB_ISSUER = settings.sso.gitlab.issuer; 110 111 + SSO_OIDC_ENABLED = boolToStr settings.sso.oidc.enable; 112 SSO_OIDC_CLIENT_ID = settings.sso.oidc.clientId; 113 SSO_OIDC_ISSUER = settings.sso.oidc.issuer; 114 SSO_OIDC_NAME = settings.sso.oidc.name; 115 116 + SSO_APPLE_ENABLED = boolToStr settings.sso.apple.enable; 117 SSO_APPLE_CLIENT_ID = settings.sso.apple.clientId; 118 SSO_APPLE_TEAM_ID = settings.sso.apple.teamId; 119 SSO_APPLE_KEY_ID = settings.sso.apple.keyId; ··· 124 options.services.tranquil-pds = { 125 enable = lib.mkEnableOption "tranquil-pds AT Protocol personal data server"; 126 127 + package = lib.mkOption { 128 + type = lib.types.package; 129 + default = self.packages.${pkgs.stdenv.hostPlatform.system}.tranquil-pds; 130 + defaultText = lib.literalExpression "self.packages.\${pkgs.stdenv.hostPlatform.system}.tranquil-pds"; 131 + description = "The tranquil-pds package to use"; 132 + }; 133 134 user = lib.mkOption { 135 type = lib.types.str; ··· 161 ''; 162 }; 163 164 + database.createLocally = lib.mkOption { 165 + type = lib.types.bool; 166 + default = false; 167 + description = '' 168 + Create the postgres database and user on the local host. 169 + ''; 170 + }; 171 + 172 + frontend.package = lib.mkOption { 173 + type = lib.types.nullOr lib.types.package; 174 + default = self.packages.${pkgs.stdenv.hostPlatform.system}.tranquil-frontend; 175 + defaultText = lib.literalExpression "self.packages.\${pkgs.stdenv.hostPlatform.system}.tranquil-frontend"; 176 + description = "Frontend package to serve via nginx (set null to disable frontend)"; 177 + }; 178 + 179 + nginx = { 180 + enable = lib.mkEnableOption "nginx reverse proxy for tranquil-pds"; 181 + 182 + enableACME = lib.mkOption { 183 + type = lib.types.bool; 184 + default = true; 185 + description = "Enable ACME for the pds domain"; 186 + }; 187 + 188 + useACMEHost = lib.mkOption { 189 + type = lib.types.nullOr lib.types.str; 190 + default = null; 191 + description = '' 192 + Use a pre-configured ACME certificate instead of generating one. 193 + Set this to the cert name from security.acme.certs for wildcard setups. 194 + REMEMBER: Handle subdomains (*.pds.example.com) require a wildcard cert via DNS-01. 195 + ''; 196 + }; 197 + 198 + openFirewall = lib.mkOption { 199 + type = lib.types.bool; 200 + default = true; 201 + description = "Open ports 80 and 443 in the firewall"; 202 + }; 203 + }; 204 + 205 settings = { 206 server = { 207 host = lib.mkOption { ··· 229 }; 230 231 maxConnections = lib.mkOption { 232 + type = lib.types.int; 233 + default = 100; 234 description = "Maximum database connections"; 235 }; 236 237 minConnections = lib.mkOption { 238 + type = lib.types.int; 239 + default = 10; 240 description = "Minimum database connections"; 241 }; 242 243 acquireTimeoutSecs = lib.mkOption { 244 + type = lib.types.int; 245 + default = 10; 246 description = "Connection acquire timeout in seconds"; 247 }; 248 }; ··· 255 }; 256 257 blobPath = lib.mkOption { 258 + type = lib.types.str; 259 + default = "${cfg.dataDir}/blobs"; 260 + defaultText = lib.literalExpression ''"''${cfg.dataDir}/blobs"''; 261 description = "Path for filesystem blob storage"; 262 }; 263 ··· 281 }; 282 283 backup = { 284 + enable = lib.mkEnableOption "automatic repo backups"; 285 286 backend = lib.mkOption { 287 type = lib.types.enum ["filesystem" "s3"]; ··· 290 }; 291 292 path = lib.mkOption { 293 + type = lib.types.str; 294 + default = "${cfg.dataDir}/backups"; 295 + defaultText = lib.literalExpression ''"''${cfg.dataDir}/backups"''; 296 description = "Path for filesystem backup storage"; 297 }; 298 ··· 303 }; 304 305 retentionCount = lib.mkOption { 306 + type = lib.types.int; 307 + default = 7; 308 description = "Number of backups to retain"; 309 }; 310 311 intervalSecs = lib.mkOption { 312 + type = lib.types.int; 313 + default = 86400; 314 description = "Backup interval in seconds"; 315 }; 316 }; ··· 325 326 security = { 327 allowInsecureSecrets = lib.mkOption { 328 + type = lib.types.bool; 329 + default = false; 330 description = "Allow default/weak secrets (development only, NEVER in production ofc)"; 331 }; 332 }; 333 334 plc = { 335 directoryUrl = lib.mkOption { 336 + type = lib.types.str; 337 + default = "https://plc.directory"; 338 description = "PLC directory URL"; 339 }; 340 341 timeoutSecs = lib.mkOption { 342 + type = lib.types.int; 343 + default = 10; 344 description = "PLC request timeout in seconds"; 345 }; 346 347 connectTimeoutSecs = lib.mkOption { 348 + type = lib.types.int; 349 + default = 5; 350 description = "PLC connection timeout in seconds"; 351 }; 352 ··· 359 360 did = { 361 cacheTtlSecs = lib.mkOption { 362 + type = lib.types.int; 363 + default = 300; 364 description = "DID document cache TTL in seconds"; 365 }; 366 }; ··· 375 376 firehose = { 377 bufferSize = lib.mkOption { 378 + type = lib.types.int; 379 + default = 10000; 380 description = "Firehose broadcast channel buffer size"; 381 }; 382 ··· 389 390 notifications = { 391 batchSize = lib.mkOption { 392 + type = lib.types.int; 393 + default = 100; 394 description = "Notification queue batch size"; 395 }; 396 397 pollIntervalMs = lib.mkOption { 398 + type = lib.types.int; 399 + default = 1000; 400 description = "Notification queue poll interval in ms"; 401 }; 402 ··· 433 434 limits = { 435 maxBlobSize = lib.mkOption { 436 + type = lib.types.int; 437 + default = 10737418240; 438 description = "Maximum blob size in bytes"; 439 }; 440 }; 441 442 import = { 443 accepting = lib.mkOption { 444 + type = lib.types.bool; 445 + default = true; 446 description = "Accept repository imports"; 447 }; 448 449 maxSize = lib.mkOption { 450 + type = lib.types.int; 451 + default = 1073741824; 452 description = "Maximum import size in bytes"; 453 }; 454 455 maxBlocks = lib.mkOption { 456 + type = lib.types.int; 457 + default = 500000; 458 description = "Maximum blocks per import"; 459 }; 460 461 skipVerification = lib.mkOption { 462 + type = lib.types.bool; 463 + default = false; 464 description = "Skip verification during import (testing only)"; 465 }; 466 }; 467 468 registration = { 469 inviteCodeRequired = lib.mkOption { 470 + type = lib.types.bool; 471 + default = false; 472 description = "Require invite codes for registration"; 473 }; 474 ··· 479 }; 480 481 enableSelfHostedDidWeb = lib.mkOption { 482 + type = lib.types.bool; 483 + default = true; 484 description = "Enable self-hosted did:web identities"; 485 }; 486 }; ··· 507 508 rateLimiting = { 509 disable = lib.mkOption { 510 + type = lib.types.bool; 511 + default = false; 512 description = "Disable rate limiting (testing only, NEVER in production you naughty!)"; 513 }; 514 }; 515 516 scheduling = { 517 deleteCheckIntervalSecs = lib.mkOption { 518 + type = lib.types.int; 519 + default = 3600; 520 description = "Scheduled deletion check interval in seconds"; 521 }; 522 }; ··· 537 538 misc = { 539 ageAssuranceOverride = lib.mkOption { 540 + type = lib.types.bool; 541 + default = false; 542 description = "Override age assurance checks"; 543 }; 544 545 allowHttpProxy = lib.mkOption { 546 + type = lib.types.bool; 547 + default = false; 548 description = "Allow HTTP for proxy requests (development only)"; 549 }; 550 }; 551 552 sso = { 553 github = { 554 + enable = lib.mkOption { 555 + type = lib.types.bool; 556 + default = false; 557 description = "Enable GitHub SSO"; 558 }; 559 ··· 565 }; 566 567 discord = { 568 + enable = lib.mkOption { 569 + type = lib.types.bool; 570 + default = false; 571 description = "Enable Discord SSO"; 572 }; 573 ··· 579 }; 580 581 google = { 582 + enable = lib.mkOption { 583 + type = lib.types.bool; 584 + default = false; 585 description = "Enable Google SSO"; 586 }; 587 ··· 593 }; 594 595 gitlab = { 596 + enable = lib.mkOption { 597 + type = lib.types.bool; 598 + default = false; 599 description = "Enable GitLab SSO"; 600 }; 601 ··· 613 }; 614 615 oidc = { 616 + enable = lib.mkOption { 617 + type = lib.types.bool; 618 + default = false; 619 description = "Enable generic OIDC SSO"; 620 }; 621 ··· 639 }; 640 641 apple = { 642 + enable = lib.mkOption { 643 + type = lib.types.bool; 644 + default = false; 645 description = "Enable Apple Sign-in"; 646 }; 647 ··· 667 }; 668 }; 669 670 + config = lib.mkIf cfg.enable (lib.mkMerge [ 671 + (lib.mkIf (cfg.settings.notifications.mailFromAddress != null) { 672 + services.tranquil-pds.settings.notifications.sendmailPath = 673 + lib.mkDefault "/run/wrappers/bin/sendmail"; 674 + }) 675 + 676 + (lib.mkIf (cfg.settings.notifications.signalSenderNumber != null) { 677 + services.tranquil-pds.settings.notifications.signalCliPath = 678 + lib.mkDefault (lib.getExe pkgs.signal-cli); 679 + }) 680 + 681 + (lib.mkIf cfg.database.createLocally { 682 + services.postgresql = { 683 + enable = true; 684 + ensureDatabases = [cfg.user]; 685 + ensureUsers = [ 686 + { 687 + name = cfg.user; 688 + ensureDBOwnership = true; 689 + } 690 + ]; 691 }; 692 693 + services.tranquil-pds.settings.database.url = 694 + lib.mkDefault "postgresql:///${cfg.user}?host=/run/postgresql"; 695 + 696 + systemd.services.tranquil-pds = { 697 + requires = ["postgresql.service"]; 698 + after = ["postgresql.service"]; 699 + }; 700 + }) 701 + 702 + (lib.mkIf cfg.nginx.enable (lib.mkMerge [ 703 + { 704 + services.nginx = { 705 + enable = true; 706 + recommendedProxySettings = lib.mkDefault true; 707 + recommendedTlsSettings = lib.mkDefault true; 708 + recommendedGzipSettings = lib.mkDefault true; 709 + recommendedOptimisation = lib.mkDefault true; 710 + 711 + virtualHosts.${cfg.settings.server.pdsHostname} = { 712 + serverAliases = ["*.${cfg.settings.server.pdsHostname}"]; 713 + forceSSL = hasSSL; 714 + enableACME = useACME; 715 + useACMEHost = cfg.nginx.useACMEHost; 716 + 717 + root = lib.mkIf (cfg.frontend.package != null) "${cfg.frontend.package}"; 718 + 719 + extraConfig = "client_max_body_size ${toString cfg.settings.limits.maxBlobSize};"; 720 + 721 + locations = let 722 + proxyLocations = { 723 + "/xrpc/" = { 724 + proxyPass = backendUrl; 725 + proxyWebsockets = true; 726 + extraConfig = '' 727 + proxy_read_timeout 86400; 728 + proxy_send_timeout 86400; 729 + proxy_buffering off; 730 + proxy_request_buffering off; 731 + ''; 732 + }; 733 + 734 + "/oauth/" = { 735 + proxyPass = backendUrl; 736 + extraConfig = '' 737 + proxy_read_timeout 300; 738 + proxy_send_timeout 300; 739 + ''; 740 + }; 741 + 742 + "/.well-known/" = { 743 + proxyPass = backendUrl; 744 + }; 745 + 746 + "/webhook/" = { 747 + proxyPass = backendUrl; 748 + }; 749 + 750 + "= /metrics" = { 751 + proxyPass = backendUrl; 752 + }; 753 + 754 + "= /health" = { 755 + proxyPass = backendUrl; 756 + }; 757 + 758 + "= /robots.txt" = { 759 + proxyPass = backendUrl; 760 + }; 761 + 762 + "= /logo" = { 763 + proxyPass = backendUrl; 764 + }; 765 + 766 + "~ ^/u/[^/]+/did\\.json$" = { 767 + proxyPass = backendUrl; 768 + }; 769 + }; 770 + 771 + frontendLocations = lib.optionalAttrs (cfg.frontend.package != null) { 772 + "= /oauth/client-metadata.json" = { 773 + root = "${cfg.frontend.package}"; 774 + extraConfig = '' 775 + default_type application/json; 776 + sub_filter_once off; 777 + sub_filter_types application/json; 778 + sub_filter '__PDS_HOSTNAME__' $host; 779 + ''; 780 + }; 781 + 782 + "/assets/" = { 783 + extraConfig = '' 784 + expires 1y; 785 + add_header Cache-Control "public, immutable"; 786 + ''; 787 + tryFiles = "$uri =404"; 788 + }; 789 + 790 + "/app/" = { 791 + tryFiles = "$uri $uri/ /index.html"; 792 + }; 793 + 794 + "= /" = { 795 + tryFiles = "/homepage.html /index.html"; 796 + }; 797 + 798 + "/" = { 799 + tryFiles = "$uri $uri/ /index.html"; 800 + priority = 9999; 801 + }; 802 + }; 803 + in 804 + proxyLocations // frontendLocations; 805 + }; 806 + }; 807 + } 808 + 809 + (lib.mkIf cfg.nginx.openFirewall { 810 + networking.firewall.allowedTCPPorts = [80 443]; 811 + }) 812 + ])) 813 + 814 + { 815 users.users.${cfg.user} = { 816 isSystemUser = true; 817 inherit (cfg) group; ··· 822 823 systemd.tmpfiles.rules = [ 824 "d ${cfg.dataDir} 0750 ${cfg.user} ${cfg.group} -" 825 + "d ${cfg.settings.storage.blobPath} 0750 ${cfg.user} ${cfg.group} -" 826 + "d ${cfg.settings.backup.path} 0750 ${cfg.user} ${cfg.group} -" 827 ]; 828 829 systemd.services.tranquil-pds = { ··· 832 wants = ["network.target"]; 833 wantedBy = ["multi-user.target"]; 834 835 + environment = settingsToEnv cfg.settings; 836 837 serviceConfig = { 838 Type = "exec"; ··· 864 RemoveIPC = true; 865 866 ReadWritePaths = [ 867 + cfg.settings.storage.blobPath 868 + cfg.settings.backup.path 869 ]; 870 }; 871 }; 872 + } 873 + ]); 874 }
+213 -28
test.nix
··· 13 }: { 14 imports = [self.nixosModules.default]; 15 16 - services.postgresql = { 17 - enable = true; 18 - ensureDatabases = ["tranquil"]; 19 - ensureUsers = [ 20 - { 21 - name = "tranquil"; 22 - ensureDBOwnership = true; 23 - } 24 - ]; 25 - authentication = '' 26 - local all all trust 27 - host all all 127.0.0.1/32 trust 28 - host all all ::1/128 trust 29 - ''; 30 - }; 31 - 32 services.tranquil-pds = { 33 enable = true; 34 - package = self.packages.${pkgs.stdenv.hostPlatform.system}.tranquil-pds; 35 secretsFile = pkgs.writeText "tranquil-secrets" '' 36 JWT_SECRET=test-jwt-secret-must-be-32-chars-long 37 DPOP_SECRET=test-dpop-secret-must-be-32-chars-long 38 MASTER_KEY=test-master-key-must-be-32-chars-long 39 ''; 40 41 settings = { 42 server.pdsHostname = "test.local"; 43 server.host = "0.0.0.0"; 44 45 - database.url = "postgres://tranquil@localhost/tranquil"; 46 - 47 storage.blobBackend = "filesystem"; 48 - backup.backend = "filesystem"; 49 }; 50 }; 51 - 52 - networking.firewall.allowedTCPPorts = [3000]; 53 }; 54 55 testScript = '' 56 server.wait_for_unit("postgresql.service") 57 server.wait_for_unit("tranquil-pds.service") 58 server.wait_for_open_port(3000) 59 60 with subtest("service is running"): 61 status = server.succeed("systemctl is-active tranquil-pds") 62 assert "active" in status 63 64 - with subtest("blob storage directory exists"): 65 server.succeed("test -d /var/lib/tranquil-pds/blobs") 66 server.succeed("test -d /var/lib/tranquil-pds/backups") 67 68 - with subtest("healthcheck responds"): 69 - server.succeed("curl -sf http://localhost:3000/xrpc/_health") 70 71 - with subtest("describeServer returns valid response"): 72 - result = server.succeed("curl -sf http://localhost:3000/xrpc/com.atproto.server.describeServer") 73 - assert "availableUserDomains" in result 74 ''; 75 }
··· 13 }: { 14 imports = [self.nixosModules.default]; 15 16 services.tranquil-pds = { 17 enable = true; 18 + database.createLocally = true; 19 secretsFile = pkgs.writeText "tranquil-secrets" '' 20 JWT_SECRET=test-jwt-secret-must-be-32-chars-long 21 DPOP_SECRET=test-dpop-secret-must-be-32-chars-long 22 MASTER_KEY=test-master-key-must-be-32-chars-long 23 ''; 24 25 + nginx = { 26 + enable = true; 27 + enableACME = false; 28 + }; 29 + 30 settings = { 31 server.pdsHostname = "test.local"; 32 server.host = "0.0.0.0"; 33 34 storage.blobBackend = "filesystem"; 35 + rateLimiting.disable = true; 36 + security.allowInsecureSecrets = true; 37 }; 38 }; 39 }; 40 41 testScript = '' 42 + import json 43 + 44 server.wait_for_unit("postgresql.service") 45 server.wait_for_unit("tranquil-pds.service") 46 + server.wait_for_unit("nginx.service") 47 server.wait_for_open_port(3000) 48 + server.wait_for_open_port(80) 49 50 + def xrpc(method, endpoint, *, headers=None, data=None, raw_body=None, via="nginx"): 51 + host_header = "-H 'Host: test.local'" if via == "nginx" else "" 52 + base = "http://localhost" if via == "nginx" else "http://localhost:3000" 53 + url = f"{base}/xrpc/{endpoint}" 54 + 55 + parts = ["curl", "-sf", "-X", method, host_header] 56 + if headers: 57 + parts.extend(f"-H '{k}: {v}'" for k, v in headers.items()) 58 + if data is not None: 59 + parts.append("-H 'Content-Type: application/json'") 60 + parts.append(f"-d '{json.dumps(data)}'") 61 + if raw_body: 62 + parts.append(f"--data-binary @{raw_body}") 63 + parts.append(f"'{url}'") 64 + 65 + return server.succeed(" ".join(parts)) 66 + 67 + def xrpc_json(method, endpoint, **kwargs): 68 + return json.loads(xrpc(method, endpoint, **kwargs)) 69 + 70 + def xrpc_status(endpoint, *, headers=None, via="nginx"): 71 + host_header = "-H 'Host: test.local'" if via == "nginx" else "" 72 + base = "http://localhost" if via == "nginx" else "http://localhost:3000" 73 + url = f"{base}/xrpc/{endpoint}" 74 + 75 + parts = ["curl", "-s", "-o", "/dev/null", "-w", "'%{http_code}'", host_header] 76 + if headers: 77 + parts.extend(f"-H '{k}: {v}'" for k, v in headers.items()) 78 + parts.append(f"'{url}'") 79 + 80 + return server.succeed(" ".join(parts)).strip() 81 + 82 + def http_status(path, *, host="test.local", via="nginx"): 83 + base = "http://localhost" if via == "nginx" else "http://localhost:3000" 84 + return server.succeed( 85 + f"curl -s -o /dev/null -w '%{{http_code}}' -H 'Host: {host}' '{base}{path}'" 86 + ).strip() 87 + 88 + def http_get(path, *, host="test.local"): 89 + return server.succeed( 90 + f"curl -sf -H 'Host: {host}' 'http://localhost{path}'" 91 + ) 92 + 93 + def http_header(path, header, *, host="test.local"): 94 + return server.succeed( 95 + f"curl -sI -H 'Host: {host}' 'http://localhost{path}'" 96 + f" | grep -i '^{header}:'" 97 + ).strip() 98 + 99 + # --- testing that stuff is up in general --- 100 + 101 with subtest("service is running"): 102 status = server.succeed("systemctl is-active tranquil-pds") 103 assert "active" in status 104 105 + with subtest("data directories exist"): 106 server.succeed("test -d /var/lib/tranquil-pds/blobs") 107 server.succeed("test -d /var/lib/tranquil-pds/backups") 108 109 + with subtest("postgres database created"): 110 + server.succeed("sudo -u tranquil-pds psql -d tranquil-pds -c 'SELECT 1'") 111 112 + with subtest("healthcheck via backend"): 113 + xrpc("GET", "_health", via="backend") 114 + 115 + with subtest("healthcheck via nginx"): 116 + xrpc("GET", "_health") 117 + 118 + with subtest("describeServer"): 119 + desc = xrpc_json("GET", "com.atproto.server.describeServer") 120 + assert "availableUserDomains" in desc 121 + assert "did" in desc 122 + assert desc.get("inviteCodeRequired") == False 123 + 124 + with subtest("nginx serves frontend"): 125 + result = server.succeed("curl -sf -H 'Host: test.local' http://localhost/") 126 + assert "<html" in result.lower() or "<!" in result 127 + 128 + with subtest("well-known proxied"): 129 + code = http_status("/.well-known/atproto-did") 130 + assert code != "502" and code != "504", f"well-known proxy broken: {code}" 131 + 132 + with subtest("health endpoint proxied"): 133 + code = http_status("/health") 134 + assert code != "404" and code != "502", f"/health not proxied: {code}" 135 + 136 + with subtest("robots.txt proxied"): 137 + code = http_status("/robots.txt") 138 + assert code != "404" and code != "502", f"/robots.txt not proxied: {code}" 139 + 140 + with subtest("metrics endpoint proxied"): 141 + code = http_status("/metrics") 142 + assert code != "502", f"/metrics not proxied: {code}" 143 + 144 + with subtest("oauth path proxied"): 145 + code = http_status("/oauth/.well-known/openid-configuration") 146 + assert code != "502" and code != "504", f"oauth proxy broken: {code}" 147 + 148 + with subtest("subdomain routing works"): 149 + code = http_status("/xrpc/_health", host="alice.test.local") 150 + assert code == "200", f"subdomain routing failed: {code}" 151 + 152 + with subtest("client-metadata.json served with host substitution"): 153 + meta_raw = http_get("/oauth/client-metadata.json") 154 + meta = json.loads(meta_raw) 155 + assert "client_id" in meta, f"no client_id in client-metadata: {meta}" 156 + assert "test.local" in meta_raw, "host substitution did not apply" 157 + 158 + with subtest("static assets location exists"): 159 + code = http_status("/assets/nonexistent.js") 160 + assert code == "404", f"expected 404 for missing asset, got {code}" 161 + 162 + with subtest("spa fallback works"): 163 + code = http_status("/app/some/deep/route") 164 + assert code == "200", f"SPA fallback broken: {code}" 165 + 166 + with subtest("firewall ports open"): 167 + server.succeed("ss -tlnp | grep ':80 '") 168 + server.succeed("ss -tlnp | grep ':3000 '") 169 + 170 + # --- test little bit of an account lifecycle --- 171 + 172 + with subtest("create account"): 173 + account = xrpc_json("POST", "com.atproto.server.createAccount", data={ 174 + "handle": "alice.test.local", 175 + "password": "NixOS-Test-Pass-99!", 176 + "email": "alice@test.local", 177 + "didType": "web", 178 + }) 179 + assert "accessJwt" in account, f"no accessJwt: {account}" 180 + assert "did" in account, f"no did: {account}" 181 + access_token = account["accessJwt"] 182 + did = account["did"] 183 + assert did.startswith("did:web:"), f"expected did:web, got {did}" 184 + 185 + with subtest("mark account verified"): 186 + server.succeed( 187 + f"sudo -u tranquil-pds psql -d tranquil-pds " 188 + f"-c \"UPDATE users SET email_verified = true WHERE did = '{did}'\"" 189 + ) 190 + 191 + auth = {"Authorization": f"Bearer {access_token}"} 192 + 193 + with subtest("get session"): 194 + session = xrpc_json("GET", "com.atproto.server.getSession", headers=auth) 195 + assert session["did"] == did 196 + assert session["handle"] == "alice.test.local" 197 + 198 + with subtest("create record"): 199 + created = xrpc_json("POST", "com.atproto.repo.createRecord", headers=auth, data={ 200 + "repo": did, 201 + "collection": "app.bsky.feed.post", 202 + "record": { 203 + "$type": "app.bsky.feed.post", 204 + "text": "hello from lewis silly nix integration test", 205 + "createdAt": "2025-01-01T00:00:00.000Z", 206 + }, 207 + }) 208 + assert "uri" in created, f"no uri: {created}" 209 + assert "cid" in created, f"no cid: {created}" 210 + record_uri = created["uri"] 211 + record_cid = created["cid"] 212 + rkey = record_uri.split("/")[-1] 213 + 214 + with subtest("read record back"): 215 + fetched = xrpc_json( 216 + "GET", 217 + f"com.atproto.repo.getRecord?repo={did}&collection=app.bsky.feed.post&rkey={rkey}", 218 + ) 219 + assert fetched["uri"] == record_uri 220 + assert fetched["cid"] == record_cid 221 + assert fetched["value"]["text"] == "hello from lewis silly nix integration test" 222 + 223 + with subtest("upload blob"): 224 + server.succeed("dd if=/dev/urandom bs=1024 count=4 of=/tmp/testblob.bin 2>/dev/null") 225 + blob_resp = xrpc_json( 226 + "POST", 227 + "com.atproto.repo.uploadBlob", 228 + headers={**auth, "Content-Type": "application/octet-stream"}, 229 + raw_body="/tmp/testblob.bin", 230 + ) 231 + assert "blob" in blob_resp, f"no blob: {blob_resp}" 232 + blob_ref = blob_resp["blob"] 233 + assert blob_ref["size"] == 4096 234 + 235 + with subtest("export repo as car"): 236 + server.succeed( 237 + f"curl -sf -H 'Host: test.local' " 238 + f"-o /tmp/repo.car " 239 + f"'http://localhost/xrpc/com.atproto.sync.getRepo?did={did}'" 240 + ) 241 + size = int(server.succeed("stat -c%s /tmp/repo.car").strip()) 242 + assert size > 0, "exported car is empty" 243 + 244 + with subtest("delete record"): 245 + xrpc_json("POST", "com.atproto.repo.deleteRecord", headers=auth, data={ 246 + "repo": did, 247 + "collection": "app.bsky.feed.post", 248 + "rkey": rkey, 249 + }) 250 + 251 + with subtest("deleted record gone"): 252 + code = xrpc_status( 253 + f"com.atproto.repo.getRecord?repo={did}&collection=app.bsky.feed.post&rkey={rkey}", 254 + ) 255 + assert code != "200", f"expected non-200 for deleted record, got {code}" 256 + 257 + with subtest("service still healthy after lifecycle"): 258 + xrpc("GET", "_health") 259 ''; 260 }
+8
docs/install-debian.md
··· 175 proxy_request_buffering off; 176 } 177 178 location /oauth/ { 179 proxy_pass http://127.0.0.1:3000; 180 proxy_http_version 1.1;
··· 175 proxy_request_buffering off; 176 } 177 178 + location = /oauth/client-metadata.json { 179 + root /var/www/tranquil-pds; 180 + default_type application/json; 181 + sub_filter_once off; 182 + sub_filter_types application/json; 183 + sub_filter '__PDS_HOSTNAME__' $host; 184 + } 185 + 186 location /oauth/ { 187 proxy_pass http://127.0.0.1:3000; 188 proxy_http_version 1.1;

History

8 rounds 13 comments
sign up or login to add to the discussion
6 commits
expand
feat: nix module
fix: better defaults, add in pg & nginx
feat: actual good config from Isabel
fix: set env vars of paths for sendmail & signal from nix pkg not manual
refactor(nix): toml conf
fix: minor format cleanup and description fixing in the module
expand 0 comments
pull request successfully merged
6 commits
expand
feat: nix module
fix: better defaults, add in pg & nginx
feat: actual good config from Isabel
fix: set env vars of paths for sendmail & signal from nix pkg not manual
refactor(nix): toml conf
fix: minor format cleanup and description fixing in the module
expand 0 comments
4 commits
expand
feat: nix module
fix: better defaults, add in pg & nginx
feat: actual good config from Isabel
fix: set env vars of paths for sendmail & signal from nix pkg not manual
expand 0 comments
3 commits
expand
feat: nix module
fix: better defaults, add in pg & nginx
feat: actual good config from Isabel
expand 0 comments
4 commits
expand
feat: nix module
fix: better defaults, add in pg & nginx
fix: bulk type safety improvements, added a couple of tests
feat: actual good config from Isabel
expand 0 comments
2 commits
expand
feat: nix module
fix: better defaults, add in pg & nginx
expand 5 comments

Generally much better! I'll go through with a more fine-toothed comb at some point, but one immediate comment is that you should probably be using lib.mkEnableOption rather than lib.mkOption for your enable options - it's more idiomatic.

I'd also be interested in seeing your default paths for sendmail and signal moved up (and the options can be made no longer nullOr in that case too) - remember that in Nix nothing is evaluated unless you actually look at it, so as long as you check that email/signal are enabled when you place those options into the environment file (e.g. with mkIf) then this still needn't cause a dependency when it's not desirable...

https://search.nixos.org/packages?channel=25.11&show=system-sendmail&query=sendmail

^ You should probably also consider using system-sendmail which looks in a few places rather than your own best-guess sendmail path

The starred alias in nginx should really be tied to availableUserDomains rather than the hostname

I'm not a huge fan of some of the nginx stuff you're doing - recommendedProxySettings, etc. are good settings but you're leaking config outside the module which might change stuff for other modules. It might be better to enable recommendedProxySettings per-location and then setup whatever settings for the other categories you feel work best for tranquil in a lower scope...

...on a similar note, openFirewall tends to default to false to avoid unknowingly opening ports in firewalls that you didn't mean to

openFirewall tends to default to false

(I think there's one place in nixpkgs this isn't true - and it's with SSH to avoid making stuff unreachable unexpectedly)

lewis.moe submitted #1
7 commits
expand
feat: nix module
feat: fun handle resolution during migration
feat: initial in-house cache distribution
feat: cache locks less
fix: smaller docker img
fix: removed some bs
fix: better defaults, add in pg & nginx
expand 1 comment

Sorry, just updating to main

lewis.moe submitted #0
1 commit
expand
feat: nix module
expand 7 comments

Is there any reason the package doesn't seem to default to packages.tranquil-pds from this flake? I think it might be kinda nice if it did... (though I guess it would mean you have to have a reference to the flake in your module - that shouldn't really be much of a problem though)

What's the default for settings.storage.blobPath - is it actually null? what happens if I leave the blob backend as filesystem and then this as null? Could we default it in terms of dataDir (as I suspect it must really be) so that it's easier to introspect or could we put what this'll be in the description of the option

Why is backup.backend set in the test file - given it has the same value as the default and backups aren't enabled by default (or are they? they're set to null?) ... is this a mistake or something else? blobBackend is also set to its default but this is a little more understandable incase the default were to change/etc.

I also wonder if backup.enabled should be called backup.enable and should be a mkEnableOption rather than what you are doing here...... it seems quite unidiomatic

You're using optional types in a bunch of places. I'm going to assume that they are being set as overrides to some other config that is secretly elsewhere (e.g. plc.directoryUrl)... it would be much more idiomatic to have these required and have the true defaults in nix

Can you default sendmailPath (and signal CLI path) to whatever it should be from nixpkgs? I think that'd be nice :) - you need to make sure these options don't get evaluated unless mail or signal is enabled respectively

Thanks for all the comments! I am no nix connoisseur and i'll read through carefully and try to fix