gbl08ma.com / didplcbft
A very experimental PLC implementation which uses BFT consensus for decentralization
fork atom

Improve README

gbl08ma.com 7bce97e5 f020ec86

verified
+84 -53
unified split
+84 -53
README.md
··· 1 1 # didplcbft 2 2 3 - A very experimental [PLC](https://web.plc.directory/) implementation, for `did:plc` credentials, which uses [BFT consensus](https://docs.cometbft.com/v0.38/introduction/) to decentralize the hosting/maintenance of the directory. ⚠️ This is not a cryptocurrency system, nor is it intended to become one! 3 + didplcbft is an experimental [PLC](https://web.plc.directory/) implementation. It uses BFT consensus (via CometBFT) to decentralize the hosting and maintenance of did:plc credentials. 4 + 5 + ⚠️ Despite using blockchain technology, this is not and will not be a cryptocurrency. It is currently in an experimental phase and is not intended for production use. 6 + 7 + ## The concept 8 + 9 + The current plc.directory is a centralized point in the otherwise increasingly decentralized [AT Protocol](https://atproto.com/) ecosystem. didplcbft explores an alternative approach where the operation of the PLC is distributed across a network of independent validators (home labs, PDS hosts, or other community members) rather than a single organization. 10 + 11 + From a foundational point of view, didplcbft is designed to operate independently from plc.directory, such that it could eventually replace it, becoming the _de facto_ PLC service. However, as a transitional approach, didplcbft can act as more of a mirror of an authoritative PLC implementation, namely, plc.directory. This transitional period can last indefinitely; with didplcbft being a consensus-based distributed system, this period would last until a sufficient quorum of participants agrees that it should operate independently from the centralized directory. 12 + 13 + Even though a bespoke peer-to-peer protocol for data replication and consensus could be developed for this use case, for convenience and practicality reasons didplcbft is built on top of existing blockchain-oriented technology, most notably [CometBFT](https://docs.cometbft.com/v0.38/introduction/). CometBFT is used throughout multiple cryptocurrency and non-cryptocurrency projects deployed in the real world. It seems to offer a sufficiently battle-tested solution for replicating an arbitrary deterministic application across different computers, that are not necessarily operated by trusted parties. Some of the aspects of CometBFT are desirable for our purposes, while others may be considered unnecessary or undesirable, but so far it has proven to be flexible enough such that most criticisms of "blockchain" can be mitigated or avoided. 14 + 15 + ### What is the point, now? 16 + 17 + It is safe to assume that a network of didplcbft nodes is not becoming the official PLC implementation any time soon (and probably never), so one may wonder: why bother? Even in an initial phase where it is non-authoritative, didplcbft can prove useful: 18 + 19 + - For AT Protocol service operators and researchers, the hope is that didplcbft can be a sufficiently efficient local replica of the centralized PLC directory. This can be useful for applications with a high volume of read queries, where any latency and reliability issues associated with accessing a more "distant" internet service become a concern. 20 + - For AT Protocol users who may not necessarily want to trust the data provided by the official plc.directory at face value, didplcbft can, through its deterministic and consensus-backed state transitions, can offer a more auditable history of events. In a sense, it can help "keep plc.directory in check." This is somewhat similar to what initiatives like [plcbundle](https://tangled.org/atscan.net/plcbundle) offered, but didplcbft is more ambitious in that it tries to be a path towards full decentralization of the directory, including for write operations, rather than being a mere read-only replica. 21 + - From a R&D point of view, it can help with understanding whether a blockchain-based approach can deal with the transactional volume of plc.directory without introducing excessive overhead or reliability issues, when compared to a centralized solution. 22 + 23 + In a hypothetical future where AT Protocol stewardship decides that didplcbft should become the effective PLC implementation, meaning that the didplcbft network becomes the authoritative source of PLC data, it could offer some benefits over the current centralized approach, most of which are common to many blockchain implementations: 24 + 25 + - Elimination of a single point of failure 26 + - Better multi-region availability 27 + - User privacy preservation (no centralized point where statistics/telemetry can be collected) 28 + - (Arguable) censorship resistance 29 + - With transparency around the transitions that led to the current state 30 + - Looking very far into the future, interoperability between different node implementations could help highlight/avoid certain classes of bugs 4 31 5 - _The release of this to the public is being rushed, because in light of [developments](https://github.com/bluesky-social/atproto/discussions/4508), I want to bring this into the open ASAP in order to showcase why sequence IDs make my life a bit more difficult, **if the intention for this implementation were to achieve 1:1 response parity with the official plc.directory** (I don't know if I want that yet). A very unorganized readme follows!_ 32 + **Isn't did:web the answer to decentralizing identities?** 6 33 7 - This is definitely not ready for any type of production use, even if it already implements the entirety of the PLC HTTP API (with some caveats related to the export endpoint). 34 + did:web is certainly a valid option to be able to participate in ATProto without depending on the plc.directory. However, all the reasons that led to the development of did:plc remain valid. To summarize, did:web probably "isn't for everyone;" it presents trade-offs too (e.g. dependency on DNS). We are aware that did:web too will keep evolving, but as the name implies, it will always be connected to "web" technologies in some capacity (once again, DNS comes to mind) and this might be enough to disqualify it from some use cases. did:plc definitely has its merits, and to further improve its chance of long-term success, ideally even beyond ATProto, we believe that sooner or later a concrete path to the decentralization of the PLC must be developed. We are trying to work on precisely that. 8 35 9 - The general theme behind this project is, 36 + ## Features 10 37 11 - > Man, plc.directory seems so centralized, particularly considering most other atproto components are, at least in theory, decentralizable! There's this idea of forming an [independent organization](https://docs.bsky.app/blog/plc-directory-org) to maintain it, but what if we just skipped that entirely and made the ownership and operation of the entire system even more murky, while giving those nerds that run their own PDS/relay/appview/etc. another thing to run on their servers/homelabs? All while giving such a nerd - gbl08ma - an excuse to learn more about the tech that powers all the many blockchains that use Tendermint consensus! 38 + ### Currently implemented 12 39 13 - Hopefully the tone of that communicates that this initiative isn't meant to be taken too seriously, _at least yet._ 40 + - **PLC HTTP API:** Full support for GET and POST operations (resolve DID docs, logs, audit logs, and submit new operations), with the detail that only sequence-based export pagination is supported 41 + - didplcbft is already able to work as a decentralized, standard-compliant PLC implementation, independently from plc.directory 42 + - **Validation:** On-chain validation of PLC operations to ensure history integrity. 43 + - **Node-to node fast syncing:** Support for snapshot-based sync, to quickly bring new replicas online, making use of the facilities offered by CometBFT. 44 + - A custom compact serialization format is used, able to archive/transmit the entire directory using around 30 GB of space/data transfer as of January 2026. 14 45 15 - The idea for how this would work, on an initial phase, would be to continue using plc.directory as an authoritative source while mirroring the operations in the blockchain. 16 - Validator nodes would gradually import operations from plc.directory ("authoritative import"). Eventually they'd catch up to the present moment, at which point this import would keep going as a way to bring in all the operations not submitted to didplcbft - because I recognize that this would definitely not become the official PLC implementation any time soon, and new operations would continue to be submitted to plc.directory by the general population. 17 - Nodes would also submit any operations observed on the blockchain (i.e. operations submitted directly to didplcbft) to plc.directory. 18 - Conflict/fork resolution within each DID history would happen by deferring to whatever plc.directory says the truth is. 46 + ### In progress 19 47 20 - **_Hypothetically_**, at some point in the future, and should an hypothetical network of didplcbft nodes reach a consensus on that, such network could act independently (for this, it'd just need to cease the ongoing import of operations, and cease publishing operations to plc.directory. In fact, the current code is already capable of acting like this - it's the syncing with the official centralized plc.directory that's more tricky to implement). 48 + - **Authoritative Mirroring:** (mostly functional, needs adjustments): An "authoritative import" mechanism to pull historical data from the official plc.directory. 49 + - **Reputation System:** A "proof of useful storage" system where nodes submit periodic proofs to maintain validator status. 50 + - Due to the way CometBFT works, it is ideal if there is a limit to how many nodes perform validator duties. The reputation system is meant to select, over time, the "best" nodes for this task, and manage their "voting power." 51 + - The reputation system will have some resistance against Sybil attacks - a single entity operating N identities should not be able to get away with putting in less effort than that of N different honest entities combined. 21 52 22 - ## What's implemented 53 + ### Planned 23 54 24 - - The entirety of the PLC HTTP API, 25 - - Both read (GET) and write (POST) operations 26 - - ...but export only works with timestamps (sequence-based export and websocket-based export is not implemented) 27 - - Validation of operations (hopefully it's foolproof enough - needs revising, strengthening... and nullification might not be as simple as I made it out to be?) 28 - - Snapshot-based sync for quickly bringing replicas up to date 29 - - An initial, not very well tested, version of the "authoritative import" that gradually brings in operations from the official plc.directory 55 + - **Websocket-based Export** equivalent to that of the official plc.directory service. 56 + - **Bi-directional Sync:** submitting operations observed on the didplcbft network back to the official plc.directory, while still deferring to operations served by the latter in case of conflict. 57 + - **Spam Prevention:** developing a non-currency-based throttling mechanism. 58 + - For example, by gossipping hashes of IP addresses and AS numbers across the network in order to limit how quickly spammers can create new identities in the PLC. The challenge is that certain entities (e.g. Bluesky's own official PDSs) will naturally need to create many more identities than others... maybe some sort of allowlisting mechanism would need to be implemented. 59 + - **Dynamic Validator Sets:** automatic management of which nodes perform validator duties based on their reputation. 60 + - **Public Testnet:** moving from local clusters to a distributed internet-based environment. 61 + - Not really a feature, but more of a cross-cutting aspect: proper test coverage (will probably require refactoring the code first to better separate concerns), and perhaps some proper specs around how the proofs and reputation system work. 30 62 31 - ## What's yet to be implemented 63 + ### Out of scope 32 64 33 - - Actually syncing with the authoritative source, plc.directory (in progress - fetching from the official directory is mostly implemented as indicated above; submitting blockchain happenings to the official directory is yet to be implemented) 34 - - Spam/abuse prevention (typical blockchains do this with transaction fees and some lesser known ones - e.g. Nano - use proof of work, but this is not a cryptocurrency and PoW is a bit ewww) 35 - - A process for nodes to become validators. Unless everyone agreed that it's best if the set of validators is centralized. I mean, it's not worse than the current state of plc.directory while still allowing people to easily have their own local synced mirror through the magic of CometBFT... 36 - - Testing, testing, testing, validation, validation, validation... 37 - - A live testnet on the internet, so this can go from something that runs just within the computers of the developers, and actually becomes distributed 38 - - This might be a matter of someone just figuring out the right CometBFT configs and bringing it up? 39 - - I am planning to take care of this after the "authoritative import" mechanism is done (its first version, anyway) 65 + - **Financialization:** no tokens, no fees, and no "store of value" features will be added. 66 + - Separation of concerns: why bring finance into an identity system? Identities exist irrespective of their financials. 67 + - This will hopefully ensure that didplcbft does not run afoul of financial regulations. 68 + - This incentivizes participants to join the network not because of a direct financial incentive, but because it proves useful to them or their service/business. 69 + - For the better or for the worse, this also effectively means we can't depend on transaction fees as a rate limiting mechanism. 70 + - Timestamp-based pagination on the export endpoint: this would complicate the storage layer (requiring extra space for an additional index, at the very least) and is marked to eventually be deprecated by the official PLC implementation 40 71 41 - ## What's not to be implemented by the original authors 72 + ## Technical notes 42 73 43 - - Features that would allow for didplcbft to act as some form of currency or store of value. 74 + - Consensus-backed data is stored in an [IAVL+ tree]((https://pkg.go.dev/github.com/cosmos/iavl)), which can cryptographically prove and disprove the presence of specific leaves. 75 + - This tree and some non-consensus-backed auxiliary/derived data live in a couple [badger](https://github.com/dgraph-io/badger) key-value stores, using settings specifically tuned for the use cases at hand. 76 + - The PLC HTTP API reads the IAVL+ tree directly for read operations, and submits write operations as blockchain transactions. 77 + - The current implementation of the "proofs of useful storage" involves zkSNARK proofs, mostly as the means to attain a "bounded proof of work" mechanism, where proofs require some effort to produce, but can be verified instantly. By "bounded proof of work" we mean that honest nodes don't gain anything from throwing more resources at the system (so it shouldn't devolve into an arms race like Bitcoin's proof of work), and dishonest nodes attempting a Sybil attack will need to spend as many compute resources to operate each Sybil identity, as a single honest node would have to spend operating its own honest identity. 78 + - Why not rely on enforcing that each directory copy is unique through node-specific XOR operations, as with Filecoin sealing? Because this appeared to make implementation much more complex, would probably couple the storage layer to the proving mechanism in a way that wouldn't be easy to back out of, and after some research, it looked like all workable existing systems of "proof and space in time" (a la Filecoin) required excessive resources for their "sealing" processes (dozens of gigabytes of RAM, etc.). They did not seem adequate for somewhat dynamic data, as is the case with the PLC operations (which can be modified after creation, when being nullified, and in ultra-rare circumstances where history is rewritten). 44 79 45 - ## How it works 80 + ## Getting started 46 81 47 - - Ugh, I'll explain later. The gist of it is that all of the data is stored in an [AVL+ tree](https://pkg.go.dev/github.com/cosmos/iavl) which is basically the perfect match for CometBFT. The core logic of the PLC is implemented as an ABCI 2.0 application, and the HTTP API communicates with it (turning POST requests into blockchain transactions, and for GET requests it ends up just reading the tree directly, why even bother going through the ABCI `Query` logic for that, am I right?). The tree is "synced" across replicas by having CometBFT replay transactions or whatever it is that it does, what matters is that in the end the root node of the tree has the same hash on all replicas which means everything was replicated properly (CometBFT takes care of ensuring this is the case, as long as our application is deterministic and stores all relevant state in the tree). 82 + This project is at a stage where it is very much oriented towards further development work, and not yet towards the operation of "real" nodes. 48 83 49 - ## Some random notes and comments 84 + Note: the code currently contains various debug prints and unstructured logging. There are multiple `// TODO` comments spread throughout. It is definitely a work in progress. 50 85 51 - - Yes, this project is bound to elicit knee-jerk reactions, understandably. I was planning to make it public only once it was more complete, but I figured the public discussion could benefit from the awareness that this _exists_, even if it isn't necessarily going somewhere. And with the official implementation [introducing global sequential IDs](https://github.com/bluesky-social/atproto/discussions/4508), I figured time is of the essence before reimplementing things in a distributed fashion becomes even more difficult. 52 - - To be clear, the sequence ID only poses a challenge if we wanted the different blockchain nodes to present the same sequence IDs. And honestly, even if we wanted that, it's still perfectly doable, but it'd require more complexity on the "authoritative import" reconciliation mechanism (we'd potentially need to renumber operations that already exist within the didplcbft tree). And 53 - - Yes, this is a blockchain, but only because it was relatively easy for me to bring in CometBFT as a means of making things happen in sync across different machines (even if this is my first time using it, which is fantastic, I'm sure there are 300 pitfalls I'm not aware of). 54 - - If I had come across a different P2P framework of sorts with equally nice sync primitives and resistance to byzantine faults that was not a blockchain and which supported the languages I like to use, I'd have used it. 55 - - Because each DID history is its own mini-blockchain, I'm convinced that all nodes could actively throw away older blockchain history (CometBFT supports this) and from then on, The World could rely exclusively on snapshot sync (which is already implemented). Of course, there are trust implications with this approach (but are they really worse than trusting plc.directory as it is right now?). 56 - - For "a blockchain approach," the better defined operation validation is, the better (in the past, the plc.directory was not very good at this, and they had to go and retroactively mess with existing data...). But the fun thing about blockchain is that it works as long as some network of nodes agrees on what the truth is. So of course history can be re-written on a blockchain too, it just takes slightly more effort. 57 - - I am convinced the current storage implementation can fit the entire state of the plc.directory in under 100 GB. It isn't great but it isn't terrible either, considering it's a blockchain and all. But I don't know for sure that it can, this is merely what tests with roughly 10 million operations stored have shown (operations taken from plc.directory, so real world data). 58 - - There are plenty of unstructured debug prints throughout the code, and in general the code quality is meh... like I said, this all is very experimental. 59 - - I have some thoughts about using proof-of-storage and some kind of "leaderboard system" (not a currency) in order to elect/demote validators 60 - - Another option would be to tie it to some type of social metrics (did:plc is mostly used by atproto stuff after all right?) but I can't think of anything that can't be manipulated 61 - - CometBFT coupled with the AVL+ tree implementation I'm using has some nice properties for proofs of storage (and proofs that one doesn't have something in storage) which _could_ make it easier to prove that nodes are not omitting newer operations of a DID's history on purpose (which is really the main attack I can see a malicious PLC implementation performing, as far as the contract with its consumers is concerned). But it's very early in the life of this project, and I haven't thought sufficiently about this. 62 - - Spam prevention will be a difficult problem (it already is in the official plc.directory; of course it'd be even more difficult to solve in a decentralized system) 63 - - But see also the above notes on exclusively using snapshot sync, and on consensus... the truth is whatever the nodes agree it is, so spam could conceivably be removed even after it is inserted, etc. Just because it uses a blockchain it doesn't mean that it must store everything forever, history can be forgotten if everyone agrees on a snapshot, etc. A distributed system just means that rewriting history takes more effort compared to simply running some Postgres migrations... this would be the case even without "blockchain" in the mix. 64 - - This project would kind of reduce the need for something like `plcbundle` (even if it could learn something from it, when it comes to importing operations from the authoritative plc.directory) 86 + ### Contribution prerequisites 65 87 66 - ## How to run this 88 + - Sufficient knowledge of Go (this project targets the latest stable version) 89 + - Some knowledge of how CometBFT operates, particularly, knowledge of the interface/behavior ABCI applications should have (read the [CometBFT docs](https://docs.cometbft.com/v0.38/introduction/)) 90 + - Familiarity with the [`did:plc` method specification](https://web.plc.directory/spec/v0.1/did-plc) and [directory server API](https://web.plc.directory/api/redoc) 67 91 68 - I would rather not reveal it right now, but if you insist... 92 + ### Running locally 69 93 70 - Note that this definitely doesn't currently do enough things to be a viable PLC mirror, let alone _do them well_. This is at a stage where it's really only useful to those wanting to work on it. 94 + For development and testing, use the provided helper scripts: 71 95 72 - Let me just point out that there are some scripts in this repo (mostly LLM generated because I hate shell scripting). 96 + - For a single node: `./startfresh.sh`. 97 + - This will clear the data and configuration in the `didplcbft-data` directory and start a single node. 98 + - At least at the time this README was written, the node will immediately begin importing operations from plc.directory (1000 operations/block). 99 + - To disable this, comment the relevant code in `PrepareProposal` - the `if req.Height == 2 {` condition. 100 + - With this disabled, didplcbft will act as a fully independent PLC, only serving the operations you create through it. 101 + - Local Testnet (Multiple Nodes): `./startfresh-testnet.sh` 102 + - This will clear the data and configuration for multiple nodes in the `testnet` directory and start 4 nodes by default (you can pass the number of nodes as the first argument to the script) 73 103 74 - You need to install Go, in case the go.mod and multiple *.go files didn't make it obvious. 104 + The PLC API server listens on `127.0.0.1:28080` by default. When running a testnet, the script makes it so that only one of the nodes serves this API. 75 105 76 - To run a single node, you want to use `startfresh.sh`. To run multiple nodes, there's `startfresh-testnet.sh`. Note that using the created config files, only the first node will serve the PLC API. 106 + To easily import operations you can play around with the "test" that's within `importer/importer_test.go` - note that this imports operations by creating blockchain transactions directly, it doesn't use the PLC API, but you can probably ask your favorite LLM to change it as you see fit. 107 + You can also use your favorite existing tools for interacting with the PLC (e.g. `goat`) by pointing them at the PLC API endpoint mentioned above, rather than plc.directory. 77 108 78 - The PLC API server listens on `127.0.0.1:28080` by default and other fun facts you could learn from reading config.go. There are other ports that the server listens on, one that exposes the CometBFT RPC API, and also the P2P ports for communication between the nodes. 109 + ## Contact 79 110 80 - To easily import operations you can play around with the mess that's within `importer/importer_test.go` - note that this imports operations by creating blockchain transactions directly, it doesn't use the PLC API. And despite being called "importer" this is not how I was meaning to do the "authoritative import" mechanism I mentioned earlier - this is really just for bringing in data for testing. 111 + didplcbft is being brought to you by us, meaning, gbl08ma and his spare time 👋 Unsurprisingly, you can reach me [on Bluesky](https://bsky.app/profile/gbl08ma.com) and from there we can move to other communication platforms as we see fit. 81 112 82 113 ## Disclaimer 83 114