如何创建自己的去中心化通信协议

Let's say you want to create a system where each server can "join" a network and start exchanging something with other members within that network. How would you approach such a task? You could create a small centralized orchestrator that will accept join requests and handle that for the recipients, but that's not decentralized. What if we want a truly "plug and play" solution without any centralized servers? One such solution is federation.

A good example would be SMTP (the protocol for sending emails). It's mostly decentralized. Anyone can run their own mail server. There is no single central authority that says "yes! this server can have email, but that one? NOOO!". Each server communicates using a well-known protocol and exchanges information server-to-server. You can even call it an "API with known payload schemas", although that is a bit of a simplification.

Let's go with the email thing and start with the obvious. Have you ever considered why email addresses contain an @<domain> part? Why can't it just be my_nick?

It's necessary to know which domain is responsible for that identity! duh!

That domain plays a very important role in the whole federation exchange.

That's because if you write from your email, let's say [email protected], to [email protected], the sending mail server will take that y.com part, do DNS discovery for MX records, connect to the mail server responsible for that domain, and start exchanging messages.

This is the core idea of federation: servers talk to each other directly after some "discovery process". In our small email example, the discovery part would be a DNS lookup for MX records.

Ok, but we came here to create our own messenger protocol, so let's start.

First, we need to gather some requirements like good system (and in this case, protocol) designers.

That should be the basic minimum for our needs. So how do we even start doing it?

Let's start with the easiest, and then move to more complex stuff.

Let's also assume that every server already has some users with registration, login, etc. So we don't spend a million years doing everything from scratch.

We need to enforce that user identity will be like an email.

So <nickname>@<domain>, otherwise our federation will not know who to call.

The important thing here is that [email protected] is not an email address. It only uses the same shape because the shape is useful. The domain tells other servers where to discover federation metadata.

We could go with, for example, x.com > adam, x.com#adam, or adam on x.com. It's your call.

Now the first real work. Discovery is the part similar to the MX record lookup process mentioned earlier.

We have full freedom on how we want to handle this, so let's do it with a simple file on the server.

For this, we can use the typical .well-known directory.

In our case, we can call our protocol lnchat (from LearN chat).

This will be our first requirement for the protocol.

GET /.well-known/lnchat.json

Inside we can put the version and other important information that we will craft later in the article. For now, we can go with only the version:

{ "protocol": "lnchat.federation.v1" }

By the way, it's good practice to always version protocols. It enforces the contract and saves you from a lot of headaches.

With this, servers can check if the servers we want to reach even support our protocol. If yes, they can pull all the necessary information from this one .well-known file.

Since we want to have end-to-end encryption, we need client-side encryption keys.

Usually that means one private key that stays with the client, and one public key that the user's server can share with other servers.

The private key MUST NOT be sent to the server as plaintext. If the server can read the private key, then the server can read the messages, and we don't really have end-to-end encryption anymore.

Important: if a server is responsible for publishing users' public keys, a malicious or compromised server could lie about a key. Real end-to-end encryption needs some way to notice that, for example contact key warnings, fingerprints, TOFU, or key transparency.

So we need an endpoint for sharing users' public keys. Every server should implement it as part of our protocol. So let's craft it!

First, let's add information to our discovery file about where to look for the endpoint. This gives every server the ability to choose its own URLs.

It's better not to enforce strict paths here as it may collide with already existing routes on the server.

{ "protocol": "lnchat.federation.v1", "endpoints": { "resolve_users": "https://example.com/federation/v1/users/resolve" } }

But the resolve_users URL could be a different URL on every server.

The resolve_users endpoint will be responsible for returning current public encryption keys for users from that server.

Other servers will call it before sending a message, so they know which key should be used for which recipient.

We can craft a schema for this endpoint like this:

{ "protocol": "lnchat.federation.v1", "users": ["[email protected]", "[email protected]"] }

Why an array of users and not just one? Let's imagine that it will be used in high-volume servers. We should offer a way to ease the performance. If the server can resolve several users in one request, it will be much more performant - fewer HTTP calls.

{ "protocol": "lnchat.federation.v1", "status": "ok", "users": [ { "user": "[email protected]", "status": "ok", "key": { "key_id": "adam-enc-2026-07-a", "public_key": "base64url-raw-32-byte-x25519-public-key" } }, { "user": "[email protected]", "status": "not_deliverable" } ] }

So the user can expose a key with a key_id and public_key.

Why do we need a key_id?

Well, imagine how the flow would look like.

The client gets the user's public key and probably stores it in some cache, then sends messages using that key.

What if that key changes? The receiver will never decrypt it. We need to have a way to distinguish keys from each other and even show a warning that the key changed! Maybe it was compromised!

I also added two statuses: ok and not_deliverable. ok is self-explanatory, but I added not_deliverable for cases where we don't have a public key for that user yet.

Remember that the user needs to generate those keys. We cannot safely do it on the backend if we want real end-to-end encryption.

So there is a chance that we already have users who never generated them. We should clearly state that this user is not a valid user for receiving encrypted messages.

For now, we only handled the endpoint for server-to-server key exchange. But how will that public key appear in the first place?

The user can use the Web Crypto API to generate keys, store the private key locally, and send the public key to the backend to keep it in the DB.

const keyPair = await window.crypto.subtle.generateKey( { name: "X25519" }, false, ["deriveBits"] );

const publicKey = await window.crypto.subtle.exportKey( "raw", keyPair.publicKey );

Generally, we should avoid exporting the private key outside of the client unless we really need to. Browsers can store CryptoKey objects in IndexedDB. If the private key is non-extractable, it is harder to accidentally leak it as a string somewhere.

This key is not literally used to encrypt the text directly. It is used by the encryption method to create a shared secret between sender and receiver.

So how would the flow look from the user's perspective?

Here we assume that our home server can store the public_key, and it will be exposed through the resolve_users endpoint according to our protocol schema.

Now we can resolve the recipient's public key. Great. But we still need to actually send something.

For the first version, let's keep the responsibility split simple:

First, the client encrypts the message locally. Then the client could send something like this to its own server:

{ "recipient": "[email protected]", "recipient_key_id": "barney-enc-2026-07-a", "encapsulated_key": "base64url-hpke-enc", "ciphertext": "base64url-ciphertext" }

We will talk about the encapsulated_key and ciphertext in a minute. So for now let's just accept that it's a "secret message".

And the plaintext inside the ciphertext can be very small:

{ "protocol": "lnchat.message.v1", "type": "text", "body": "hello" }

Or the actual message could be simple text, but it's better to pack it in some sort of format so we can send more metadata alongside the message itself. Maybe later this could be used for a simple version of attachments, etc.

The server then creates its own HTTP request and sends it to the Recipient Server. But to which endpoint?

Well, to achieve that we need to add one more endpoint to our discovery file.

{ ... "endpoints": { ... "send_messages": "https://example.com/federation/v1/message" } }

Now Sender Server can take the encryption data and send it to Recipient Server.

{ "protocol": "lnchat.federation.v1", "message_id": "019f47a9-b138-79d2-8ec3-38a2a9326955", "sender": "[email protected]", "recipient": "[email protected]", "recipient_key_id": "barney-enc-2026-07-a", "encapsulated_key": "base64url-hpke-enc", "ciphertext": "base64url-ciphertext" }

Notice that we added message_id and sender, but they were not present in the client message. This is intentional. We shouldn't trust the client to provide the real sender id. The user is already authenticated, so we can always put their name there.

And message_id is there for idempotency. If the server sends the same message twice, and the first one was already processed, we should reject the second one.

The message_id should be a randomly generated string. We can go with UUIDv7, for example.

recipient_key_id should be the same id we get from the resolve_users endpoint. It tells the recipient which key was used to encrypt the message.

What's up with the encapsulated_key, you may ask?

Well, for a modern messaging system it's good to use something called HPKE.

HPKE itself is a bit too heavy to explain fully in this post, so let's use a simpler mental model. Encrypting directly with asymmetric keys can be computationally heavy. So the idea is:

This is not a precise HPKE specification, but it is close enough to understand why the payload has two parts: the encapsulated key material and the ciphertext.

So how would we do it in JS? Better not to do it ourselves anymore using raw crypto methods. Let's delegate it to smarter people from GitHub, like this: https://github.com/dajiaji/hpke-js

For readability, let's assume that recipientPublicKey is already imported from the resolved base64url public key.

import { Aes256Gcm, CipherSuite, HkdfSha256, } from "@hpke/core"; import { DhkemX25519HkdfSha256 } from "@hpke/dhkem-x25519";

const suite = new CipherSuite({ kem: new DhkemX25519HkdfSha256(), kdf: new HkdfSha256(), aead: new Aes256Gcm(), });

function b64url(bytes) { return new Uint8Array(bytes).toBase64({ alphabet: "base64url", omitPadding: true, }); }

const plaintext = new TextEncoder().encode(JSON.stringify({ protocol: "lnchat.message.v1", type: "text", body: "hello", }));

const sender = await suite.createSenderContext({ recipientPublicKey, });

const ciphertext = await sender.seal(plaintext);

const payload = { recipient: "[email protected]", recipient_key_id: "barney-enc-2026-07-a", encapsulated_key: b64url(sender.enc), ciphertext: b64url(ciphertext), };

Signing server messages

So we can discover servers, resolve users, get users' keys, and send encrypted messages. Nice. But we still have one big problem.

How does y.com know that the message really came from x.com? Anyone can send an HTTP request and put this in JSON:

{ "sender": "[email protected]" }

That proves nothing. So federation requests need to be signed by the sending server.

We need a separate key pair. Do not mix it with user encryption keys.

So let's add yet another field to the discovery file. This time it