How to self-host all of Bluesky (except the AppView (for now))
https://alice.bsky.sh/post/3laega7icmi2q
alice · 7 Nov 2024 ·
3 min read
Did you know? You can self-host and/or mirror almost all of Bluesky's infrastructure, today!
This article assumes familiarity with the architecture of Bluesky, general *nix knowledge and basic knowledge of the toolchains of TypeScript, Go and Rust
PDS #
The most obvious one: having your own PDS and owning your data. You can find the GitHub repository with instructions here; you can migrate your existing account with the GOAT tool, for which Bryan Newbold has written an excellent howto post.
Relay #
Bryan has a great blog post on how to set up your own Relay right here. Things have grown since, so make sure you have at least ~4.5 TB of disk space (maybe more). Two important comments, not in the post: make sure to use the --disk-persister-dir=/data/events flag as well as enable compaction.
Jetstream #
Jetstream is like having your own Relay firehose, but uses a fraction of bandwidth, storage, and gives you friendly JSON instead of CBOR-encoded MST blocks. After cloning the GitHub repository, you can either use the included docker-compose.yaml with make up or build it directly with make build and use your favorite keep-this-process-running-pretty-please tool (systemd, pm2 etc.) Don't forget to take a look at the available CLI arguments/env variables which lets you do things like change the default retention (24 hours) or override the Firehose cursor and backfill it with 3 days of data—which is what's available from the official relays.
plc.directory mirror #
To get a mirror of plc.directory, with almost all users on Bluesky (they do support did:web but they are few and far between), clone str4d's plc repo, switch to the mirror branch, run cargo run --features mirror -- mirror run mirror.db, then sit back and wait 5-10 hours. Once it's done and up-to-date, you have a full replica in a neat sqlite3 DB. You can monitor its status with something like watch -n60 'sqlite3 mirror.db "select count(*) from identity;"' in a tmux pane.
The official web/mobile app, also known as `social-app` #
Clone the repo, run yarn && yarn web, and you have a fully-functional copy of it in mere minutes that you can modify to your heart's desire even. Doing this on mobile is a lot more involved and documented here.
AppView #
The elephant in the room is, of course, running your own Bluesky AppView. If you're interested, DM me on Bluesky to join my working group to make it happen!