# selfsync **English** · [Русский](README.ru.md) Sync your Obsidian vault through a server you run yourself. One small Rust binary, data in SQLite and plain files, a plugin for Obsidian on desktop and mobile, and a command-line client for servers and NAS boxes. The main promise: **no edit is ever lost silently.** - Edits to different parts of a file are merged automatically (diff3). - When edits overlap, both versions are kept: the server's in place, yours as a copy next to it. - An edit wins over a delete. - Files deleted on any device stay in the server's trash, and every file has a revision history. Also: - renames and moves without re-uploading; - resumable transfers of large files in both directions; - case-insensitive file systems (Windows, macOS, iOS, Android) and Unicode names (NFC/NFD); - optional end-to-end encryption of contents and file names; - connecting a new device with a link or a QR code; - revoking devices. ## Before you install the plugin - **You need your own selfsync server.** The plugin does nothing without one; there is no hosted service, account or subscription. Setup is described below. - **Network use.** The plugin talks only to the server address you enter: it uploads and downloads the files of your vault and their metadata (names, sizes, modification times). Without encryption the server stores your notes as they are; with encryption it stores only ciphertext. - **No telemetry.** The plugin sends nothing anywhere else. - **Direct file system access on desktop.** Obsidian's review flags this, so here is why. On desktop the plugin reads and writes the files of your vault through Node's `fs` module instead of Obsidian's API: a downloaded file replaces the old one atomically and is flushed to disk (`fsync`), so a crash or power loss never leaves a half-written note, and large attachments are read and hashed in chunks instead of being loaded into memory whole. It only touches paths inside the vault folder (plus Obsidian's trash when a file is deleted). On mobile it uses Obsidian's adapter API. - **Clipboard.** The "Copy link" button in "Connect a new device" writes the one-time connect link to the clipboard. The plugin never reads the clipboard. - **WebAssembly inside `main.js`.** The sync core is Rust compiled to WebAssembly ([crates/selfsync-wasm](crates/selfsync-wasm)) and embedded into `main.js`, which is minified. Nothing is obfuscated or downloaded at runtime: release assets are built from this repository by the [release workflow](.github/workflows/release.yml) and carry GitHub build provenance attestations. - Deleted files go to Obsidian's trash (system or `.trash/`, as configured in Obsidian), never deleted permanently. ## How it works ``` Obsidian (desktop, mobile) folder on a server/NAS ┌──────────────────────────┐ ┌──────────────────┐ │ plugin (TypeScript) │ │ selfsync-cli │ │ I/O and UI │ │ tokio + std::fs │ │ ┌──────────────────────┐ │ │ ┌──────────────┐ │ │ │ sync core (WASM) │ │ │ │ sync core │ │ │ └──────────────────────┘ │ │ └──────────────┘ │ └────────────┬─────────────┘ └────────┬─────────┘ │ HTTP + protobuf │ └──────────────┬───────────────┘ ┌───────┴────────┐ │ selfsync │ SQLite (metadata, history) │ (server) │ + content-addressed blobs └────────────────┘ ``` The sync logic is written once, in the `selfsync-core` crate. The plugin and the command-line client only carry out its actions: read a file, write a file, send a request. Design documents (in Russian): [architecture](spec/architecture.md), [protocol](spec/protocol.md), [decisions](spec/decisions.md). ## Server ### Quick start Both ways put the server behind Caddy, which gets a Let's Encrypt certificate by itself. You need a domain that points at the server and open ports 80 and 443. **A VPS with systemd** (Debian, Ubuntu, Fedora), one command: ```sh curl -fsSL https://raw.githubusercontent.com/Darkildo/selfsync/main/deploy/install.sh \ | sudo sh -s -- --domain notes.example.com ``` The script: 1. downloads the latest server release and checks it against `SHA256SUMS`; 2. installs the systemd units: the server started on demand by its socket, plus the verify, backup and sweep timers; 3. installs Caddy and writes its config (it leaves a Caddyfile of your own alone and prints the line to add instead); 4. prints a connect link with a QR code for the first device; 5. runs `selfsync doctor`. Run it again to update: the data is archived to `/root` first, and the previous binary stays next to the new one. With `--no-proxy` it skips Caddy, for when you already run your own proxy. Admin commands then go through `selfsync-admin`, which runs them as the service user: `selfsync-admin token list`, `selfsync-admin link --vault notes --name phone`. **Docker:** take [deploy/quickstart](deploy/quickstart). It builds a small image around the release binary, with no source code or Rust toolchain needed. ```sh cp .env.example .env # domain, email, version mkdir -p data backups && sudo chown 65532:65532 data backups docker compose up -d docker compose run --rm selfsync link --vault notes --name phone ``` The rest of this section explains the modes and settings in detail. ### Choosing a mode The server is a single `selfsync` binary that can run in several ways. They differ in how much they cost while nobody is syncing and how fast an edit on one device shows up on another. | Mode | Process between syncs | How clients learn about changes | Good for | |---|---|---|---| | **CGI** (Caddy, nginx + fcgiwrap) | none, one process per request (~1 ms start) | polling: every 15 s while Obsidian is active, backing off to 5 min when idle | shared hosting, minimal resources | | **socket** (systemd socket activation) | started by the first request, exits after 10 min idle | long-poll `/v1/wait`, nearly instant | your own VPS: fast and almost free when idle | | **serve** (Docker, plain process) | always running | long-poll | Docker, home server | | **serve + Sablier** (Docker on demand) | the proxy starts the container, Sablier stops it | polling | Docker with memory savings | | **Podman Quadlet** | like socket, in a container | long-poll | rootless containers | **Long-poll versus starting on demand is a trade-off.** Long-poll keeps a request open while Obsidian is in the foreground, which in socket mode keeps the process alive until 10 minutes after every device goes to the background. If you would rather have no process almost all the time, turn off "Server is always running" in the plugin: clients will poll instead, and changes from other devices will arrive up to one polling interval later. Keep it off for CGI, where long-poll is impossible (the client detects this and switches to polling on its own). ### Installing the binary Static binaries (musl, x86_64 and aarch64) are published in `server-X.Y.Z` releases together with `SHA256SUMS` and build provenance attestations (the `X.Y.Z` releases hold only the Obsidian plugin): ```sh sha256sum -c SHA256SUMS --ignore-missing gh attestation verify selfsync-x86_64-unknown-linux-musl --repo Darkildo/selfsync sudo install -m 0755 selfsync-x86_64-unknown-linux-musl /usr/local/bin/selfsync ``` From source: `cargo build --release -p selfsync-server` (binary at `target/release/selfsync`). ### systemd: start on demand (socket) ```sh sudo cp deploy/systemd/selfsync.{socket,service} deploy/systemd/selfsync-{sweep,verify,backup}.{service,timer} /etc/systemd/system/ sudo systemctl daemon-reload sudo systemctl enable --now selfsync.socket selfsync-sweep.timer selfsync-verify.timer selfsync-backup.timer ``` Put a reverse proxy with TLS in front of the server. Ready-made configs: [deploy/caddy/Caddyfile.socket](deploy/caddy/Caddyfile.socket) and [deploy/nginx/selfsync-proxy.conf](deploy/nginx/selfsync-proxy.conf). The unit runs with `DynamicUser`, data lives in `/var/lib/selfsync`. Run admin commands like this: ```sh sudo systemd-run --pipe -p DynamicUser=yes -p User=selfsync -p StateDirectory=selfsync \ /usr/local/bin/selfsync token add --vault notes --name laptop ``` ### CGI [deploy/caddy/Caddyfile.cgi](deploy/caddy/Caddyfile.cgi) (Caddy with the `caddy-cgi` plugin) or [deploy/nginx/selfsync-cgi.conf](deploy/nginx/selfsync-cgi.conf) (nginx + fcgiwrap). CGI mode is detected from `GATEWAY_INTERFACE`. The data directory must belong to the user CGI runs as, and admin commands must run as that user too. ### Docker ```sh cd deploy mkdir -p data && sudo chown 65532:65532 data # bind mount, not a named volume docker compose --profile always up -d # or --profile ondemand (Sablier) docker compose run --rm selfsync token add --vault notes --name laptop ``` The image is based on distroless/static: no shell, read-only file system, unprivileged process. Data is kept in `./data` through a bind mount rather than a named volume, because `docker compose down -v` deletes named volumes together with all your notes. ### Podman Quadlet [deploy/quadlet/](deploy/quadlet/) runs the container with socket activation under rootless Podman. ### Configuration Environment variables (or TOML via `--config`; variables take precedence): | Variable | Default | Meaning | |---|---|---| | `SELFSYNC_DATA_DIR` | `$STATE_DIRECTORY` or `/var/lib/selfsync` | data directory | | `SELFSYNC_PUBLIC_URL` | from request headers | external address, used in connect links | | `SELFSYNC_IDLE_TIMEOUT` | `10m` (socket) | exit after idling, `0` means never | | `SELFSYNC_MAX_BLOB_SIZE` | 512 MiB | largest file | | `SELFSYNC_MAX_BODY_SIZE` | 16 MiB | request body (metadata, upload chunk); keep in line with the proxy | | `SELFSYNC_LOG` | `info` | log level (file names never reach the log) | | `SELFSYNC_TRUSTED_PROXY` | `0` | `1`: take the client address from `X-Forwarded-For` (rightmost entry) | | `SELFSYNC_METRICS_TOKEN` | — | enables `GET /metrics` behind this bearer token (16 characters or more) | | `SELFSYNC_RATE_REDEEM_ATTEMPTS` | `10` | wrong connect codes from one address per window | | `SELFSYNC_RATE_REDEEM_WINDOW` | `15m` | counting window for connect codes | | `SELFSYNC_RATE_REDEEM_BLOCK` | `15m` | how long an address is blocked after the limit | | `SELFSYNC_RATE_REDEEM_GLOBAL` | `200` | wrong connect codes from all addresses in the same window; past it, one attempt a minute | | `SELFSYNC_RATE_AUTH_ATTEMPTS` | `30` | rejected tokens (401) from one address per window | | `SELFSYNC_RATE_AUTH_WINDOW` | `1m` | counting window for tokens | | `SELFSYNC_RATE_AUTH_BLOCK` | `1m` | how long an address is blocked after the limit | | `SELFSYNC_QUOTA_SOFT` | — | default per vault: storage size that triggers a warning (`5GiB`) | | `SELFSYNC_QUOTA_HARD` | — | default per vault: storage size past which new files are refused | | `SELFSYNC_QUOTA_DEVICES` | — | default per vault: active devices | | `SELFSYNC_QUOTA_MAX_FILE` | — | default per vault: largest file (never above `SELFSYNC_MAX_BLOB_SIZE`) | | `SELFSYNC_QUOTA_SERVER` | — | all vaults together: storage size past which new files are refused | In TOML the limits go into a `[rate]` section with the same names in lower case and without the prefix (`redeem_attempts = 10`, `auth_window = "1m"`); `trusted_proxy = true` is a top-level key. A threshold of `0` turns that limit off; windows and blocks must be at least one second, otherwise the server refuses to start. **Failed-attempt limits.** Codes and tokens are far too long to guess, but a scanner should neither load SQLite nor look like normal traffic in the log. The server counts only failures (a wrong connect code, a rejected token) and answers `429` with `Retry-After` once a limit is reached; the plugin and the command-line client wait that long without retrying. Counters live in `server.db`, so they work in CGI mode too; `selfsync sweep` removes those idle for a day. Requests without a token are not counted. The client address is the connection address (in CGI, `REMOTE_ADDR`). Behind a reverse proxy in `serve`/`socket` mode every request comes from the proxy: set `SELFSYNC_TRUSTED_PROXY=1` and let the proxy pass the address (the configs in [deploy/](deploy/) already do). Never enable it without a proxy that overwrites the header — anyone could then pick their own address. Without a known address (Unix socket, no trusted proxy) all clients share one counter. **Metrics.** With `SELFSYNC_METRICS_TOKEN` set, `GET /metrics` returns Prometheus text format; without it the route does not exist (404), with a wrong token it is 401. What it shows: server version and start time; per vault — files, folders, trash, history, current-version bytes, blob count and size, encryption, blobs requested from devices, and the last `selfsync verify` result (time, damaged and missing blobs, database errors, key record problem); per device — last contact time, client version, read-only flag; addresses currently blocked by the limiter; counters of 401, 403, 426, 429 and 5xx answers. Counters are kept in `server.db`, so a server that exits when idle does not lose them, and scraping does not count as activity: it does not keep a socket-activated server running. Device names are visible to whoever holds the token. Example: ```yaml scrape_configs: - job_name: selfsync scheme: https scrape_interval: 1m authorization: credentials: static_configs: - targets: ["sync.example.com"] ``` Each scrape lists the blob directory, so an interval of a minute or more is enough. A device silent for a week: `time() - selfsync_device_last_seen_timestamp_seconds > 7*86400`. **Quotas.** Off by default. A vault's storage is everything the server keeps for it: current versions, history and the trash. Limits are set per vault with `selfsync vault quota`; the `SELFSYNC_QUOTA_*` variables (or a `[quota]` section with `soft`, `hard`, `devices`, `max_file`, `server`) are the defaults for vaults without their own. `0` means no limit. - Past the soft limit, devices show a warning. - At the hard limit (or the server-wide `SELFSYNC_QUOTA_SERVER`), the server refuses new files. Edits that need no new upload still go through: deletes, renames, restoring a version the server already has. Downloads keep working, and so does repair from devices. Devices keep the refused files and send them once there is room. - Room is freed by emptying the trash or erasing old versions from a file's history (plugin and `selfsync-cli` 0.4.0+, not from read-only devices), by a shorter retention period, or by raising the limit. - At the device limit, new connect links and tokens are refused; revoking a device frees its place. ```sh selfsync vault quota --vault notes # show usage and limits selfsync vault quota --vault notes --soft 4GiB --hard 5GiB --devices 5 --max-file 100MiB selfsync vault quota --vault notes --hard default # back to the server default ``` ### Admin commands ```sh selfsync token add --vault notes --name laptop # device token (the vault is created on first use) selfsync link --vault notes --name phone # one-time link + QR code in the terminal selfsync token add --vault notes --name nas --read-only # receives changes, cannot send any selfsync token list | revoke --name phone # list shows each device's client (plugin/0.3.0) selfsync vault list # usage and limits of each vault selfsync vault quota --vault notes --hard 5GiB # see "Quotas" selfsync import --vault notes --from ~/Notes # upload an existing folder selfsync backup /backups/selfsync-$(date +%F) # consistent copy while running selfsync gc --vault notes # garbage collection plan; --yes to run it selfsync sweep # erase deleted files older than the retention window (timer) selfsync verify # integrity check: databases, blob hashes, missing blobs (timer) selfsync doctor # setup check: data dir, disk, address, proxy, timers, backups, verify, quotas selfsync healthcheck ``` `selfsync doctor` checks the installation and says what to fix: the data directory (writable, closed to other users), free disk space, the databases, `SELFSYNC_PUBLIC_URL` and TLS, `SELFSYNC_TRUSTED_PROXY` behind a reverse proxy, whether the public address answers (with `curl`, if installed), the verify, backup and sweep timers (with systemd), whether backups are fresh and on another disk, the last `verify` of each vault, and quotas. Exit code 1 means something is broken; warnings alone give 0. Back up with `backup`: the copy is consistent even while the server is running. To restore, put the directory back. Devices notice that the server went back in time and run a full reconciliation without deleting anything. What a backup protects against depends on how blobs get into it: | Command | Blobs | Protects against | |---|---|---| | `selfsync backup DIR` | hard links to the storage (copies on another file system) | `gc`, mistakes, a lost database; **not** a byte going bad on disk: a damaged blob is damaged in both places | | `selfsync backup DIR --copy` | real copies | also corruption of the storage | | `selfsync backup --rotate ROOT --keep 7 --copy` | real copies; between backups shared hard links, so each new one takes only the changes | the same, with the last 7 copies | `--rotate` writes each copy to `ROOT/` (first under a hidden `.partial` name, renamed when done), then removes the oldest beyond `--keep`; it only touches directories with its own naming. The `selfsync-backup.timer` unit does this daily into `/var/lib/selfsync-backups`. Mount another disk there: a copy on the same disk does not survive a failed disk. A backup is checked like the storage itself: `selfsync --data /var/lib/selfsync-backups/