From 38ff96c5760a80fb231494896d165c8f869a2e70 Mon Sep 17 00:00:00 2001 From: Jon Date: Thu, 4 Jun 2026 12:31:37 +0100 Subject: [PATCH] Scaffold SimpleX Orchestrate: supervisor over official binaries MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A standalone control-plane app that spawns and drives the official SimpleX binaries (never modifies simplex source). Validated against simplex-chat built from source (stable v6.5.4, GHC 9.6.3). - CLAUDE.md: architecture notes mined from the upstream docs (WebSocket bot API, per-profile DBs, directory/broadcast bot config) - supervisor/: process registry + port allocation (supervisor.py), corrId/cmd<->resp WebSocket client (ws_client.py), binary locator (binaries.py), FastAPI front with REST control + /events stream (server.py) - smoke_test.py: Pattern-1 handshake (spawn simplex-chat -p, create+read user) — PASS - group_test.py: two accounts, invitation connect + group invite/join, verified membership over the real SMP network — PASS - build_chat.sh / install_ghc.sh: reproducible toolchain + from-source build Key finding: fresh DB prompts for a display name on stdin; spawn with --create-bot-display-name to start the WebSocket server non-interactively. Co-Authored-By: Claude Opus 4.8 --- .gitignore | 11 +++ CLAUDE.md | 124 +++++++++++++++++++++++++++++++ build_chat.sh | 14 ++++ group_test.py | 125 ++++++++++++++++++++++++++++++++ install_ghc.sh | 11 +++ requirements.txt | 3 + smoke_test.py | 67 +++++++++++++++++ supervisor/__init__.py | 9 +++ supervisor/binaries.py | 24 ++++++ supervisor/server.py | 90 +++++++++++++++++++++++ supervisor/supervisor.py | 153 +++++++++++++++++++++++++++++++++++++++ supervisor/ws_client.py | 73 +++++++++++++++++++ 12 files changed, 704 insertions(+) create mode 100644 .gitignore create mode 100644 CLAUDE.md create mode 100755 build_chat.sh create mode 100644 group_test.py create mode 100755 install_ghc.sh create mode 100644 requirements.txt create mode 100644 smoke_test.py create mode 100644 supervisor/__init__.py create mode 100644 supervisor/binaries.py create mode 100644 supervisor/server.py create mode 100644 supervisor/supervisor.py create mode 100644 supervisor/ws_client.py diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..83e89c5 --- /dev/null +++ b/.gitignore @@ -0,0 +1,11 @@ +# Upstream simplex-chat checkout (cloned separately; never modified/committed here) +simplex-chat/ +# Built / downloaded binaries +bin/ +# Python env and caches +.venv/ +__pycache__/ +*.pyc +# Runtime per-profile databases + logs +data/ +*.log diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..0901113 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,124 @@ +# SimpleX Orchestrate — project notes for Claude + +Goal: a web-managed **orchestrator** that runs the **official SimpleX binaries** (instead of +re-implementing bot logic against the FFI `libsimplex` SDK, as the older `simplex-manager` +prototype did). The manager spawns/supervises real SimpleX processes, drives the interactive +ones over WebSocket, and reads the autonomous ones' output. + +Reference checkout: `./simplex-chat/` (cloned from https://github.com/simplex-chat/simplex-chat). +All facts below are taken from that repo's own docs — cited inline. There is **no** CLAUDE.md / +AGENTS.md in the upstream repo; the authoritative interface docs are: +- `simplex-chat/bots/README.md` — bot architecture & WebSocket API +- `simplex-chat/bots/api/COMMANDS.md`, `EVENTS.md`, `TYPES.md` — full command/event/type reference +- `simplex-chat/docs/CLI.md` — terminal client / DB / server flags +- `simplex-chat/apps/*/README.md` + `src/.../Options.hs` — the official bot binaries +- Official TypeScript SDK: `simplex-chat/packages/simplex-chat-client/typescript/` + +--- + +## Two integration patterns (this is the core design decision) + +**1. Interactive accounts → drive the CLI over WebSocket** (`bots/README.md`) +- Run the terminal client as a local WebSocket server: `simplex-chat -p 5225` +- Our process connects over WebSocket and sends JSON commands / receives events. +- This is how custom bots (TS/Python/Rust) and our manager talk to a profile. +- Same command strings as the app's *Settings → Developer tools → Chat Console*. + +**2. Autonomous official bots → run the prebuilt Haskell binary directly** +- `simplex-directory-service` and `simplex-broadcast-bot` are standalone executables built on + the chat core. They run their **own** logic, with their **own** DB + config, no WebSocket. +- The manager's job for these is lifecycle (spawn/stop/monitor/logs) + reading their output. +- **Use these instead of reimplementing directory/broadcast in Python** — they're battle-tested + and include captcha, moderation, approval, periodic re-checks, website generation, etc. + +--- + +## WebSocket protocol (pattern 1) — `bots/README.md` + +- `simplex-chat -p ` starts the WS server. **localhost only, NO authentication** — keep the + bot process on the same machine and firewall the port. Messages are unencrypted. +- Command: `{"corrId":"", "cmd":""}` +- Response: `{"corrId":"", "resp":{"type":"", ...}}` (matched by corrId) +- Event: `{"resp":{"type":"", ...}}` (no corrId — connections, received messages, etc.) +- `NewChatItems` arrives both as a command response (after `APISendMessages`) and as an event. +- Parser must **ignore unknown event types / union tags / extra fields** (forward-compat rule). +- Minimal bot loop: handle `NewChatItems` event → reply with `APISendMessages`. +- Network usage per command is documented: `no` / `interactive` / `background`. + +## Per-profile databases — `docs/CLI.md` + +- `simplex-chat -d ` → creates `_v1_chat.db` and `_v1_agent.db`. +- Default data dir `~/.simplex` (`%APPDATA%/simplex` on Windows). +- **One process = one DB prefix.** A single DB *can* hold multiple bot profiles via + `/create bot [files=on] []`, but the simple model is one process per profile. +- SMP servers: `-s smp://@host`. Tor: `-x` or `--socks-proxy=:9050`. `simplex-chat -h` for all. + +## Bot profile setup — `bots/README.md` (needs app/CLI v6.4.3+) + +- Mark a profile as a bot: `peerType: "bot"`. Set via: + - CLI flags `--create-bot-display-name`, `--create-bot-allow-files` on first start, or + - `/create bot [files=on] '' []`, or + - `APIUpdateProfile` (also sets command menus). +- Command menus: `/set bot commands '