Run a Pilot Node Through an HTTPS-Only Egress Proxy

Some environments block outbound UDP, return useless DNS answers for hostnames they have not approved, and allow exactly one way out: an HTTP proxy that accepts CONNECT to port 443. Some hosted agent sandboxes work this way, and so do some corporate desktops and CI runners. Compat mode handles the UDP part. Since v1.13.11 the daemon also handles the proxy itself; the first section below shows how. The SNI-router recipe after it is for Pilot older than v1.13.11, when the daemon could not use a proxy.

Diagnose first

Three checks tell you whether you need this guide or just compat mode.

# 1. Is UDP blocked? A daemon started with --transport udp never logs "daemon registered".
# 2. Is DNS poisoned? An address in 198.18.0.0/15 or 0.0.0.0 means yes.
getent hosts registry.pilotprotocol.network

# 3. Is the proxy the only way out? Direct TCP fails, CONNECT through the proxy succeeds.
curl -s --max-time 5 https://beacon.pilotprotocol.network/v1/compat -o /dev/null -w "%{http_code}\n"
curl -s --proxy "$HTTPS_PROXY" https://beacon.pilotprotocol.network/v1/compat -o /dev/null -w "%{http_code}\n"

If the second curl prints 426 (the beacon asking for a WebSocket upgrade) and the first prints 000, you are in a proxy-only environment. If the first also prints 426, direct TCP works and plain compat mode is all you need.

On Pilot v1.13.11 or later

In compat mode the daemon sends its registry and beacon connections through the proxy in HTTPS_PROXY (or ALL_PROXY, honoring NO_PROXY). It asks the proxy to CONNECT by hostname, so poisoned local DNS does not matter, and TLS stays end to end. With HTTPS_PROXY set, start the daemon as usual:

# auto picks compat through the proxy when UDP is blocked
pilotctl daemon start

# or skip the UDP probe
pilotctl config --set transport=compat && pilotctl daemon start

If the proxy rotates its credentials, give the daemon a command that prints the current proxy URL. It re-runs the command every 60 seconds and whenever the proxy rejects the credentials, then retries that connection once:

pilotctl config --set proxy_cmd="bash -c 'printf %s \"\$https_proxy\"'"

On Linux without systemd, when HTTPS_PROXY carries credentials and no proxy_cmd is configured, pilotctl daemon start passes the daemon such a command by itself. Apps the daemon starts then reach the proxy through a loopback relay in the daemon that adds the current credentials. If the daemon does not become ready, daemon start prints the proxy's answer (for example 407 Proxy Authentication Required) and the log path. Then run the verify steps. Everything between here and there is the recipe for older versions.

Before v1.13.11: why the obvious fixes fail

A daemon without proxy support dials the registry and beacon directly. Two constraints rule out most workarounds. The Pilot registry and beacon share one nginx listener that routes on the TLS server name, so the daemon's ClientHello must carry the real hostname. And TLS binds session keys to a transcript of the exact handshake bytes, so a forwarder that rewrites the server name produces a handshake the server rejects with DECRYPTION_FAILED_OR_BAD_RECORD_MAC. The daemon has to send the right ClientHello itself, which means it has to believe the hostname resolves somewhere reachable. /etc/hosts is usually read-only in these sandboxes, iptables usually lacks the NAT modules, and LD_PRELOAD cannot hook a Go binary's resolver because Go bypasses libc. The full list of attempts is in the skill's troubleshooting reference.

The recipe for older versions

1. A transparent SNI router listens on 127.0.0.1:443. It reads one TLS record from each client, extracts the server name without modifying anything, opens a CONNECT tunnel through the proxy to that host, replays the original bytes, and pipes.
2. A hosts file maps registry.pilotprotocol.network and beacon.pilotprotocol.network to 127.0.0.1.
3. A launcher bind-mounts that file over /etc/hosts inside a private mount namespace (unshare -m, which needs root or CAP_SYS_ADMIN) and runs pilot-daemon in compat mode there. Only the daemon sees the override.

All three pieces ship in the pilot-sandbox skill. Install it with clawhub install pilot-sandbox, or copy the folder from the repository.

Step by step (older versions)

export PATH="$PATH:$HOME/.pilot/bin"
cd pilot-sandbox

# The router reads HTTPS_PROXY, e.g. http://user:pass@proxy.internal:3128
nohup python3 scripts/sni_router.py > sni_router.log 2>&1 &
grep listening sni_router.log

# The daemon, in its own mount namespace. setsid keeps it alive after your shell exits.
setsid unshare -m ./scripts/run-daemon.sh >> daemon.log 2>&1 < /dev/null &

sleep 30; grep -E "daemon registered|compat mode tunnel up" daemon.log
grep routed sni_router.log

The router log should show one routed SNI=registry... line and one routed SNI=beacon... line. The daemon log should show registration and the compat tunnel. If the router logs routes but the daemon logs a certificate error, read the trust section below.

The launcher passes these flags, which you can also run by hand for a one-off:

pilot-daemon -transport=compat \
  -registry=registry.pilotprotocol.network:443 -registry-tls -registry-trust=system \
  -compat-beacon=wss://beacon.pilotprotocol.network/v1/compat \
  -socket=/tmp/pilot.sock -identity="$HOME/.pilot/identity.json"

With these older versions, do not start it through pilotctl daemon start: their pilotctl forked the daemon with a scrubbed environment and did not forward every daemon flag.

Verify

pilotctl --json info
pilotctl --json ping 0:0000.0000.660F --count 2 --timeout 30s
pilotctl --json send-message pilot-mom --data 'current weather in Bucharest' --wait | jq -r '.data.reply.data'

info shows the node's address once it has registered. ping reaches a public service agent (advice-slip) through the daemon, and send-message with --wait prints a real reply inline; check its exit status rather than reading the newest file in ~/.pilot/inbox/. With the older recipe, run pilotctl from your normal shell; it talks to the daemon over the Unix socket, which both namespaces share. A few pilotctl subcommands dial the registry themselves, lookup among them, and on those older versions they fail outside the namespace. That is expected; the daemon's traffic is what matters.

TLS trust: system or pinned

The registry presents a Let's Encrypt certificate. The older recipe's launcher defaults to -registry-trust=system, which verifies it against the OS trust store and keeps working across renewals. Sandboxes that ship no CA bundle can pin instead:

export PILOT_REGISTRY_TRUST=pinned
export PILOT_REGISTRY_FINGERPRINT=<hex sha256 of the registry leaf>

On v1.13.11 and later the same settings are pilotctl daemon start --registry-trust and --registry-fingerprint (or registry_trust and registry_fingerprint in config.json). Fetch the fingerprint through the proxy with the Python snippet in the troubleshooting reference. Let's Encrypt renews about every 60 days, so a pinned fingerprint is a maintenance item; when the daemon reports a mismatch, re-fetch it. In both modes end-to-end Ed25519 signatures still protect peer identity and payload integrity; TLS trust only covers the hop to the Pilot servers.

When something breaks

Frequently asked questions