Magnemite
Development

The wire protocol

What travels over the device socket, in both directions.

One WebSocket per box, opened by the box, authenticated with its device token. The message set is small on purpose — it is implemented twice, in packages/protocol and in Go under agent/.

Box → hub

MessageWhen
helloOn connect: serial, model, Android version, installed Pokémon GO version, free space
heartbeatPeriodically, carrying the same live facts, and what the monitor probes saw
job_progressDownload percentage, install phase, log lines
job_resultSuccess or failure, with the verified version and whether data was wiped
agent_update_resultOnly on failure: the self-update did not happen, and why
log_bundle_resultOnly on failure: the log bundle was not collected or uploaded
log_linesA batch of live logcat lines, and how many were dropped keeping up
exec_resultWhat a one-off command printed, and whether it worked
config_resultA config file landed (or did not), with the sha256 read back off the box
pongAnswering a ping

Hub → box

MessageWhen
welcomeAccepting the socket, and what to monitor
install_jobArtifact URL, sha256, target version, hooks, flags
cancel_jobYou cancelled it, or the scheduler re-queued it
rebootFrom the device page
agent_updateA newer agent build is available
collect_logsZip the logs and upload them to this URL
log_stream_startFollow logcat, or a file, for this many seconds
log_stream_stopNobody is watching any more
exec_commandRun this shell command as root and say what it printed
pingLiveness

Configs ride the install

A MITM's config reaches a box one way: on install_job, as an optional config object. A rollout of a group's own MITM carries that group's config, written after the install verifies and before the post-install hook — the one moment the app is already stopped and the hook that starts it has not run yet.

There was a second path, a write_config message that pushed a config on its own when the config had changed but the app had not. It needed a restart command to go with it, because writing a file a running process has already read changes nothing by itself. Inside an install the post-install hook already restarts the scanner, so the command only earned its keep on the push, and the two were removed together. An edited config reaches the fleet by deploying the MITM again.

It is back for one box at a time, as push_config, sent when somebody presses Rewrite config in a device's Execute dialog — and carrying what the first one lacked. Its preHook and postHook are the group's own install hooks, run either side of the write, so the MITM stops, the file goes in, and the MITM starts again on it. The agent answers with push_config_result: ok is the write, with the sha256 read back off the box, and hookError says apart that a hook failed, since the file can be in place and the scanner still stopped. Neither side logs the content, which is why this is a message rather than a command typed into Execute: those land in the audit log, the hub's log and the box's own agent.log. Claimed as the push_config capability.

The write is still atomic rather than a truncate-then-fill a scanner can read halfway through, and the JSON never goes through a shell that would have to escape it.

Capabilities, not version arithmetic

hello carries an optional capabilities list — what this build of the agent can be asked to do beyond the message set that has always existed. The hub checks it before attaching a config to a job. The name is still write_config after the message of that name was removed: it says the agent can write a config file at all, and renaming it would make every already-deployed agent look incapable until it updated.

Comparing agentVersion instead would look equivalent and is not: it reads a backported build as too old, and what that costs is a message sent to a box that drops it in silence. An agent that sends no list is simply not asked.

Monitoring rides both of those

Monitoring added no message of its own. It travels as optional fields on two that already existed, which is why a fleet running last month's agents keeps working while it updates itself:

  • welcome carries a monitor spec — whether to report the focused app and any ANR, plus a list of probes to run. Like the heartbeat interval, a box learns a changed spec on its next welcome, which the hub pushes on save rather than waiting for a reconnect.
  • heartbeat and hello carry the answers, inside metrics: foregroundPackage, anrPackages and one checks entry per probe.

The load-bearing field is monitorRan. An empty foregroundPackage means the launcher is on screen, which is a fault worth acting on; it also means "this agent has never heard of monitoring", which must be left alone. The flag is what separates the two, and without it the hub cannot read any of the rest.

Properties worth knowing

  • The box always dials out. Nothing initiates a connection towards a box, which is why no ports are open on the LAN and CGNAT is irrelevant.
  • Jobs are idempotent by version. A box already on the target version reports success without installing anything.
  • Progress is advisory, absence is not. The job stall timeout (Settings → Hub settings) of silence with no job_progress is what makes the hub re-queue a job — a box unplugged mid-install never sends a failure.
  • Downloads do not use the socket. The bundle comes over plain HTTPS from the edge with a Range header, so an interrupted download resumes and the socket stays free for control messages.

Changing it

Change packages/protocol, mirror it in Go, and remember that a fleet updates its agents on its own schedule: a hub must keep understanding the previous message set until every box has taken an agent_update.

Additions do not move PROTOCOL_VERSION. It is a floor on the message set an agent must understand, not a change counter — bumping it would say old agents are incompatible, and they are not. install_job's config and hello's capabilities both arrived without touching it, and write_config and config_result came and went the same way — a hub that never sends a message type is indistinguishable, to an agent, from one that has not needed to yet.

What makes that safe is on the agent side: the type switch has no default branch, so an unknown message type falls straight through and is ignored, and Go's JSON decoding drops fields it does not know. An old box takes an extended install_job and installs without the config. Adding a field is free; repurposing or removing one is not.

On this page