Paired 3B inference on two Raspberry Pi nodes

Research question

When a second, differently equipped Raspberry Pi joins, does the network finish more independent inference work in the same time, and does a single answer get faster?

Hypothesis

Stated in the journal as an expectation rather than a formal hypothesis: if the scheduler matched model capabilities rather than hardware names, a CPU-only Pi could advertise the same pinned GGUF as the first node and accept an appropriate job.

Methodology

Holding one node at a time produced single-node measurements; releasing both let the scheduler place overlapping requests. The same 512-token stylish contact-form prompt was repeated across the three configurations, 36 requests in total. A separate 1,024-token comparison let the page finish naturally.

Baseline

Each node alone: Node 01 83.755 s mean compute and 87.427 s mean client wait; Node 02 82.639 s and 86.367 s. One 512-token request had about 86 to 88 s of client wall time, roughly 5.9 output tokens per second from the caller's view.

Interpretation

Two overlapping jobs finished in about the wall time of one. A documented pair produced 1,024 output tokens across two jobs in an 83.9 s span, about 12.2 output tokens per second of batch wall throughput, roughly twice the single-request figure. Each answer was still generated on one Pi and did not get faster. In the 1,024-token comparison the page finished naturally after 882 output tokens, with about 148.9 s of compute on Node 01 and 145.7 s on Node 02.

Limitations

All 36 requests ended at the 512-token limit, so they measure a fixed output budget, not completed pages. One session, not repeated. Thermal conditions differed: one Node 01 sample reached 81.8°C and latched the historical soft-temperature-limit flag, with active throttling bits clear when checked; Node 02 reached 70.8°C without a throttling flag. In the first fully simultaneous attempt one request returned node_busy although the other Pi was free; the placement retry came later and is not part of these results. The 28 October outage on Node 02 was not explained. No training was run.

Reproducibility

No repository artifact or raw log is attached. The figures are copied from the published journal entry.

Measured results