Mesh LLM: distributed AI computing on iroh
Mesh LLM pools GPUs across machines using iroh's peer-to-peer networking (QUIC, hole-punching, relay) to run LLMs up to 235B parameters, splitting layers across nodes via pipeline parallelism (Skippy). It exposes an OpenAI-compatible API at localhost:9337 with no central server—each node is identified by a public key, and the pluggable architecture supports 40+ models for local, routed, or split execution.