Llama RPC Farm
Open source GitHub

Distributed inference for hardware you own

Your GPUs.
One farm.

Turn spare gaming rigs, workstations, and Linux boxes into one coherent local AI cluster. One dashboard. One API. Every model served by the hardware you already have.

Get started Explore the source
Control planes
One
Machines
As many as you have
License
Apache 2.0
farm://overview live
Cluster overview Everything, at a glance
4 nodes online
VRAM 186/ 256 GB
RAM 312/ 512 GB
CPU 34%
aibox-01master
2× RTX 409048 GB
GPU 0
81%
GPU 1
67%
qwen3-32b · streaming
aibox-02worker
RTX 309024 GB
GPU 0
54%
ready for work
aibox-03worker
RTX 408016 GB
GPU 0
42%
ready for work
aibox-04worker
2× RTX 306024 GB
GPU 0
63%
GPU 1
58%
llama-3.3-70b · warm
↗OpenAI-compatible/v1/chat/completions
◈Live telemetrystreamed per machine

01 One control plane

Every box in the room,
working as one.

Llama RPC Farm sits above the mess: mixed GPUs, mixed operating systems, models spread across machines. You get one place to see it, route it, and serve it.

Your hardware
workstation2× RTX 4090 · 48 GB
gaming rigRTX 3090 · 24 GB
spare tower2× RTX 3060 · 24 GB
linux boxRX 7900 XTX · 24 GB
Llama RPC Farm one dashboard · one API cluster healthy
Served models
qwen3-32bchat · streamingGPU 0+1
llama-3.3-70bchat · warmsplit
whisper-largeaudio · idleGPU 2
embeddinggemmaembeddings · on demandauto
01Discover

Find every agent, GPU, and loaded model across the farm.

02Route

Place models where they fit. Split, shard, or keep them local.

03Serve

Expose one stable, OpenAI-compatible endpoint to your tools.

04Watch

Stream live VRAM, RAM, CPU, and model activity as it happens.

02 Built for real rigs

Serious infrastructure.
Not a science project.

Every surface is designed to make a heterogeneous pile of hardware feel like one deliberate machine.

One API. Every machine.

Route chat, embeddings, and completions through one OpenAI-compatible surface. llama-swap decides which model wakes up and where it lives.

Live telemetry, not a stale chart

Per-machine stats arrive as an NDJSON stream. Cards update the moment a host answers, so one slow box never freezes the rest of the farm.

RPC without the ritual

Agents, binaries, builds, and launch configs are handled for you. CUDA, ROCm, Vulkan, and mixed vendors are all first-class citizens.

Access that follows the user

Scoped API keys, web logins, per-key model visibility, and per-user themes. The right person sees the right models — and their own dashboard.

A dashboard with a pulse

Dot matrix. Voronoi. Circuit boards.
Floating wireframes.

Twenty background modes and dozens of color themes — from gemstone geology to famous sites, shows, books, and retro hardware. It should feel good to stare at.

CLI when you want it

farmctl speaks the same API as the dashboard. Script status checks, warm a model, or snapshot the farm for support without opening a browser.

03 Make it yours

A farm with taste.

Pick a palette, tune the glow, choose a background. Your theme travels with your access key, not the browser.

Theme studio

Obsidian

Volcanic black, iridescent violet, and molten copper. Built for late-night tinkering.

theme preview ● live
Cluster overview Good evening.
VRAM186 GB
RAM312 GB
CPU34%
qwen3-32b streaming llama-3.3-70b warm

04 Download

Run it locally.
Choose your platform.

Grab a release for your OS, or use the one-line installer. Every build and source wheel comes straight from the latest GitHub release.

latest release Release notes ↗
Farm appmaster control plane
Recommended
.deb

Linux

Ubuntu / Debian · x64

Download .deb latest release
.whl

Source wheel

Any OS · Python 3.11+

Download wheel latest release
One-line install
curl -fsSL https://raw.githubusercontent.com/dakotacookson/Llama-RPC-Farm/main/scripts/install-remote.sh | bash

Clones to ~/Llama-RPC-Farm, installs everything, and puts farmctl on your PATH.

Python 3.11+gitcurl
01Install

Use an installer or the source wheel. No admin account required for a local source install.

02Add machines

Discover peers and let the app provision the lightweight agent.

03Open the farm

Visit http://127.0.0.1:8199 and load your first model.

All releases and checksums ↗ Setup guide ↗ Peer agent downloads ↓
Peer agent one lightweight agent per remote machine

Optional, until you add a peer

Give every box a voice.

The peer agent is the typed, token-authenticated bridge between the farm and each remote machine. Install it once; the app discovers and pairs it automatically.

Token-authenticated Typed operations, no remote shell Linux · macOS · Windows
.whl

Universal wheel

Any OS · Python 3.11+

Download wheel latest release
One-line agent install
curl -fsSL https://raw.githubusercontent.com/dakotacookson/Llama-RPC-Farm/main/scripts/install-remote-agent.sh | bash

Installs the user service on port 8198. The farm app pairs it automatically during discovery.

05 Headless by default

The dashboard is optional.

farmctl is a first-class client over the same API. Perfect for scripts, cron jobs, remote shells, and the terminal you already live in.

farmctl statuscluster + llama-swap status
farmctl modelsconfigured models and variants
farmctl startstart the daemon, no warmup
farmctl launch --model IDwarm one model
farmctl unload IDfree its VRAM / RAM
farmctl doctorreadiness checklist
zazi@farm ~ connected 
$ farmctl status
cluster     4 nodes online
llama-swap  up · 127.0.0.1:8080
models      2 loaded / 14 configured
vram        186 GB used / 256 GB
$ farmctl launch --model qwen3-32b
✓ qwen3-32b loaded on aibox-01 (18.4s)
$
CLI reference ↗ API reference ↗

Your hardware is already enough

Stop renting compute.
Start farming it.

Open source, Apache 2.0, and built to run on the machines you already own.

Star on GitHub Setup guide