Private, peer-to-peer AI inference for coworking spaces and hubs. Members get a desktop app with a chat UI and a local OpenAI-compatible API — all powered by GPU machines on the local network, connected directly via P2P. No cloud, no external servers, no data leaves your space.
A coworking hub has one or more GPU machines running AI models. Members install an Electron app, join with an invite code, and get:
- Chat interface with markdown rendering, conversation history, and performance stats
- Voice input via Whisper speech-to-text (runs locally, no network needed)
- OpenAI-compatible API on
localhost:11434— plug into VS Code Continue, LangChain, Python scripts, or any tool that speaks OpenAI
Everything connects over Hyperswarm P2P. The GPU provider and the member's app find each other automatically via DHT — no IP addresses, no port forwarding, no infrastructure.
┌─────────────────────────────────────────────────────────────────┐
│ Your Coworking Hub │
│ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ Auth API │ HTTP │ GPU Provider │ │
│ │ (admin PC) │◄───────►│ (GPU machine)│ │
│ │ :3000 │ │ Llama 3.2 1B │ │
│ └──────┬───────┘ └───────┬───────┘ │
│ │ │ │
│ │ login, quotas │ Hyperswarm P2P │
│ │ model registry │ (auto-discovery via DHT) │
│ │ │ │
│ ┌──────┴─────────────────────────┴──────┐ │
│ │ Member's Electron App │ │
│ │ │ │
│ │ Chat UI ──► P2P ──► GPU Provider │ │
│ │ Whisper ──► runs locally │ │
│ │ API :11434 ──► any OpenAI tool │ │
│ └────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
No central inference server. The Electron app connects directly to the GPU machine over P2P. The Auth API only handles authentication, quotas, and model registration — it never sees prompts or responses.
Members never deal with IP addresses, topics, or public keys. The invite code handles everything:
1. Admin creates invite code
└─► code grants access to the Auth API
2. Member pastes invite code in Electron app
└─► app registers with Auth API, gets a JWT token
3. App calls GET /models on the Auth API
└─► receives model entries with topic_hex + provider_public_key
┌──────────────────────────────────────────────┐
│ { "name": "llama-3.2-1b", │
│ "topic_hex": "66646f69686572...", │
│ "provider_public_key": "8d5b83fa..." } │
└──────────────────────────────────────────────┘
4. App uses topic_hex to find the provider on Hyperswarm DHT
└─► DHT connects the two peers directly (P2P)
└─► provider_public_key verifies it's the right machine
5. Member sees models in a dropdown, sends a message
└─► inference happens over encrypted P2P — Auth API is never in the path
The topic is like a room name on the DHT — the provider announces on it, and the app looks it up. The public key ensures no rogue peer can impersonate the provider. All of this is invisible to the member.
Runs the Auth API and manages the system. Typically the person who manages the coworking space.
Responsibilities:
- Start the Auth API server
- Register GPU providers and their models
- Generate invite codes for members
- Set and manage token quotas
A machine with a GPU that serves AI models. Can be a dedicated server, a workstation, or any machine with enough compute.
Responsibilities:
- Run the provider script with a model
- Stay online for members to connect
Anyone in the coworking space who wants to use AI. Gets an Electron app with chat + API access.
What they get:
- Chat UI with conversation history, markdown rendering, and voice input
- Local OpenAI-compatible API at
http://localhost:11434/v1 - Token quota managed by the hub admin
- Node.js 22+
- macOS 14+ (arm64) or Linux with Vulkan support
git clone https://github.com/builders-garden/hub-inference.git
cd hub-inference
npm install
cd packages/auth-api
# Initialize the database (first time only)
node src/cli.js init
# Start the Auth API
node src/cli.js start
# Auth API listening on 0.0.0.0:3000After the GPU Provider is running (see below), register the model and create invites:
# Register the LLM model (use the public key printed by the provider)
node src/cli.js model add llama-3.2-1b \
66646f696865726f6569686a726530776a66646f696865726f6569686a726530 \
<provider-public-key>
# Optional: register whisper for voice input (runs locally on member machines)
node src/cli.js model add whisper-tiny \
0000000000000000000000000000000000000000000000000000000000000000 \
0000000000000000000000000000000000000000000000000000000000000000
# Generate an invite code with 50,000 token quota
node src/cli.js member invite --api http://192.168.1.50:3000 --quota 50000
# Prints: Invite code: <code>Replace 192.168.1.50 with your machine's LAN IP so members on other devices can reach it.
git clone https://github.com/builders-garden/hub-inference.git
cd hub-inference
npm install
cd packages/provider
# Start serving an LLM
node provider.js 66646f696865726f6569686a726530776a66646f696865726f6569686a726530
# Prints: Public Key: <key> — give this to the hub adminThe topic (64-char hex string) is a shared identifier. The provider announces itself on this topic, and member apps find it automatically via DHT.
To serve multiple models, run additional providers in separate terminals with different topics:
# Whisper (speech-to-text) — optional, different topic
node provider.js aabbccddeeff00112233445566778899aabbccddeeff00112233445566778899 whispergit clone https://github.com/builders-garden/hub-inference.git
cd hub-inference
npm install
cd packages/desktop
npm run dev- Enter the hub address (e.g.
http://192.168.1.50:3000) - Paste the invite code from the admin
- Fill in name, email, password
- Start chatting
After the first login, just use Log in with your email and password.
While the Electron app is running, every member gets an OpenAI-compatible API on their machine at:
http://localhost:11434/v1
No API key needed. Any tool that works with OpenAI can connect.
| Method | Path | Description |
|---|---|---|
GET |
/v1/models |
List available models |
POST |
/v1/chat/completions |
Chat completion (streaming supported) |
curl:
# List models
curl http://localhost:11434/v1/models
# Chat
curl http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"llama-3.2-1b","messages":[{"role":"user","content":"Hello!"}]}'
# Streaming
curl http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"llama-3.2-1b","messages":[{"role":"user","content":"Hello!"}],"stream":true}'Python (OpenAI SDK):
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:11434/v1",
api_key="not-needed"
)
response = client.chat.completions.create(
model="llama-3.2-1b",
messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)Node.js (OpenAI SDK):
import OpenAI from 'openai'
const client = new OpenAI({
baseURL: 'http://localhost:11434/v1',
apiKey: 'not-needed'
})
const response = await client.chat.completions.create({
model: 'llama-3.2-1b',
messages: [{ role: 'user', content: 'Hello!' }]
})
console.log(response.choices[0].message.content)VS Code Continue (~/.continue/config.yaml):
models:
- model: llama-3.2-1b
title: Hub Inference
provider: openai
apiBase: http://localhost:11434/v1
apiKey: not-neededThe Electron app has a built-in API & Integrations page (accessible from the sidebar) with endpoint details, code snippets, and tool configuration guides.
Everything works out of the box. Members enter the admin machine's LAN IP as the hub address.
| Component | Tunnel needed? | Why |
|---|---|---|
| Auth API | Yes | Members need HTTP access. Use ngrok http 3000 |
| GPU Provider | No | Hyperswarm uses DHT + UDP hole-punching automatically |
| Electron App | No | Finds provider via DHT, reaches Auth API via tunnel |
Only the Auth API needs a tunnel. Members enter the ngrok URL (e.g. https://xxxx.ngrok-free.app) as the hub address.
To handle more concurrent users, run more GPU providers on the same topic:
# Machine B1
node provider.js 66646f696865726f6569686a726530776a66646f696865726f6569686a726530
# Machine B2 (same topic)
node provider.js 66646f696865726f6569686a726530776a66646f696865726f6569686a726530
# Machine B3 (same topic)
node provider.js 66646f696865726f6569686a726530776a66646f696865726f6569686a726530Hyperswarm distributes new connections across peers on the same topic. No load balancer needed.
All commands run from packages/auth-api. Accept --db <path> to specify the database file (defaults to ./hub.db).
node src/cli.js init # Initialize database
node src/cli.js start [--port 3000] [--host 0.0.0.0] # Start Auth API server
node src/cli.js model add <name> <topic-hex> <pub-key> # Register a model
node src/cli.js model list # List registered models
node src/cli.js model remove <name> # Remove a model
node src/cli.js member invite --api <url> [--quota N] # Generate invite code
node src/cli.js member list # List members + usage
node src/cli.js member set-quota <member-id> <tokens> # Set token quota- QVAC SDK — local AI inference with P2P delegation via Hyperswarm
- Hyperswarm — DHT-based peer discovery and connection (NAT traversal included)
- Electron + React + Tailwind — desktop app with chat UI
- Fastify + better-sqlite3 — lightweight Auth API
- JWT — 24h member authentication tokens
- react-markdown — rendered assistant responses with GFM support
hub-inference/
├── packages/
│ ├── provider/ # GPU provider script
│ │ └── provider.js # Loads model, starts P2P provider
│ ├── auth-api/ # Auth API + Admin CLI
│ │ ├── src/
│ │ │ ├── server.js # Fastify REST API
│ │ │ ├── cli.js # Admin CLI (init, start, model, member)
│ │ │ ├── db.js # SQLite schema + queries
│ │ │ ├── auth.js # JWT sign/verify
│ │ │ └── invite.js # Invite code + QR generation
│ │ └── test/ # Vitest tests
│ └── desktop/ # Electron member app
│ └── src/
│ ├── main/ # SDK consumer, local API server, IPC
│ ├── preload/ # Context bridge (IPC)
│ └── renderer/ # React UI (Chat, DevPage, Onboarding)
└── package.json # npm workspace root