GPU for Agents · open source

Your agent rents the GPU it needs, on its own.

You pay only while it works: we watch every machine and shut down the idle ones.

Claude Code, Cursor or any MCP client rents a GPU, serves a model behind an OpenAI-compatible API and runs its experiment. No dashboard, no ticket, no GPU left billing after the conversation ends.

Watch the 1-minute video

The pitch, in one minute

1:04 · sound on

Two promises

The agent gets the GPU.
You pay only for the work.

01 · PROVISION

Your agent rents what it needs.

Ask for a benchmark. The agent checks prices against your budget, rents the machine, serves the model with vLLM and gets an endpoint back. It runs the experiment with any OpenAI SDK. No dashboard, no ticket, no one waiting on a human.

Compare Qwen 7B and Llama 8B on these 50 prompts.

gpu_offersRTX 4090 · 24 GB · $0.38/h ✓ under $0.60/h cap

gpu_setuprenting the machine…

readybase_url https://…:8000/v1 · model Qwen/Qwen2.5-7B-Instruct

02 · WATCH

You pay only while it works.

We watch every machine. A watchdog runs on the GPU itself, not on your laptop: after 15 minutes without a request, it shuts the machine down and the billing ends. It works if the agent forgot, if the chat closed, if your laptop is asleep.

requestsidle 00:00 / 15:00

agent workinglast requeststop

RUNNING$0.38/h

How it works

One request, start to finish.

The agent drives GPU for Agents through MCP tools (or the CLI with --json). The skill tells it to show you the price and ask before renting.

Ask

"Run this eval on Llama 8B." The agent writes jambu.yaml with gpu_init.

Price

gpu_offers lists matching GPUs under your cap. You say yes.

Serve

gpu_setup rents, boots vLLM and returns base_url, api_key, model. A stalled host is replaced.

Run, then stop

gpu_run runs the job. When requests stop, the watchdog stops the GPU.

Guardrails

Limits the agent can't talk its way past.

They live in jambu.yaml and on the machine. The agent can't raise them without editing the file, and the skill tells it to ask you first.

15 min

Idle timeout

No requests, no GPU. The watchdog on the instance stops it. lifecycle.idle_timeout

$/h

Hourly and session budget

Offers above the cap are never rented; a session that hits its total stops. budget.max_hourly_cost_usd

6 h

Maximum lifetime

A hard ceiling, busy or not. lifecycle.max_lifetime

10 min

Stalled hosts replaced

An image pull that stops moving gets a new host instead of a 30-minute wait. health.pull_stall_timeout

sweep

Last line of defense

gpu ls shows everything billing; gpu sweep --install-timer cleans up orphans.

jambu.yaml

gpu_runtime:
  provider: {name: vast}
  model_profile: qwythos-9b
  lifecycle:
    idle_timeout: 15m   # watchdog stops it
    max_lifetime: 6h
  budget:
    max_hourly_cost_usd: 0.60
    max_session_cost_usd: 5.00
  health:
    pull_stall_timeout: 10m

Install

Two commands.

Python 3.10+, a Vast.ai API key in .env or ~/.gpu/credentials. The agent never sees the key.

$ pipx install gpu4agents

$ cd your-project

$ echo "VAST_API_KEY=..." >> .env

$ gpu agent install # Claude Code

-t cursor-t desktop--global

# or drive it from any app, any language

$ gpu --json setup

{ "base_url": "https://…/v1",

"api_key": "…", "model": "…" }

$ gpu run python eval.py

$ gpu destroy # or just walk away