The Haruka manual.

Introduction

Haruka combines four things most assistants keep separate: a real-time voice loop, an expressive 3D character, a model runtime that can stay entirely on your device, and a payment rail that lets the agent buy the tools it needs. The result behaves less like a chat box and more like a presence you speak to.

Architecture

Everything that can run in the browser does. The only outbound traffic is the model call you configure and the x402 settlement when a paid tool is used.

browser
├── voice layer      mic capture → ASR → turn detection → TTS playback
├── character layer  Live2D rig, touch + audio driven motion, 60fps loop
├── model layer      WebGPU local runtime  |  provider adapter (20+)
├── tool layer       function calling, sandboxed runners
└── payment layer    x402 client → Robinhood Chain wallet

Quickstart

Three steps from a cold open to a spoken conversation.

1. 01Open the app and allow microphone access when prompted.
2. 02Choose a model: WebGPU stays on device, or paste a provider key under Providers.
3. 03Press Start voice and talk. Enable paid tools under x402 only if you want them.

Real-time voice

The voice loop keeps latency low by streaming: partial transcripts feed the model before you finish speaking, and synthesis starts on the first sentence instead of the full answer. Barge-in is supported, so speaking over Haruka cancels playback and starts a new turn.

WebGPU acceleration

When the browser exposes WebGPU, Haruka loads a quantized model into GPU memory and runs inference locally. No prompt, transcript or audio leaves your machine in this mode. Check availability in the console:

if ("gpu" in navigator) {
  const adapter = await navigator.gpu.requestAdapter();
  console.log(adapter ? "WebGPU ready" : "No adapter");
}

First load downloads the weights once and caches them. Subsequent sessions start in a few seconds and work offline.

Live2D expressions

The character is rigged with blendshapes driven by three inputs: audio amplitude and phonemes for the mouth, sentiment of the current reply for the expression set, and pointer or touch position for gaze and body lean. Idle motion keeps breathing and blinking alive between turns.

PWA and cross-platform

Haruka is a progressive web app. Install it from the browser menu on desktop, or Add to Home Screen on mobile, and it opens in its own window with cached assets. The layout adapts from a phone portrait stage to a wide desktop split view.

20+ LLM providers

Providers are adapters over a single chat interface, so switching one out never changes the voice or character behaviour.

Keys and privacy

Provider keys are stored in your browser and sent only to that provider. Local modes need no key at all. Haruka has no account system and no server side transcript store.

Tool calling

Tools are declared as JSON schemas and offered to whichever model you selected. Free tools run immediately; tools marked as paid go through the x402 flow below.

{
  "name": "deep_search",
  "description": "Search the web and return cited passages",
  "paid": true,
  "price": { "asset": "ETH", "amount": "0.002", "chain": "robinhood" },
  "parameters": { "type": "object", "properties": { "query": { "type": "string" } } }
}

x402 pay-per-action

x402 revives the HTTP 402 Payment Required status as a machine-readable handshake. The tool answers with a price, Haruka signs a payment inside your session budget, then retries the request with the receipt attached. No subscription, no account, no card.

GET /v1/search?q=... HTTP/1.1

HTTP/1.1 402 Payment Required
X-Payment-Required: chain=robinhood; asset=ETH; amount=0.002; to=0xTool

GET /v1/search?q=... HTTP/1.1
X-Payment: <signed receipt>

HTTP/1.1 200 OK

Wallet and network

Payments settle on Robinhood Chain. Connect any injected EVM wallet such as MetaMask or Rabby, and approve the network switch if prompted.

Troubleshooting

Glossary