Web

The Web SDK (@xybrid/web) runs Xybrid models directly in the browser. It has two surfaces: a tensor surface (XybridModel) for TfLite models and a text-generation surface (XybridLlm) for .litertlm language models. Inference executes locally through WebAssembly, with WebGPU acceleration when available — no server round-trip.

Try the Demo

From bindings/web in the repository:

pnpm install
pnpm dev:example

The predev step downloads two pinned models (SHA-256 verified) and copies both runtimes' wasm assets — no model binary is committed.

• / streams SmolLM2-135M-Instruct (a 136 MB .litertlm model) on WebGPU or the CPU engine, with live download progress and generation timings.
• /tensor.html runs a deterministic 708-byte addition model and shows PASS only when every output matches.

pnpm test:browser drives both pages through the real adapters in Chromium.

Quick Start

Text Generation

import { XybridLlm } from "@xybrid/web";

const llm = await XybridLlm.load("https://models.example/llm/model_metadata.json", {
  wasmPath: "/llm-runtime",
  accelerator: "auto",
  onDownloadProgress: ({ loadedBytes, totalBytes }) => render(loadedBytes, totalBytes),
});

for await (const delta of llm.generateStream("Hello!", { maxOutputTokens: 256 })) {
  append(delta);
}

await llm.dispose();

Each generate / generateStream call is a fresh single-turn conversation; the model's embedded prompt template is applied by the engine. Abandoning the stream early or calling dispose() cancels in-flight decoding.

Tensor Inference

import { XybridModel } from "@xybrid/web";

const model = await XybridModel.load("https://models.example/model_metadata.json", {
  wasmPath: "/litert",
  accelerator: "auto",
});

const result = await model.run({
  a: new Float32Array(100),
  b: new Float32Array(100),
});

console.log(result.byName["Identity"]?.data);
await model.dispose();

Inputs are positional arrays or name-keyed records of Float32Array, Int32Array, or Uint8Array. When a model input has one dynamic dimension it is inferred from the data length; with several dynamic dimensions pass { data, shape } explicitly.

Serving Wasm Assets

Each surface loads its own wasm JavaScript and binary at runtime from wasmPath:

Wasm paths are executable code and must stay on the page's own HTTP(S) origin. Metadata and model files may live on another origin when that host permits CORS requests. Metadata, registry, Hugging Face, and model acquisition requests use credentials: "omit"; cross-origin sources still need to permit the requests with CORS. The preview initializes with threads: false and jspi: false, so there is no SharedArrayBuffer or COOP/COEP requirement.

Each runtime's initialization is a per-page singleton: concurrent loads sharing a wasm configuration reuse one initialization, and a later load with a different wasmPath fails with RuntimeConfigurationError.

Model Metadata

Both surfaces load Xybrid model_metadata.json documents. The browser boundary is forward-compatible — unrecognized fields are accepted — while malformed metadata, unsupported templates, and unavailable browser features become typed Xybrid errors.

API Reference

XybridModel

class XybridModel {
  static load(metadataUrl: string | URL, options: LoadOptions): Promise<XybridModel>;
  static fromRegistry(id: string, options?: RegistryLoadOptions): Promise<XybridModel>;
  static fromHuggingFace(repo: string, options?: HuggingFaceLoadOptions): Promise<XybridModel>;

  readonly inputs: readonly TensorDetail[];   // { name, shape, dataType }
  readonly outputs: readonly TensorDetail[];
  readonly accelerator: "wasm" | "webgpu";    // compile path actually selected
  readonly isFullyAccelerated: boolean;       // false when some ops fell back to CPU

  run(input: TensorInputs): Promise<RunResult>; // { outputs, byName }
  dispose(): Promise<void>;                     // idempotent, waits for in-flight run
}

XybridLlm

class XybridLlm {
  static load(metadataUrl: string | URL, options: LlmLoadOptions): Promise<XybridLlm>;
  static fromRegistry(id: string, options?: RegistryLoadOptions): Promise<XybridLlm>;
  static fromHuggingFace(repo: string, options?: HuggingFaceLoadOptions): Promise<XybridLlm>;

  readonly accelerator: "wasm" | "webgpu";

  generate(prompt: string, options?: GenerateOptions): Promise<string>;
  generateStream(prompt: string, options?: GenerateOptions): AsyncGenerator<string>;
  dispose(): Promise<void>;
}

Load Options

type LoadOptions = {
  wasmPath?: string | URL;                      // same-origin wasm asset directory
  accelerator?: "auto" | "wasm" | "webgpu";
  signal?: AbortSignal;
};

type LlmLoadOptions = LoadOptions & {
  onDownloadProgress?: (progress: { loadedBytes: number; totalBytes: number | undefined }) => void;
};

type RegistryLoadOptions = LoadOptions & {
  registryUrl?: string | URL;                   // HTTPS; defaults to the built-in registries
  version?: string;
  onDownloadProgress?: (progress: { loadedBytes: number; totalBytes: number | undefined }) => void;
};

type HuggingFaceLoadOptions = LoadOptions & {
  revision?: string;                            // defaults to "main"
  file?: string;
  onDownloadProgress?: (progress: { loadedBytes: number; totalBytes: number | undefined }) => void;
};

type GenerateOptions = {
  maxOutputTokens?: number; // positive integer
};

All load paths accept signal: AbortSignal. If the caller aborts, the load rejects with an abort error rather than a XybridError. A failure in one load stage aborts the shared load cancellation signal so sibling in-flight requests are cancelled.

The model is downloaded exactly once per load. With accelerator: "webgpu", initialization and the WebGPU availability probe complete before the model download; unavailable WebGPU therefore fails before any model bytes are fetched. With "auto", an unavailable WebGPU is skipped and wasm is selected silently. Other WebGPU initialization or compilation failures fall back to wasm using the same downloaded bytes; if wasm also fails, the RuntimeInitializationError keeps both failures in its AggregateError cause. "wasm" does not request WebGPU. On the LLM surface webgpu maps to the engine's GPU backend and wasm to its CPU backend. LiteRT may still execute unsupported WebGPU operations on CPU — check isFullyAccelerated to distinguish a fully delegated graph.

XybridModel.fromRegistry(id, options?) resolves a tensor model through the registry, and XybridLlm.fromRegistry(id, options?) resolves a LiteRT-LM model. registryUrl must be HTTPS; when omitted, the implementation tries its built-in registry endpoints. A registry resolution must provide a browser-compatible metadata document, an HTTPS download URL, a declared size no greater than 512 MiB, and a 64-character lowercase SHA-256 value. The downloaded bytes must match both the declared size and SHA-256.

XybridModel.fromHuggingFace(repo, options?) and XybridLlm.fromHuggingFace(repo, options?) accept a repository in org/name form. revision defaults to "main"; file selects a top-level .tflite or .litertlm file when needed. If the repository has model_metadata.json, that metadata selects and validates the model file; otherwise the loader synthesizes compatible metadata in memory when the selected format is unambiguous. The tree response supplies the declared size, using the LFS size when present, and a valid LFS OID is used as the SHA-256 to verify the download. Resolved downloads reject oversized or truncated bytes through size enforcement; registry artifacts and Hugging Face LFS files also fail closed on SHA-256 mismatches. Hugging Face files without a valid LFS OID have no tree-provided hash and therefore receive size enforcement only.

Error Handling

All SDK errors extend XybridError, which carries a stable code and the original causeValue:

Each model allows one in-flight run or generation at a time (ConcurrentRunError). DeviceLostError applies only to the tensor surface: XybridModel subscribes to WebGPU device loss and requires a new model after loss. XybridLlm has no device-loss subscription; its generation failures surface as InferenceError. XybridLlm.dispose() also settles after a consumer has stopped iterating generateStream, cancelling the abandoned generation before deleting the engine. Post-dispose calls throw DisposedError.

Limits

Browser memory guards: 1 MiB metadata, 512 MiB models, 256 MiB tensor I/O. Tensor dtypes are float32, int32, and uint8.

Deliberately deferred in this preview: preprocessing/postprocessing pipelines, voices and vision_encoder metadata, threads, JSPI, WebNN, GPU buffer I/O, higher-level multimodal APIs, and — on the LLM surface — multi-turn conversations, system prompts, sampling controls, constrained decoding, and tool calling.

Browser Support