Unity
On-device ML inference SDK for Unity applications
The Unity SDK (ai.xybrid.sdk) provides C# bindings to the Xybrid runtime via BoltFFI, enabling on-device ML inference in Unity games and applications.
Installation
The package contains only the C# API. It downloads the native libraries for its version from the matching GitHub Release, verifies their SHA-256, and installs them into Assets/Xybrid/Plugins/: the editor's own platform on first import, and the build target's before a player build if they aren't installed yet. The first import needs network access. To download them again, use Xybrid > Native Libraries > Download for Current Editor or Download for Active Build Target.
Minimum Unity version: 2021.3 LTS
OpenUPM (recommended)
Xybrid is published on OpenUPM. With the OpenUPM CLI:
openupm add ai.xybrid.sdkWithout the CLI, open Edit > Project Settings > Package Manager and add a scoped registry:
| Field | Value |
|---|---|
| Name | OpenUPM |
| URL | https://package.openupm.com |
| Scope(s) | ai.xybrid |
Then install Xybrid SDK from Window > Package Manager > My Registries. To declare the registry in Packages/manifest.json instead, or to install from a local clone, see the package README.
Git URL
In Unity, open Window > Package Manager > + > Add package from git URL (Install package from git URL in Unity 6) and enter:
https://github.com/xybrid-ai/xybrid.git?path=/bindings/unityWithout a tag, Unity installs the newest commit on master, but the native libraries come from the release named by that commit's package.json version, normally the latest release. To pin a release, append its tag after the path, as in ?path=/bindings/unity#v<version>. Tags are listed on GitHub Releases.
Initialization
Initialize the SDK once at startup:
using Xybrid;
void Awake()
{
XybridClient.Initialize();
}Initialize() is thread-safe and idempotent — multiple calls are no-ops.
On Android, models are cached under Application.persistentDataPath/xybrid (the SDK is given its models subfolder and keeps runtime-ready files beside it in extracted). To use another folder, call XybridBolt.XybridBolt.InitSdkCacheDir(path) before loading or listing any model; the first folder set wins.
Inference runs entirely on-device whether or not you authenticate. The apiKey authenticates the cloud gateway; gatewayUrl overrides it:
XybridClient.Initialize(apiKey: "sk_live_...");Key prefixes and scopes, what gets exported, and provisional workspaces: see API keys.
Observability
Trace export is off until you configure it — an API key alone does not enable it. Start it once with one of two presets:
// Traces to Xybrid
XybridClient.InitializeObservability(
ObservabilityConfig.Xybrid("sk_live_...")
.WithService(Application.productName, Application.version));
// Traces, metrics, and logs to an OTLP/HTTP endpoint (e.g. an OpenTelemetry Collector)
XybridClient.InitializeObservability(ObservabilityConfig.Otlp("http://collector:4318"));Prompts and outputs are captured by default; chain .WithMetadataOnly() to export metadata only. .WithTracesEndpoint(url) points the Xybrid preset at a self-hosted endpoint. Until export is configured, the first inference logs a one-shot hint (suppress with XYBRID_QUIET=1). Get a key at console.xybrid.dev.
ObservationOptions carries per-run trace context, and each result's Observation identifies its trace:
var result = model.Run(envelope, observation: new ObservationOptions
{
Traceparent = parentTraceparent,
ConversationId = chatId,
});
if (result.Observation is XybridBolt.XybridObservationRef run)
{
XybridClient.RecordFeedback(run, score: 1.0, label: "thumbs_up");
}
XybridClient.FlushObservability(TimeSpan.FromSeconds(2));Feedback and evaluations (XybridClient.RecordEvaluation(target, name, ...)) need a logs signal, so they return false with the Xybrid preset, which sends traces only. Call FlushObservability from OnApplicationPause(true) and ShutdownObservability from OnApplicationQuit: mobile OSes can kill a backgrounded app without a quit callback.
Quick Start
using Xybrid;
// Load and run in one line
var model = XybridClient.LoadModel("kokoro-82m");
using (var envelope = Envelope.Text("Halt! None shall pass without the king's seal."))
using (var result = model.Run(envelope))
{
result.ThrowIfFailed();
Debug.Log($"Output: {result.Text}");
Debug.Log($"Latency: {result.LatencyMs}ms");
}Model Loading
From Registry
using (var loader = ModelLoader.FromRegistry("whisper-tiny-en-ggml"))
{
var model = loader.Load();
// Use model for inference...
}From Local Bundle
using (var loader = ModelLoader.FromBundle("path/to/model.xyb"))
{
var model = loader.Load();
}Convenience Methods
// Load directly without managing the loader
var model = XybridClient.LoadModel("kokoro-82m");
var model = XybridClient.LoadModelFromBundle("path/to/model.xyb");Download Progress
load() blocks until the weights are on disk, so it has no progress to report
while it runs. Start the download separately and watch it:
using var download = ModelLoader.FromRegistry("qwen3-0.6b").StartDownload();
while (!download.IsFinished())
{
// Status() never blocks, so it is safe once per frame.
slider.value = download.Status().Progress;
yield return null;
}
using var loader = ModelLoader.FromRegistry("qwen3-0.6b");
var model = loader.Load(); // cached: returns at onceprogress spans every file the model needs — weights plus companions such as a
vision projector — and never moves backwards, so the bar neither restarts per
file nor rewinds when a download retries. totalBytes is null when the source
publishes no size (a Hugging Face repo); downloadedBytes is exact either way,
which is what you need for megabytes, speed and time remaining.
Cancelling the consuming task only unsubscribes from updates. Call cancel()
to stop the transfer itself — it takes effect within one chunk read and
discards the partial file.
Input Envelopes
Text
using (var envelope = Envelope.Text("The dragon sleeps in the northern cave."))
{
using (var result = model.Run(envelope))
{
Debug.Log(result.Text);
}
}Text with Role (for conversations)
using (var envelope = Envelope.Text("You are a blacksmith in a medieval village.", MessageRole.System))
{
// Use with ConversationContext
}Audio
using System.IO;
using UnityEngine;
using Xybrid;
// English ASR; use "whisper-tiny-ggml" for multilingual recognition.
using var model = XybridClient.LoadModel("whisper-tiny-en-ggml");
// A complete WAV file: 16-bit PCM, mono, 16 kHz.
byte[] wavBytes = File.ReadAllBytes("voice-command.wav");
using (var envelope = Envelope.Audio(wavBytes, sampleRate: 16000, channels: 1))
{
using (var result = model.Run(envelope))
{
result.ThrowIfFailed();
Debug.Log($"Transcription: {result.Text}");
}
}These GGML models use the speech-recognition runtime included in the prebuilt native libraries. For these models, Envelope.Audio and Model.RunAudio require a complete WAV file, including its header. For Unity Microphone input, encode the recorded samples as WAV first.
Convenience Methods
// Text-in, text-out shorthand
string output = model.RunText("Tell me about the ancient ruins.");
// Audio-in, text-out shorthand (player voice command)
// wavBytes contains a complete WAV file, as in the audio example above.
string transcription = model.RunAudio(wavBytes, sampleRate: 16000);Live Transcription
run() transcribes a finished audio buffer. A session transcribes speech as
it arrives, which is what dictation and live captioning need.
Audio must be PCM float32, mono, 16 kHz — converting from the platform's microphone format is the caller's job.
using System.Threading;
using Xybrid;
using XybridBolt; // the Partials() extension
using var session = model.Stream(StreamingConfigs.VoiceActivity(vadModelDir, language: "en"));
ShowPartials(session, cancellationToken); // not awaited: runs alongside the feed loop
// From the Microphone clip, converted to float mono 16 kHz:
session.Feed(pcm);
XybridTranscriptionResult result = session.Flush();
string transcript = result.Text;
// Started from the main thread, each await resumes there, so the UI can be updated.
async void ShowPartials(XybridStreamingSession live, CancellationToken token)
{
await foreach (var partial in live.Partials(token))
{
label.text = partial.Text;
}
}Partial text is cumulative, not a delta: render each one in place of the previous rather than appending. A partial produced before you subscribe is delivered immediately, so the opening words of an utterance are never lost.
Flush() ends the session and returns an XybridTranscriptionResult: the full
transcript in Text, plus DurationMs, ChunksProcessed and the session's
Observation reference. It throws if any chunk failed to transcribe, because
that audio is gone and the transcript would otherwise look complete.
Cancel() ends the session and discards queued audio.
Earlier releases returned the transcript as a string. Read .Text from the
result instead.
Voice-activity detection cuts chunks on speech boundaries instead of a fixed
clock, which avoids the stutter you get when a window lands mid-word. It needs
a Silero VAD model directory containing a model.onnx; none ships with the
SDK, and without one the engine falls back to fixed windows.
Result Handling
using (var result = model.Run(envelope))
{
if (result.Success)
{
Debug.Log($"Output: {result.Text}");
Debug.Log($"Latency: {result.LatencyMs}ms");
}
else
{
Debug.LogError($"Error: {result.Error}");
}
}
// Or throw on failure
using (var result = model.Run(envelope))
{
result.ThrowIfFailed();
// Safe to use result.Text here
}InferenceResult Properties
| Property | Type | Description |
|---|---|---|
Success | bool | Whether inference succeeded |
Error | string | Error message if failed |
Text | string | Text output (ASR, LLM) |
LatencyMs | uint | Inference latency in ms |
Observation | XybridObservationRef? | Trace/span ids of the run, when traced |
Conversation Context
Multi-turn LLM conversations with automatic history management:
using (var context = new ConversationContext())
{
context.SetSystem("You are a wise elder in a forest village. You speak in short, cryptic phrases.");
context.SetMaxHistoryLength(50); // FIFO pruning after 50 messages
// First turn
context.Push("Where is the lost temple?", MessageRole.User);
using (var envelope = Envelope.Text("Where is the lost temple?"))
using (var result = model.Run(envelope, context))
{
result.ThrowIfFailed();
context.Push(result.Text, MessageRole.Assistant);
Debug.Log(result.Text);
}
// Second turn (has full history)
context.Push("What dangers await there?", MessageRole.User);
using (var envelope = Envelope.Text("What dangers await there?"))
using (var result = model.Run(envelope, context))
{
result.ThrowIfFailed();
context.Push(result.Text, MessageRole.Assistant);
Debug.Log(result.Text);
}
// Clear history but keep system prompt
context.Clear();
}ConversationContext Properties
| Property | Type | Description |
|---|---|---|
Id | string | Conversation UUID |
HistoryLength | uint | Current message count |
HasSystem | bool | System prompt set? |
LLM Streaming
Unity supports token-by-token streaming for LLM models via callbacks:
// Check if the model supports streaming
if (model.SupportsTokenStreaming)
{
using (var envelope = Envelope.Text("Tell me a story about a brave knight."))
using (var result = model.RunStreaming(envelope, token =>
{
// Called for each token as it's generated
Debug.Log(token.Token);
}))
{
result.ThrowIfFailed();
Debug.Log($"Full response: {result.Text}");
}
}Streaming with Conversation Context
using (var context = new ConversationContext())
{
context.SetSystem("You are a tavern keeper in a fantasy RPG.");
context.Push("What's on the menu today?", MessageRole.User);
using (var envelope = Envelope.Text("What's on the menu today?"))
using (var result = model.RunStreaming(envelope, context, token =>
{
Debug.Log(token.Token); // Each token as it arrives
}))
{
result.ThrowIfFailed();
context.Push(result.Text, MessageRole.Assistant);
}
}Convenience Streaming
// Text-in, streaming text-out
string response = model.RunStreamingText("Describe the dungeon entrance.", token =>
{
// Update UI with each token
dialogueText.text += token.Token;
});IDisposable Pattern
All resource-holding classes implement IDisposable. Use using blocks to ensure cleanup:
using (var loader = ModelLoader.FromRegistry("kokoro-82m"))
using (var model = loader.Load())
using (var envelope = Envelope.Text("Your quest is complete, hero."))
using (var result = model.Run(envelope))
{
result.ThrowIfFailed();
Debug.Log(result.Text);
}Classes with IDisposable: Model, ModelLoader, Envelope, InferenceResult, ConversationContext
Error Handling
try
{
var model = XybridClient.LoadModel("nonexistent-model");
}
catch (ModelNotFoundException e)
{
Debug.LogError($"Model not found: {e.ModelId}");
}
catch (InferenceException e)
{
Debug.LogError($"Inference failed: {e.Message}");
}
catch (XybridException e)
{
Debug.LogError($"SDK error: {e.Message}");
}Exception Hierarchy
| Exception | Description |
|---|---|
XybridException | Base exception for all SDK errors |
ModelNotFoundException | Model ID not found in registry (has ModelId property) |
InferenceException | Inference execution failed |
Platform Support
| Platform | Architecture | Status |
|---|---|---|
| macOS 14+ | Apple Silicon (arm64) | Supported |
| macOS | Intel (x86_64) | Not supported |
| Windows | x64 | Supported |
| Linux | x64 | Supported |
| iOS | arm64 (devices, iOS 16+) | Supported |
| Android | arm64-v8a, armeabi-v7a | Supported |
| Android | x86_64 | Supported up to Unity 6.4 |
macOS: the native library requires macOS 14 or later and Apple Silicon (arm64) for both the editor and players. Set the player's minimum macOS version to 14.0 or later. Use the Apple silicon Unity Editor: the Intel editor can't load the library, on an Intel Mac or under Rosetta 2, which runs x86_64 code on Apple Silicon and not the reverse. Build macOS players for Apple Silicon. An Intel 64-bit + Apple Silicon player can load Xybrid only when it runs natively on an Apple Silicon Mac, not on an Intel Mac or when opened using Rosetta.
iOS: the library is built for devices running iOS 16 or later, with no Simulator build. Set Target minimum iOS Version to 16.0 or later.
Android: Unity 6.5 removed the x86_64 architecture (announcement), and Unity 6.3 and 6.4 offer it only in projects that already target it (Android Player settings). armeabi-v7a ships without ONNX Runtime, so ONNX models such as kokoro-82m don't run on it, and its library hasn't been loaded on a device yet (#680).