Xybrid
SDKs

Flutter

On-device ML inference for Flutter applications

The Flutter SDK (xybrid_flutter) provides Dart bindings to the Xybrid Rust core via FFI, enabling on-device ML inference in mobile and desktop applications.

Requirements

  • iOS 16.0+
  • macOS 14.0+ on Apple Silicon
  • Android 9 / API 28+

The precompiled native libraries are built for these versions. For releases after 0.16.0 the plugin also declares them, so an app that targets an older version fails at build time. Raise your app's minimums:

  • Android: set minSdk = 28 or higher in defaultConfig of android/app/build.gradle.kts. Flutter's default, flutter.minSdkVersion, is lower.
  • iOS: set platform :ios, '16.0' in ios/Podfile and the Runner target's iOS Deployment Target to 16.0 in Xcode.
  • macOS: set platform :osx, '14.0' in macos/Podfile and the Runner target's macOS Deployment Target to 14.0 in Xcode.

Installation

Add to your pubspec.yaml:

dependencies:
  xybrid_flutter: ^0.20.0

Initialization

Initialize the SDK before using any Xybrid APIs:

import 'package:xybrid_flutter/xybrid_flutter.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await Xybrid.init();
  runApp(MyApp());
}

Local inference is available without authentication. An apiKey configures cloud access. To send execution traces to the Xybrid console, also configure observability explicitly:

await Xybrid.init(
  apiKey: const String.fromEnvironment('XYBRID_API_KEY'),
  observability: ObservabilityConfig.xybrid(
    apiKey: const String.fromEnvironment('XYBRID_API_KEY'),
    metadataOnly: true,
  ),
);

An API key alone does not enable export. The example selects metadata-only capture; otherwise enabled observability captures governed text by default. Use ObservabilityConfig.otlp(endpoint: 'http://collector:4318') for a Collector without Xybrid authentication. CPU/memory sampling is separately configured with resourceTelemetry. ingestUrl remains a legacy explicit opt-in to OTLP trace export; new integrations should use observability.

Key prefixes and scopes, what gets exported, and provisional workspaces: see API keys.

Xybrid.init() also subscribes to OS battery state on iOS and Android via battery_plus and forwards each reading to the routing engine.

Core Pattern: Load → Run

All inference follows the same pattern:

  1. Create a loader — specify the model by registry ID or local bundle
  2. Load — download (if needed) and initialize the model
  3. Run — execute inference with an input envelope
final model = await Xybrid.model('kokoro-82m').load();
final result = await model.run(XybridEnvelope.text('Hello world'));
print(result.text);

Text-to-Speech

final model = await Xybrid.model('kokoro-82m').load();
final envelope = XybridEnvelope.text('Hello, how are you?', voiceId: 'af_heart');
final result = await model.run(envelope);

// Get WAV-wrapped audio bytes for playback
final wavBytes = result.audioAsWav(sampleRate: 24000);

Speech-to-Text

final model = await Xybrid.model('whisper-tiny-ggml').load();
final envelope = XybridEnvelope.audio(bytes: audioBytes, sampleRate: 16000);
final result = await model.run(envelope);
print(result.text); // "Hello, how are you?"

For live transcription, open a session and feed PCM float32 mono 16 kHz as it arrives. flush() ends the session and returns an FfiTranscriptionResult: the transcript in text, plus durationMs, chunksProcessed and the session's observation reference.

final session = await XybridStreamSession.fromModel(model);
session.partials.listen((partial) => caption = partial.text); // listen first
session.feed(samples);
final transcript = (await session.flush()).text;

Earlier releases returned the transcript from flush() as a String. Read .text from the result instead.

LLM Chat with Streaming

final model = await Xybrid.model('gemma-3-1b').load();

final context = ConversationContext();
context.setSystem('You are a helpful assistant.');
context.pushText('Tell me a joke', MessageRole.user);

final envelope = XybridEnvelope.text('Tell me a joke');

await for (final token in model.runStreamingWithContext(envelope, context)) {
  stdout.write(token.token); // Print each token as it arrives
}

Loading with Progress

final loader = XybridModelLoader.fromRegistry('kokoro-82m');

await for (final event in loader.loadWithProgress()) {
  switch (event) {
    case LoadProgress(:final percentage, :final downloadedBytes, :final totalBytes):
      print('Loading: $percentage% ($downloadedBytes / ${totalBytes ?? '?'} bytes)');
    case LoadComplete():
      print('Model ready!');
    case LoadError(:final message):
      print('Error: $message');
  }
}

progress spans every file the model needs — weights plus companions such as a vision projector — and never moves backwards, so the bar neither restarts per file nor rewinds when a download retries. totalBytes is null when the source publishes no size (a Hugging Face repo); downloadedBytes is exact either way, which is what you need for megabytes, speed, and time remaining.

Loading a Local GGUF File

import 'package:path_provider/path_provider.dart';

// One directory per GGUF: a file the user picked, or an asset copied out of
// the app bundle.
final support = await getApplicationSupportDirectory();
final path = '${support.path}/models/my-model/my-model.gguf';
final model = await XybridModelLoader.fromModelFile(path).load();

A raw llama.cpp GGUF needs no model_metadata.json. Creating the loader touches no disk; load() generates the metadata from the GGUF header and writes it next to the file, with the context capped at 4,096 tokens (raise its context_length for longer chats). Put each GGUF in its own writable directory, such as one under getApplicationSupportDirectory(): assets declared in pubspec.yaml are not files on disk and the iOS app bundle is read-only, so model_metadata.json cannot be written next to either. A model_metadata.json already in the directory is used as is: a second GGUF in the same directory would load the first one's model. The generated metadata has no vision projector, so a vision GGUF with an mmproj still needs fromDirectory. A missing file makes load() throw an XybridException whose message contains GGUF file not found: <path>. With nothing to download, loadWithProgress() emits only LoadComplete or LoadError.


API Reference

Xybrid

Main entry point for the SDK.

class Xybrid {
  static Future<void> init({
    String? apiKey,            // cloud authentication; does not enable export
    String? gatewayUrl,        // override the LLM gateway URL
    String? ingestUrl,         // override the telemetry ingest URL (self-hosting)
    String? resourceTelemetry, // enable CPU/memory sampling
    ObservabilityConfig? observability, // explicit OTLP export configuration
  });
  static bool get isInitialized;
  static void setApiKey(String apiKey);

  static XybridModelLoader model(String modelId);
  static XybridPipeline pipeline({String? yaml, String? filePath});
}

XybridModelLoader

Creates and loads models from different sources.

class XybridModelLoader {
  factory XybridModelLoader.fromRegistry(String modelId);
  factory XybridModelLoader.fromRegistrySpeculative(String modelId);
  factory XybridModelLoader.fromBundle(String path);
  factory XybridModelLoader.fromDirectory(String path);
  factory XybridModelLoader.fromHuggingFace(String repo);
  factory XybridModelLoader.fromModelFile(String path);

  Future<XybridModel> load();
  Stream<LoadEvent> loadWithProgress();
}

Load events:

EventPropertiesDescription
LoadProgressprogress (0.0–1.0), percentage (0–100), downloadedBytes, totalBytes (nullable)Download/load progress
LoadComplete—Model is ready
LoadErrormessageLoading failed

XybridModel

A loaded model ready for inference.

class XybridModel {
  Future<XybridResult> run(XybridEnvelope envelope, {GenerationConfig? config});
  Future<XybridResult> runWithContext(XybridEnvelope envelope, ConversationContext context, {GenerationConfig? config});
  Stream<StreamToken> runStreaming(XybridEnvelope envelope, {GenerationConfig? config});
  Stream<StreamToken> runStreamingWithContext(XybridEnvelope envelope, ConversationContext context, {GenerationConfig? config});
}

XybridEnvelope

Input container for model inference.

class XybridEnvelope {
  factory XybridEnvelope.audio({
    required List<int> bytes,
    required int sampleRate,
    int channels = 1,
  });

  factory XybridEnvelope.text(String text, {String? voiceId, double? speed});
  factory XybridEnvelope.embedding(List<double> data);
  factory XybridEnvelope.textWithRole(String text, MessageRole role);

  XybridEnvelope withRole(MessageRole role);
}

XybridResult

Output from model inference.

class XybridResult {
  bool get success;
  String? get text;                  // ASR / LLM output
  Uint8List? get audioBytes;         // TTS output (raw PCM)
  List<double>? get embedding;       // Embedding output
  int get latencyMs;

  Uint8List? audioAsWav({int sampleRate = 24000, int channels = 1});
}

StreamToken

Token from streaming LLM inference.

class StreamToken {
  final String token;                    // Generated token text
  final int index;                       // Zero-based token position
  final String cumulativeText;           // Text emitted so far, tool-call blocks left out
  final bool isFinal;                    // Last token in sequence
  final String? finishReason;            // "stop", "length", "tool_calls", or "error: <message>"
  final XybridInferenceMetrics? metrics; // Final token only
  final ObservationRef? observation;     // Trace reference, final token only
  final List<ToolCall> toolCalls;        // Final token only; empty otherwise
  final String? rawText;                 // Raw turn text with the tool-call block; null without tool calls

  bool get hasToolCalls;                 // toolCalls is not empty
  bool get isError;                      // finishReason starts with "error:"
  String? get errorMessage;              // The message of an error token
}

ConversationContext

Multi-turn conversation management for LLM models.

class ConversationContext {
  ConversationContext();
  ConversationContext.withId(String id);

  String get id;
  int get historyLength;
  bool get hasSystem;

  void setSystem(String text);
  void setMaxHistoryLength(int length);
  void push(XybridEnvelope envelope);
  void pushText(String text, MessageRole role);
  void clear();  // Clears history, keeps system prompt
}
enum MessageRole { system, user, assistant }

XybridPipeline

Multi-stage pipeline for chaining models.

class XybridPipeline {
  factory XybridPipeline.fromYaml(String yaml);
  factory XybridPipeline.fromFile(String path);

  String? get name;
  int get stageCount;
  List<String> get stageNames;

  Future<XybridResult> run(XybridEnvelope envelope, {ObservationOptions? observation});
}

Pipeline Example

final yaml = '''
name: voice-assistant
stages:
  - whisper-tiny
  - gemma-3-1b
  - kokoro-82m
''';

final pipeline = XybridPipeline.fromYaml(yaml);
final result = await pipeline.run(
  XybridEnvelope.audio(bytes: audioBytes, sampleRate: 16000),
);
// Result is TTS audio from the final stage
final wav = result.audioAsWav();

observation: takes the same ObservationOptions as RunOptions.observation. result.observation is the pipeline's workflow reference, and result.metrics.stageLatenciesMs[i].observation is each stage's inference reference.


XybridException

class XybridException implements Exception {
  final String message;
}

All SDK errors throw XybridException, except in the token-streaming methods (runStreaming, runStreamingWithContext, runStreamingWithFallback): those end the stream with an error token instead (StreamToken.isError, StreamToken.errorMessage). Loading errors are surfaced via LoadError events when using loadWithProgress().


Platform Support

Precompiled libraries ship for iOS (device and Simulator, arm64), macOS on Apple Silicon, Android (arm64-v8a, x86_64), Linux (x86_64) and Windows (x86_64). Where a model runs depends on its format:

Model formatiOS deviceiOS SimulatormacOSAndroid, Linux, Windows
GGUF (llama.cpp: LLMs, vision-language models)Metal GPUCPUMetal GPUCPU
Whisper (whisper.cpp)CPU; optional Core ML encoderCPU; optional Core ML encoderMetal GPUCPU
ONNX (ONNX Runtime)CPUCPUCPUCPU

On iOS, whisper.cpp runs the encoder on Core ML when a <model>-encoder.mlmodelc (e.g. ggml-base.en-encoder.mlmodelc) sits beside the model file. ONNX Runtime's Core ML execution provider is linked on iOS and macOS, but the SDK does not select it yet, so ONNX models run on the CPU. No path uses NNAPI or DirectML.

On this page