Xybrid

xybrid run

Execute a hybrid pipeline

Execute a hybrid pipeline on the current device.

Usage

xybrid run --config <path>
xybrid run --pipeline <name>
xybrid run --pipeline <name> --dry-run
xybrid run --directory <dir> --input-text <context> --choice <id>=<text>

Options

OptionShortDescription
--config-cLoad a pipeline YAML file directly
--pipeline-pLook up <name>.yml under examples/
--bundle-bRun a .xyb bundle file directly
--input-textText input for text-based models (LLM, TTS)
--input-audioPath to an audio file for audio-based models (ASR) — WAV, MP3, OGG, or FLAC
--choiceA candidate for a choice-scoring model, as ID=TEXT; repeatable. See Choice scoring
--output-oSave the result: audio as .wav, text as .txt, choice scores as JSON
--targetForce model format (onnx, coreml, tflite); auto-detected if omitted
--dry-runSimulate routing without execution
--policyLoad a policy bundle (YAML/JSON) that constrains routing — see Hybrid Execution › Policies
--traceEnable execution tracing with flame graph + LLM metrics output
--trace-exportExport the trace to a JSON file (Chrome trace format)

Behavior

  1. Parses YAML into PipelineConfig
  2. Runs policy → routing → execution loop
  3. Prints stage latency and routing summaries

In dry-run mode the CLI decides the first stage (policy and route) exactly as a real run would, at the current device snapshot, without running inference. Later stages are reported as unknown: their input is the previous stage's output, which does not exist without execution.

Examples

Run from config file

xybrid run --config ./pipelines/voice-assistant.yaml

Run named pipeline

xybrid run --pipeline hiiipe

Looks for hiiipe.yml or hiiipe.yaml in:

  • xybrid-cli/examples/
  • ./examples/

Dry run (simulate only)

xybrid run --pipeline hiiipe --dry-run

Output shows the first-stage decision without running inference:

  Stage 1 llm
  Declared         target: auto
  Provider         openai
  Policy           ALLOWED
  Reason           all policy checks passed
  Routing          local (default_local)

  Stage 2 tts
  Policy           UNKNOWN
  Routing          UNKNOWN
  Output           upstream output unavailable without execution

With a policy and Xybrid Platform

The primary hybrid example keeps the local model on the device and uses Xybrid Platform for cloud inference. Only the Xybrid API key belongs on the device; the upstream provider credentials are configured on the platform.

XYBRID_API_KEY="<your-xybrid-api-key>" xybrid run \
  -c crates/xybrid-cli/examples/hybrid-platform.yaml \
  --policy crates/xybrid-cli/examples/policies/prefer-cloud.yaml \
  --input-text "Summarise the plot of Dune in two sentences." -v

Swap in policies/local-only.yaml to keep the same prompt on the device, or policies/privacy.yaml to route by content. -v also prints the raw routing_decision JSON line.

Use --platform-url <base-url> for a staging or self-hosted platform. The example's cloud model is openai/gpt-4o-mini: the platform names frontier models by their canonical OpenRouter ID, and returns 404 model_not_found for a short ID such as gpt-4o-mini.

Direct provider testing

crates/xybrid-cli/examples/hybrid-deepseek.yaml uses backend: direct and DEEPSEEK_API_KEY for direct DeepSeek calls. Keep it for manual provider checks; automated policy-routing tests use a loopback fake DeepSeek endpoint with dummy credentials.

Sample Output

  Stage 1 llm
  Routing          cloud
  Reason           [local] policy_route_cloud: policy rule 'route_cloud_if_0' prefers cloud: true (confidence: 90%)
  Backend          cloud:openai:gateway
  Time             842ms
  Output           Text

    Dune follows Paul Atreides ...

Choice scoring

A choice-scoring model (a ChoiceScorer in its model_metadata.json) reads one context and a closed list of candidates and returns one score per candidate, without generating text. Pass the context with --input-text and each candidate with --choice ID=TEXT. The id names the candidate in the result and the model reads the text; the text is everything after the first = and may be empty. The model scores its own fixed choices after yours, so a scorer with at least two of them, like CUA-S1-FORMS, also runs without --choice.

CUA-S1-FORMS scores form actions against a page. From a checkout, stage it with ./integration-tests/download.sh cua-s1-forms (it regenerates the ONNX export from the pinned checkpoint), then:

xybrid run --directory integration-tests/fixtures/models/cua-s1-forms \
  --input-text 'TASK fill the form from the document, then submit
FORM Intake
ELEMENT Edit "Phone" value=""' \
  --choice tel="fill Tel: (503) 555-0142" --choice name="fill Name: Jane Doe" \
  --output scores.json
  ID     Score   Logit
  ────────────────────────────────
  tel    0.9997   29.9987  best
  name   0.0000    3.8055
  check  0.0000   -1.2357  fixed
  click  0.0000  -13.4527  fixed
  skip   0.0003   21.8863  fixed

  Margin           0.9994
  › Scores rank these candidates against each other; they are not calibrated probabilities

The rows come in the order offered: your choices, then the model's fixed choices (check, click and skip for FORMS). Score is the softmax of the logits over these candidates, Logit the model's raw output, and Margin the best score's lead over the runner-up. The highest score is marked best: equal scores go to the higher logit, and an exact tie to the first candidate. A score says how the model ranks the offered candidates, not how likely one is to be right, and acting on it is up to the caller. --output saves the scores at full precision, without the rounding:

{
  "entries": [
    { "id": "tel", "logit": 29.99869, "score": 0.9997003, "fixed": false },
    { "id": "name", "logit": 3.80547, "score": 4.2101643e-12, "fixed": false },
    { "id": "check", "logit": -1.2357253, "score": 2.7222987e-14, "fixed": true },
    { "id": "click", "logit": -13.452731, "score": 1.3463486e-19, "fixed": true },
    { "id": "skip", "logit": 21.886269, "score": 0.00029970234, "fixed": true }
  ],
  "label_mass": null
}

FORMS reads the first 224 bytes of the context and the first 96 bytes of each candidate, and cuts longer input without saying so.

A request is checked before the model runs, and by --dry-run too. It is refused when the context is empty, when an id is empty, repeated or one of the model's fixed ids, and when there are fewer than two candidates or more than the model accepts (64 for FORMS), fixed ones included. These refusals count positions from 0: a candidate in the order offered, a caller choice among your --choice flags and a fixed choice among the model's fixed choices. They never quote your ids or texts. A --choice with no =, or nothing before it, is a usage error, which repeats the argument like any other. A model that is not a choice scorer refuses --choice.

--choice needs --input-text, and cannot be combined with --config, --pipeline, --input-audio, --input-image, --voice, --max-tokens or --speculative-cloud. A choice scorer refuses --input-audio, --input-image, --voice and --max-tokens without --choice too. xybrid repl refuses a choice scorer, also as a pipeline stage, and so does --speculative-cloud, also with --dry-run.

Pipeline Format

The run command expects YAML pipelines:

name: "voice-assistant"
stages:
  - whisper-tiny@1.0
  - target: integration
    provider: openai
    model: openai/gpt-4o-mini
  - kokoro-82m@0.1

Tracing LLM Inference

When --trace is passed and the pipeline includes an LLM stage, the CLI prints an additional metrics block after the output:

📊 LLM Trace:
------------------------------------------------------------
  TTFT                  : 412 ms
  Decode TPS            : 42.80 tok/s
  Prefill TPS           : 184.20 tok/s
  Wallclock TPS         : 38.10 tok/s
  Mean ITL              : 14.30 ms
  p95 ITL               : 28 ms
  Emitted chunks        : 64
  Tokens generated      : 64
  Finish reason         : stop
------------------------------------------------------------

These values are read from the output envelope's metadata, populated by the LLM adapter's streaming path. Non-LLM stages (ASR, TTS, embedding) do not emit these fields and the block is skipped.

On this page