Choosing a GPU Inference Path for On-Device AI: ML Drift vs. Framework Delegates vs. Native APIs

On-device inference used to be a problem you solved once per platform. You wrote a Metal path for iOS, an OpenCL or OpenGL ES path for Android, and something ugly for the browser, then spent the next two years chasing driver bugs. Google's release of ML Drift — an Apache 2.0, cross-platform GPU compute engine for on-device AI/ML inference that targets OpenGL ES, OpenCL, Metal, and WebGPU — is an attempt to collapse that work back into a single abstraction layer.

That's an attractive pitch, but "one engine for every GPU" is a claim that has been made, and partially abandoned, many times before. This guide compares ML Drift against the realistic alternatives, gives you a criteria table to score them, and recommends a path by scenario.

图示：来源页预览图（仅作编辑上下文；版权归属原发布方） · 来源引用：www.infoq.com

What ML Drift actually is (and isn't)

ML Drift is a GPU compute engine, not a model zoo or a training framework. Its job is to take the shader/kernel work that inference demands — matrix multiplies, convolutions, normalization, attention blocks — and schedule it across whatever GPU API the host device exposes, hiding the differences between OpenGL ES, OpenCL, Metal, and WebGPU behind one interface. The Apache 2.0 license matters: it means you can vendor it, patch it, and ship it inside a closed-source app without a commercial negotiation.

What it isn't: a guarantee of peak performance. Any abstraction layer that spans four graphics APIs cannot simultaneously exploit every vendor-specific tensor core or neural accelerator. That trade-off is the whole decision.

The options on the table

Option	What it gives you	What it costs you
ML Drift	One engine, four GPU backends, Apache 2.0, portable kernels	Newest entrant; ecosystem, operator coverage, and tuning depth still maturing
Native GPU APIs (Metal, OpenCL, OpenGL ES, WebGPU)	Maximum control, vendor-specific kernels, no third-party dependency	Four codebases, four bug surfaces, four shader languages
Framework delegates (LiteRT/TFLite GPU delegate, ONNX Runtime execution providers, ExecuTorch backends)	Model-format-native path, existing tooling, documented op coverage	Delegate quality varies wildly per platform; you inherit the framework's release cadence
Vendor SDKs (Core ML, Qualcomm QNN, NNAPI, MediaPipe)	Best-in-class perf on the vendor's silicon, NPU access	Lock-in, per-vendor integration, inconsistent model support
Ported engines (NCNN, MNN, ggml/llama.cpp)	Battle-tested mobile paths, strong LLM support in ggml	Performance is community-tuned, not vendor-tuned; API stability varies

The key insight: these are not mutually exclusive. Most production apps in 2026 will run a framework delegate or vendor SDK as the fast path and keep a portable engine as the fallback.

Criteria that actually decide it

Score each option 1–5 on these. Weights are typical for a cross-platform consumer app.

Criterion	Weight	Why it matters
Platform/backend coverage	High	How many of your shipping devices get a GPU path at all
Model/operator coverage	High	A fast path that can't run your model is worthless
Peak performance headroom	High	Gap between abstraction and hand-tuned kernels
Integration effort	Medium	Days to first token vs. weeks
Debuggability & profiling	Medium	Can you see which kernel is slow?
License & redistribution	Medium	Apache 2.0 vs. copyleft vs. commercial
Maturity & community	Medium	Bug response time, Stack Overflow density
WebGPU/browser story	Situational	Only if web is a target
NPU/accelerator access	Situational	Matters for sustained inference and thermal budgets

图示：HEXOPS 原创配图（受选题启发，非复制原文图片） · 来源引用：www.infoq.com

Where ML Drift wins

Portfolio breadth. If you ship an Android app, an iOS app, and a web build from one model artifact, ML Drift is the only option on this list that addresses all four backends with a single engine. Native APIs cost you four implementations; framework delegates often leave the browser as a separate project.

Licensing hygiene. Apache 2.0 with no commercial tier is a genuine differentiator versus vendor SDKs that require acceptance flows or restrict redistribution. For OEMs and app vendors with legal review gates, this shortens procurement.

Fallback economics. Even if ML Drift isn't your primary path, it makes an excellent floor. Devices whose drivers break your optimized path still get a working GPU implementation rather than a CPU fallback that burns battery.

Where ML Drift loses

Peak performance on flagship silicon. Vendor-tuned kernels and NPU delegates will beat a portable abstraction on the newest chips. If your app's value proposition is real-time video segmentation on a flagship phone, the abstraction tax is real.

Ecosystem maturity. A newly released engine has fewer answered questions, fewer published op-coverage matrices, and fewer people who've already hit your driver bug. Budget for that.

Operator gaps. Portable engines typically cover the common CNN/transformer ops well and the long tail poorly. Check your model's op list against the engine before committing — this is the single most common reason a portable path gets abandoned.

Recommendation by scenario

Cross-platform consumer app, mixed device fleet, CNN or small transformer models. Start with ML Drift as the primary engine. You get a uniform path across Android, iOS, and web, and you avoid maintaining four shader stacks. Keep a CPU fallback for pathological drivers.

Flagship-only experience with hard latency SLAs (AR, live camera effects). Use the vendor path — Core ML/Metal on Apple, QNN or NNAPI on Qualcomm — and keep ML Drift as the fallback for everything else. Measure the gap on your own models rather than trusting either side's numbers.

Model already in TFLite or ONNX and the team is small. Start with the framework delegate. LiteRT's GPU delegate and ONNX Runtime execution providers are the lowest-effort route, and you can swap in ML Drift later if delegate quality disappoints on specific devices.

Browser-first or WebGPU-forward product. ML Drift's WebGPU backend is the most interesting part of the release. If web is a first-class target, this is the shortest path to one engine across native and web.

LLM / generative inference on-device. Neither ML Drift nor framework delegates are the obvious answer yet; ggml-based stacks and vendor NPU SDKs currently dominate that niche. Treat ML Drift as a candidate for the non-LLM parts of your pipeline.

Regulated or long-support-cycle product. Apache 2.0 plus source availability is a strong argument for ML Drift, provided you can vendor and maintain a fork. Plan for that.

Adoption checklist

1. Dump your op list and diff it against ML Drift's supported kernel set. Do this first.
2. Build a device matrix — not just OS versions, but GPU families: Mali, Adreno, PowerVR, Apple silicon, and a WebGPU-capable browser.
3. Benchmark end-to-end latency, not kernel time. Driver overhead and shader compilation cost often dominate.
4. Instrument the fallback path so you can see how often devices silently drop to CPU.
5. Vendor the dependency rather than relying on a package manager version, given how young the project is.

Bottom line

ML Drift doesn't eliminate the trade-off between portability and peak performance — nothing does. What it changes is the default. Previously, portability meant CPU inference or a patchwork of delegates. Now there's a credible Apache 2.0 GPU engine that spans OpenGL ES, OpenCL, Metal, and WebGPU, and for a large class of apps that is the correct starting point. Reach for vendor SDKs when your latency budget demands it, and reach for framework delegates when your team is too small to own a backend. For everyone else, ML Drift is worth the integration week.

Sources

• InfoQ, "Google ML Drift Accelerates On-Device AI Inference Across Diverse GPUs," by Sergio De Simone — https://www.infoq.com/news/2026/10/google-ml-drift/
• ML Drift project documentation and Apache 2.0 license terms (refer to the upstream repository for the current supported-backend and operator-coverage matrix)