Hybrid AI: The Next Pattern for AI-Powered Apps

Most applications treat AI as a cloud-only feature — send a request, wait for a response, pay per token. Open models like Gemma, Llama, DeepSeek, and Phi freed developers from proprietary APIs, but not from costs: you still need to host them somewhere — GPU servers, infrastructure, per-request pricing. You can run models directly on the user's device — zero inference costs, offline access, full data privacy. But there's a trade-off: models small enough to run on a phone or in a browser can't handle every task. Hybrid AI is the answer to this dilemma. Combine cloud and on-device models in a single application: simple and privacy-sensitive tasks run on-device, complex reasoning and multimodal workloads go to the cloud. The problem is that you end up maintaining two completely different inference stacks: different runtimes (TFLite, LiteRT-LM, llama.cpp), different model formats (TFLite flatbuffers vs GGUF), different quantization strategies, and different streaming APIs — all within one app. In this talk, I'll show how to solve this problem with Genkit — an open-source AI framework from Google with an open plugin system. Each plugin adapts a specific runtime or model format to a unified pipeline: genkit_flutter_gemma for TFLite/LiteRT models, genkit_llamadart for GGUF. Engine-specific details are hidden behind a single API for flows, structured output, tool calling, and agentic workflows. Need a new runtime? Write a plugin. New model format? Same story. On-device inference runs across Android, iOS, macOS, Windows, Linux, and Web — switching between cloud and local execution is a one-line model reference change. We'll explore hybrid patterns in practice: how to route between on-device and cloud based on model capabilities, connectivity, and task complexity. We'll cover the real trade-offs — quantization impact on output quality, memory constraints across platforms, cold start latency, and when hybrid actually makes sense versus pure cloud or pure edge. Open-source models made on-device AI possible. Open-source tooling makes hybrid AI extensible.