On-Device AI: Running Gemma 4 E2B Offline on an iPhone with LiteRT

Melvin Vivas · X video post · 2026-09-24 · 0:44 · 2,046 views · Open on X

Topics: Industry Trends & Job Market, LLMOps, Deployment & Monitoring, LLM Fundamentals · Level: intermediate

Summary

Melvin Vivas shows an iPhone app that runs Google's open model Gemma 4 E2B entirely on the phone, with no internet connection. He built the app using Devin's Mac cloud environment. The app downloads the model, loads it with Google's LiteRT runtime, and then answers questions offline. He calls on-device AI one of the most exciting directions in AI.

Key points

• Small open models like Gemma 4 E2B are small enough to run locally on a phone.
• How the app works: download the model once, load it with the LiteRT on-device runtime, then run inference on the phone.
• Once the model is loaded, the phone needs no internet to answer questions. The demo asks 'what is AI?' with the phone offline.
• The app was built with Devin, an AI coding agent, using its Mac cloud environment, which is useful for iOS builds.
• The creator presents on-device AI as an important trend: a 'ChatGPT-like' model running locally on your phone.

Resources mentioned

• Gemma 4 · tool · ai.google.dev · free
  Google's family of open-weight models in several sizes, built to run on devices and offline, with multimodal and agentic abilities, and open to fine-tuning.
  Also in: Gemma 4 Runs Locally On-Device in the Antigravity SDK (Melvin Vivas on X · notes), Running Gemma 4 Models Offline on an iPhone (Melvin Vivas on X · notes), Fine-tuning Gemma4-E2B on your own tweet style with Unsloth (Melvin Vivas on X · notes), Running Gemma4-E2B tool calling locally on an iPhone (Melvin Vivas on X · notes) and 25 more
• LiteRT · tool · ai.google.dev · free
  Google's on-device runtime (formerly TensorFlow Lite) for running ML models and LLMs on mobile and edge devices.
  Also in: Gemma 4 Runs Locally On-Device in the Antigravity SDK (Melvin Vivas on X · notes), LiteRT: Google's on-device AI runtime (Melvin Vivas on X · notes)
• Devin · tool · x.com · paid
  A desktop app from the makers of the Devin coding agent for planning, delegating, reviewing and shipping work across fleets of local and cloud coding agents.
  Also in: Using the Devin iOS App to Build iOS Apps (Quick Demo) (Melvin Vivas on X · notes), Use Your ChatGPT Plus/Pro Subscription to Run Devin (Melvin Vivas on X · notes), OpenAI DevDay recap: Dots, GPT-6.1 Sol, Codex Cloud, Agents API (Melvin Vivas on X · notes), Devin price cuts and top score on FrontierCode 1.1 Extended (Melvin Vivas on X · notes) and 44 more

Try this

• Watch the on-device AI space ('Watch out for on-device AI').
• Build an iPhone app that downloads a small open model (e.g., Gemma 4 E2B) and runs it offline with LiteRT as a local chat assistant.