On-Device AI: Running Gemma 4 E2B Offline on an iPhone with LiteRT
Melvin Vivas · X video post · 2026-09-24 · 0:44 · 2,046 views · Open on X
Topics: Industry Trends & Job Market, LLMOps, Deployment & Monitoring, LLM Fundamentals · Level: intermediate
Summary
Melvin Vivas shows an iPhone app that runs Google's open model Gemma 4 E2B entirely on the phone, with no internet connection. He built the app using Devin's Mac cloud environment. The app downloads the model, loads it with Google's LiteRT runtime, and then answers questions offline. He calls on-device AI one of the most exciting directions in AI.
Key points
- Small open models like Gemma 4 E2B are small enough to run locally on a phone.
- How the app works: download the model once, load it with the LiteRT on-device runtime, then run inference on the phone.
- Once the model is loaded, the phone needs no internet to answer questions. The demo asks 'what is AI?' with the phone offline.
- The app was built with Devin, an AI coding agent, using its Mac cloud environment, which is useful for iOS builds.
- The creator presents on-device AI as an important trend: a 'ChatGPT-like' model running locally on your phone.
Resources mentioned
- Gemma 4 · tool · ai.google.dev · free
Google's family of open-weight models in several sizes, built to run on devices and offline, with multimodal and agentic abilities, and open to fine-tuning.
Also in: Meetily: Open-Source Local AI Meeting Notes App (Melvin Vivas on X · notes), New Gemma Developer Documentation Site (Melvin Vivas on X · notes), A100 80GB GPUs in Google Colab via Google AI Pro plans (Melvin Vivas on X · notes), A day of work done from a phone with Grok Bot agents (Melvin Vivas on X · notes) and 32 more - LiteRT · tool · ai.google.dev · free
Google's on-device runtime (formerly TensorFlow Lite) for running ML models and LLMs on mobile and edge devices.
Also in: Gemma 4 Runs Locally On-Device in the Antigravity SDK (Melvin Vivas on X · notes), LiteRT: Google's on-device AI runtime (Melvin Vivas on X · notes) - Devin · tool · x.com · paid
A desktop app from the makers of the Devin coding agent for planning, delegating, reviewing and shipping work across fleets of local and cloud coding agents.
Also in: Devin's open-source vulnerability-fixes agent plugin (Melvin Vivas on X · notes), Gemini 4 Argon now available in Devin (Melvin Vivas on X · notes), Devin Defaults to Opus 5.5 in Normal and Fusion Modes (Cloud) (Melvin Vivas on X · notes), Devin: Preview Websites in Its Built-in Cloud Browser (Melvin Vivas on X · notes) and 57 more
Try this
- Watch the on-device AI space ('Watch out for on-device AI').
- Build an iPhone app that downloads a small open model (e.g., Gemma 4 E2B) and runs it offline with LiteRT as a local chat assistant.
More in Industry Trends & Job Market
- ElevenLabs Eleven v4 & v4 Turbo: Emotive, Multilingual Text-to-Speech Models
- Claude Sonnet 5.5 release beats Opus 5.5 on Terminal-Bench
- Cognition's Devin Hits $1B Run Rate: AI Coding Agents in the Enterprise
- Unsloth passes 500M model downloads on Hugging Face
- ChatGPT Voice Update: Plugins, Model Switching and ChatGPT Work
- Gemini 3.8 Flash and Flash-Lite TTS: new text-to-speech models