Run Local Models on a Free GPU with Google Colab (T4)
Melvin Vivas · X video post · 2026-09-25 · 0:54 · 637 views · Open on X
Topics: Fine-tuning & Model Customization, LLMOps, Deployment & Monitoring, AI Dev Tools & Productivity · Level: beginner
Summary
Melvin Vivas shows how to use Google Colab to run open, local-style models on a free GPU, without buying hardware. You log in with a Google account, open any Jupyter notebook (.ipynb), choose the free T4 GPU, and check it with nvidia-smi. He suggests starting with inference before moving on to fine-tuning, and shares his own Colab notebooks for both.
Key points
- Google Colab (colab.research.google.com) gives you a free GPU, so you don't need to buy one to start running models.
- Log in with your Gmail/Google account. Colab can open any Jupyter notebook (.ipynb file).
- The free GPU is an NVIDIA T4 with about 15 GB of VRAM.
- Run
nvidia-smiin a notebook cell (!nvidia-smi) to confirm a GPU is attached and see how much VRAM you have. - Suggested order: get inference working first, then try fine-tuning.
- The creator's donvito/notebooks repo has fine-tuning and inference notebooks for local models that run in Colab.
Resources mentioned
- Google Colab · tool · colab.research.google.com · free · recommended by both Bashiri Smith & Melvin Vivas
Free GPU notebooks.
Also in: Run LightOnOCR-3 0.8B and Other Open Models on a Free Colab T4 (Melvin Vivas on X · notes), Try Liquid AI's open d1-3B decision model in Colab (Melvin Vivas on X · notes), Google Colab notebooks for new open models, built by a coding agent (Melvin Vivas on X · notes), Liquid AI's LFM2.5-24B-A2B: a local MoE model for agent tasks (Melvin Vivas on X · notes) and 27 more - donvito/notebooks · repo · github.com · free
The creator's notebooks for fine-tuning and running local models, which you can run in Google Colab, including a GLiNER2.5-Decide intent classification example.
Also in: Run LightOnOCR-3 0.8B and Other Open Models on a Free Colab T4 (Melvin Vivas on X · notes), Try Liquid AI's open d1-3B decision model in Colab (Melvin Vivas on X · notes), Try EmbeddingGemma 2 multimodal embeddings in Colab (Melvin Vivas on X · notes), Colab Notebook for Liquid AI's d1-3B Decision Model (Melvin Vivas on X · notes) and 10 more - Jupyter Notebook · tool · jupyter.org · free
An interactive notebook format (.ipynb) that mixes code and output. Colab can open these files.
Also in: AI DevBox v1.3.0: a GPU-ready Docker image with coding-agent CLIs (Melvin Vivas on X · notes), Q&A Over Your Own PDFs with LlamaIndex, OpenAI and Python (2023) (Melvin Vivas on melvinvivas.com · notes) - nvidia-smi · tool · developer.nvidia.com · free
An NVIDIA command-line tool that shows the attached GPU, its VRAM and how much is in use.
Also in: Monitor GPU Usage With nvtop Instead of nvidia-smi (Melvin Vivas on X · notes) - NVIDIA T4 GPU · tool · nvidia.com · free
The GPU that comes with Colab's free tier, with about 15 GB of VRAM. It's enough for small-model fine-tuning and inference.
Also in: Fine-tune and Run Local Models for Free on Google Colab's T4 GPU (Melvin Vivas on X · notes), Run Notebooks on a Free GPU with Google Colab (T4, 15GB VRAM) (Melvin Vivas on X · notes)
Try this
- Go to colab.research.google.com and log in with your Google/Gmail account.
- Switch the runtime to the free T4 GPU and run `!nvidia-smi` to check the available VRAM.
- Open a Jupyter notebook (.ipynb) in Colab and run model inference first.
- After inference works, try the fine-tuning notebooks in github.com/donvito/notebooks.
- Run inference with a small open-weight model on Colab's free T4, then fine-tune it with one of the creator's notebooks.
More in Fine-tuning & Model Customization
- Agent tasks: Unsloth fine-tuning, ComfyUI face swap, script writing
- Train Your Own Decision Model Locally with Unsloth
- Using an ML agent to train an open-source TTS model on your voice
- Free Colab Notebooks for Local Model Fine-Tuning and Inference
- Train Your Own LLM Model Router by Fine-Tuning ModernBERT as a Classifier
- Fine-tuning Liquid AI LFM2/LFM2.5 MoE models with the new Halo framework