Run Local Models on a Free GPU with Google Colab (T4)

Melvin Vivas · X video post · 2026-09-25 · 0:54 · 637 views · Open on X

Topics: Fine-tuning & Model Customization, LLMOps, Deployment & Monitoring, AI Dev Tools & Productivity · Level: beginner

Summary

Melvin Vivas shows how to use Google Colab to run open, local-style models on a free GPU, without buying hardware. You log in with a Google account, open any Jupyter notebook (.ipynb), choose the free T4 GPU, and check it with nvidia-smi. He suggests starting with inference before moving on to fine-tuning, and shares his own Colab notebooks for both.

Key points

• Google Colab (colab.research.google.com) gives you a free GPU, so you don't need to buy one to start running models.
• Log in with your Gmail/Google account. Colab can open any Jupyter notebook (.ipynb file).
• The free GPU is an NVIDIA T4 with about 15 GB of VRAM.
• Run nvidia-smi in a notebook cell (!nvidia-smi) to confirm a GPU is attached and see how much VRAM you have.
• Suggested order: get inference working first, then try fine-tuning.
• The creator's donvito/notebooks repo has fine-tuning and inference notebooks for local models that run in Colab.

Resources mentioned

• Google Colab · tool · colab.research.google.com · free · recommended by both Bashiri Smith & Melvin Vivas
  Free GPU notebooks.
  Also in: Run Notebooks on a Free GPU with Google Colab (T4, 15GB VRAM) (Melvin Vivas on X · notes), Free Colab Notebooks for Local Model Fine-Tuning and Inference (Melvin Vivas on X · notes), Intent classification for support using GLiNER2.5-Decide notebook (Melvin Vivas on X · notes), Getting started with GLiNER2.5-Decide in a Colab notebook (Melvin Vivas on X · notes) and 12 more
• donvito/notebooks · repo · github.com · free
  The creator's notebooks for fine-tuning and running local models, which you can run in Google Colab, including a GLiNER2.5-Decide intent classification example.
  Also in: Free Colab Notebooks for Local Model Fine-Tuning and Inference (Melvin Vivas on X · notes), Intent classification for support using GLiNER2.5-Decide notebook (Melvin Vivas on X · notes), Getting started with GLiNER2.5-Decide in a Colab notebook (Melvin Vivas on X · notes), Base vs fine-tuned Gemma 4 E2B as a model router (Melvin Vivas on X · notes) and 3 more
• Jupyter Notebook · tool · jupyter.org · free
  An interactive notebook format (.ipynb) that mixes code and output. Colab can open these files.
  Also in: AI DevBox v1.3.0: a GPU-ready Docker image with coding-agent CLIs (Melvin Vivas on X · notes), Q&A Over Your Own PDFs with LlamaIndex, OpenAI and Python (2023) (Melvin Vivas on melvinvivas.com · notes)
• nvidia-smi · tool · developer.nvidia.com · free
  An NVIDIA command-line tool that shows the attached GPU, its VRAM and how much is in use.
  Also in: Monitor GPU Usage With nvtop Instead of nvidia-smi (Melvin Vivas on X · notes)
• NVIDIA T4 GPU · tool · nvidia.com · free
  The data-center GPU (about 15 GB of VRAM) that Colab offers on its free tier.
  Also in: Run Notebooks on a Free GPU with Google Colab (T4, 15GB VRAM) (Melvin Vivas on X · notes)

Try this

• Go to colab.research.google.com and log in with your Google/Gmail account.
• Switch the runtime to the free T4 GPU and run `!nvidia-smi` to check the available VRAM.
• Open a Jupyter notebook (.ipynb) in Colab and run model inference first.
• After inference works, try the fine-tuning notebooks in github.com/donvito/notebooks.
• Run inference with a small open-weight model on Colab's free T4, then fine-tune it with one of the creator's notebooks.