RickLLM
Fine-tuned LLaMA 3.1 8B on a Rick and Morty transcript corpus (converted to a ChatML instruction dataset) using QLoRA via Unsloth on a single Colab T4 GPU, then quantized to GGUF for offline inference through Ollama with a Streamlit chat UI.
Rick and Morty transcripts | ChatML instruction dataset | LLaMA 3.1 8B (4-bit, Colab T4) | QLoRA adapters (Unsloth, gradient checkpointing + micro-batching) | Merged / exported model | GGUF | Ollama (local inference) -> Streamlit chat UI
Problem
Explore full fine-tuning mechanics on a single free-tier Colab GPU (16GB T4): taking a base instruction model and giving it a specific, consistent character voice rather than prompting for it, without renting production-grade hardware.
Architecture
Rick and Morty episode transcripts were converted into a multi-turn ChatML instruction dataset, then used to fine-tune LLaMA 3.1 8B in 4-bit with QLoRA via Unsloth. Gradient checkpointing and micro-batch training kept the run inside the T4’s memory budget; the merged adapters were exported to GGUF for offline inference through Ollama, wrapped in a custom Streamlit chat UI.
My Contribution
Built the dataset pipeline (transcripts to ChatML), the QLoRA/Unsloth training run, the memory-constrained training strategy (4-bit quantization, gradient checkpointing, micro-batching) needed to survive repeated Colab out-of-memory crashes, the GGUF export, and the Streamlit interface.
Outcome
A locally runnable, offline fine-tuned model with a consistent character voice, chattable through a custom Streamlit UI — distinct from the base model’s generic responses.