Toni D Voice Model

Toni D Voice Model screenshot

Llama 3.1 8B fine-tuned with QLoRA to write rap lyrics in my own voice, from a dataset built on my roughly 97-song catalog. A Custom GPT given the same data wrote the songs on the Suno playlist.

Listen on SunoView source

Built with

  • Python
  • PyTorch
  • Hugging Face
  • Llama 3.1
  • Kaggle
  • Weights & Biases
  • Supabase
  • Ollama

What it does

  • Writes verses, hooks and full songs in my style from a short prompt, ready for Suno to render as audio.
  • Runs entirely on my own machine: the adapter is converted to GGUF (Q8_0) with llama.cpp and served through Ollama.
  • A stylometric evaluation suite scores each output for syllable density, type-token ratio, verbatim memorization and output diversity.
  • Rehearsal data (persona prose and general-assistant examples) limits style bleed and keeps the base model's general abilities.

How it's built

  • QLoRA fine-tune (4-bit, r=16) on Kaggle GPUs, with experiments tracked in Weights & Biases and adapters hosted on Hugging Face.
  • Dataset pipeline: per-song phonetic, rhyme and structure analysis behind automated audit gates, then named-entity masking, section-marker normalization and prompt diversification, with a song-level train/validation holdout.
  • Custom Python tooling, including an AAVE-tuned phoneme mapper for syllable counting and rhyme detection.
  • Across three training runs I traced verbatim memorization to high-frequency completions, corrected syllable-density drift and overfitting, and fixed a TRL / Llama 3.1 chat-template loss-masking mismatch. Memorization is reduced, not eliminated.
  • Lyrics, analyses and training pairs live in Supabase (PostgreSQL).
  • The small model was promising, so I gave a frontier model the same data by compiling it into a Custom GPT (a feature OpenAI has since retired). Its songs were much stronger, and they are the ones on the Suno playlist.