QLoRA fine-tune of Llama-2-7B that turns natural-language questions plus a table schema into SQL queries, trained on a free-tier Colab T4.

Dataset token-length distribution

I built this to understand fine-tuning from the inside. Text-to-SQL is a strong sandbox: a real, economically valuable task (analysts use it every day), clean evaluation (either the query runs and returns the right rows or it doesn't), and a simple dataset format that lets the focus stay on training mechanics. The whole project was constrained to a 16 GB VRAM + 12-hour free-tier Colab — a constraint that turned out to be useful. It forced sharp choices.

Problem

Base LLMs like Llama-2-7B can continue text but don't follow instructions well, and out of the box they hallucinate SQL that doesn't respect a given schema. Ask a base Llama-2-7B "count the students scoring greater than 85" with a student(marks INTEGER) schema and you'll often get back a plausible-looking query that references student.score, students (plural), or invents a students_table name entirely. That's the instruction-following gap.

Full fine-tuning a 7B model is out of reach on free Colab hardware: you'd need ~80 GB of VRAM just for the optimiser states and gradients in FP16. The goal was a reproducible recipe that fine-tunes Llama-2-7B for text-to-SQL on a single T4 with 16 GB VRAM, and ships the result to the Hugging Face Hub.

Why text-to-SQL is hard

It's easy to underestimate how many implicit decisions a SQL writer makes. A text-to-SQL model has to handle all of them:

• Ambiguous natural language. "Show me the top customers." Top by what — revenue, order count, recency? A human analyst would ask; a model has to guess or pick a convention.
• Schema awareness. The model has to reference actual column names, not hallucinated ones. "Students who scored above 85" needs to resolve to whatever the table actually calls the score column (marks, score, grade, points).
• Join reasoning. Multi-table questions require inferring the right join keys. This is where base models fall apart — they'll happily invent foreign keys that don't exist.
• Value quoting. WHERE country = 'Canada' vs WHERE country = Canada. Base models get this wrong surprisingly often; fine-tuning fixes it fast because there are thousands of consistent examples in the dataset.
• Aggregation + grouping. Knowing when a question needs GROUP BY, when it needs HAVING, when a COUNT(*) vs COUNT(DISTINCT x) is appropriate.

Fine-tuning teaches all of these at once, implicitly, by showing the model enough examples of "question + schema -> correct SQL" that it learns the mapping. The model doesn't need to reason about SQL semantics from first principles — it needs to pattern-match to the closest training example.

Why Llama-2-7B (and not 13B or 70B)

VRAM math, plainly:

• 7B parameters × 4-bit quantisation ≈ 3.5 GB for the base weights
• LoRA adapters (r=16 on all projections) add ~50 MB
• Optimiser states for the adapters (paged AdamW 8-bit) add another ~200 MB
• Activations during training at batch size 10 and seq length 512 eat roughly 5-8 GB