A Streamlit chatbot that answers Canadian study-permit questions by retrieving from scraped Government-of-Canada pages and generating with Mixtral-8x7B-Instruct.
Architecture
As an international student applying for a Canadian study permit, I spent hours hunting across IRCC pages to answer questions that should take thirty seconds. A naive LLM hallucinates wrong answers on legal content; a retrieval-augmented pipeline grounds the answer in the actual IRCC text. I scraped the Student Direct Stream (SDS) documentation, chunked it, embedded it into a FAISS index, and served a LangChain stuff QA chain over a minimal Streamlit UI, with Mixtral-8x7B-Instruct as the generator.
The Government of Canada's Student Direct Stream documentation is spread across four long pages (overview, eligibility, how to apply, after you apply). For an applicant, the answer to a single question ("what bank drafts are accepted?", "which countries are SDS-eligible?", "what happens after I submit biometrics?") usually lives buried inside one specific section of one specific page. Ctrl+F works if you know the exact phrase IRCC used. Most applicants don't.
A naive LLM is worse than Ctrl+F for this task: it will confidently hallucinate an answer, and for legal/immigration content, a hallucinated answer is actively harmful. "ChatGPT told me" is not a defence when an officer refuses your application.
A retrieval-augmented setup keeps the model grounded: it retrieves the actual IRCC chunk before generating, so the answer stays anchored to the source text. I wanted to see how far a pure-retrieval approach could get without any fine-tuning.
The choice was between fine-tuning a model on IRCC Q&A pairs or building a retrieval pipeline over the raw pages. I picked retrieval for three reasons:
The tradeoff: retrieval needs a good chunking strategy, and chunking is harder than the tutorials suggest.
Why FAISS over Pinecone / Weaviate / Chroma. Local and free. The corpus is four pages; the entire embedded index fits in a few megabytes of RAM. FAISS builds in memory on startup, searches in microseconds, and keeps the project dependency-light. Past a few thousand chunks, a managed vector DB starts to make sense.
Why Mixtral-8x7B-Instruct over GPT-3.5 / GPT-4. Three reasons: (1) it topped the HuggingFace open instruct leaderboard at the time, (2) it's open-weight, so the project is reproducible without a paid OpenAI key, and (3) the goal was to learn the HuggingFace Hub serving path. Mixtral's mixture-of-experts architecture (8 experts, 2 active per token) gives GPT-3.5-class instruction following at a much lower effective compute cost. Temperature is 0.2 — low enough to keep the model pinned to the retrieved context, high enough to phrase answers naturally.
Why HuggingFace Hub over the OpenAI API. Using the HF Hub forces engagement with inference endpoints, rate limits, tokeniser alignment, and model selection — the model-serving layer the OpenAI SDK abstracts away. The free tier also keeps the project zero-cost.
Why Streamlit over Flask / FastAPI. The product is "a text box and an answer" — no routes, auth, or database models needed. Streamlit gives a UI for free, re-runs on input, and deploys in one command.
Why sentence-transformers/all-MiniLM-L6-v2 for embeddings. 384-dim, fast on CPU, good enough for short-passage retrieval on English government prose. Retrieval was already returning the right IRCC chunks on test queries, so benchmarking against MPNet, BGE, or E5 would have been premature optimisation.