AI/ML
Designing a Small Conversational Language Model with U.S. College-Admissions Data
A study of how Transformer-based language models work, applied to designing the architecture of a small conversational model built on U.S. college-admissions data. Work on diffusion-based language models is in progress.
- Field
- AI/ML
- Period
- 2025 – Ongoing (from Grade 11)
- Updated
- October 2026
Overview
Begun in Grade 11, this project looks at what it takes to build a conversational language model small enough to develop at school.
Scope
- Analyzed the structure and core principles of Transformer-based language models
- Examined where large language models are heading technically and where they fall short, including hallucination
- Designed a small conversational language model architecture using U.S. college-admissions data
Current Work
- Building a small language model and exploring diffusion-based language models