A series of AI models that I am building from scratch. Tokenizer, training code, and inference engine all written by hand. Trained on a single RTX 5070 Ti, based in the Northeastern part of Bangkok, Thailand.
Zeterelle is my series of experimental language models. I write everything myself — the tokenizer, the model architecture, the training loop, and the code that runs inference — and train it all at home on one GPU.
This is not a product. It is a hands-on project about how far a single person and a single graphics card can go. The models are small, honest about their limits, and published as I go.
ZTR-L-1 is the first language model I have ever made from scratch. It is a 303-million parameter transformer, pretrained on one billion tokens and then fine-tuned to hold simple conversations.
Pretraining and fine-tuning both ran on my single RTX 5070 Ti. The whole stack is written in TypeScript and runs on WebGPU, so it can chat directly in the stack it was trained in. The weights and the tokenizer are public on Hugging Face.