Research Scientist (LLM Architectures)

CommonAI CIC is hiring a Research Scientist to develop large language model architectures that lower inference costs while maintaining performance. The role involves designing and testing model components, analysing trained models, adapting open models, collaborating with engineers on reproducible training and efficient kernels, and publishing and presenting results.

Responsibilities:

• Design and implement new model components, such as attention variants, structured or sparse layers, routing mechanisms, activations and training objectives
• Run controlled pretraining experiments at matched compute, from small scale up to larger models
• Analyse the structure that trained models learn, and what it means for efficiency and interpretability
• Explore adapting existing open models through continued training or distillation
• Work with our engineers to turn research ideas into reliable, reproducible training runs and efficient kernels
• Write papers, release code and present results