Numpy LLM

This is a project I am working on during finals season of all times…

My focus is to get something that inferences

I will be attempting to make a scaled down version of Llama 4 Maverick

Architecture §

• Mixture of Experts (MoE)
• Group Query Attention