Ayush Garg

Search

SearchSearch

Recently Updated

  • Exponential Moving Average

    Jul 30, 2026

    • Revisiting Feature Prediction for Learning Visual Representations from Video

      Jul 30, 2026

      • #research-paper
    • Stop Gradient

      Jul 30, 2026

      • Clash Royale Agent

        Jul 22, 2026

        Home

        ❯

        Index of Notes

        ❯

        Transformer

        Transformer

        Apr 14, 2025, 1 min read

        The original transformer architecture from the Attention is All You Need paper

        Graph View

        Backlinks

        • Cohere Transformer Challenge
        • Pretraining
        • RMS Norm

        Created by Ayush Garg using Quartz , © 2026

        • GitHub
        • Linkedin
        • Blog
        • Twitter