Ayush Garg

Search

SearchSearch

Recently Updated

  • Hackathon Applications

    Nov 01, 2026

    • TreeHacks 2027 Application

      Nov 01, 2026

      • Dataset Aggregation

        Oct 30, 2026

        • Inverse Kinematics

          Oct 30, 2026

          Home

          ❯

          Index of Notes

          ❯

          Transformer

          Transformer

          Apr 14, 2025, 1 min read

          The original transformer architecture from the Attention is All You Need paper

          Graph View

          Backlinks

          • Cohere Transformer Challenge
          • Pretraining
          • RMS Norm

          Created by Ayush Garg using Quartz , © 2026

          • GitHub
          • Linkedin
          • Blog
          • Twitter