Ayush Garg

Search

SearchSearch

Recently Updated

  • Rapidly scaling online storage to serve over 1 billion ChatGPT users

    Sep 12, 2026

    • Agentic Systems

      Sep 01, 2026

      • Write thoughtful code

        Aug 28, 2026

        • Filesystem

          Aug 20, 2026

          Home

          ❯

          Index of Notes

          ❯

          Attention is All You Need

          Attention is All You Need

          Apr 19, 2025, 1 min read

          Paper Link: https://arxiv.org/pdf/1706.03762

          Concepts §

          • Transformer Encoder
          • Transformer Decoder
          • Scaled Dot-Product Attention
          • Multi-Head Attention
          • Self Attention
          • Sinusoidal Positional Encoding
          • Layer Normalization

          Graph View

          Backlinks

          • Transformer Encoder
          • Transformer

          Created by Ayush Garg using Quartz , © 2026

          • GitHub
          • Linkedin
          • Blog
          • Twitter