Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
5 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on transformer
5 notes
Class Activation Mapping in Explainable Computer Vision: A Method-Centered Review of CNN, Transformer, and Foundation-Model-Era Visual Explanations
This research paper provides a comprehensive review of various visual explanation methods used in explainable artificial intelligence, specifically focusing on Class Activation Mapping (CAM). The paper discusses the…
The Transformer Revolution, Part 1: Dynamic Processing through Output- Weight Interconnections
The paper proposes a new interpretation of Transformers during inference, suggesting they construct prompt-dependent transformations whose parameters are generated during inference. This form of computation is named…
Transforming Rank: How Architecture Navigates the Spectral Pathologies of Depth
The research investigates how the design of Transformer feedforward block architecture affects the preservation of 'rank' (a mathematical concept related to the expressiveness of a network) across depth during…
Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks
The authors present a theoretical framework explaining the development of inductive reasoning abilities in Transformer language models. This framework applies to a wide range of inductive tasks and provides insights…
The State-Prediction Separation Hypothesis
Researchers propose a new model for language modeling called 'state-prediction separation' which aims to improve efficiency by separating the prediction of the next token and the storage of useful state for future…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
