transformer
5 notes, newest first.
Class Activation Mapping in Explainable Computer Vision: A Method-Centered Review of CNN, Transformer, and Foundation-Model-Era Visual Explanations
This research paper provides a comprehensive review of various visual explanation methods used in explainable artificial intelligence, specifically focusing on Class Activation Mapping (CAM). The paper discusses the…
The Transformer Revolution, Part 1: Dynamic Processing through Output- Weight Interconnections
The paper proposes a new interpretation of Transformers during inference, suggesting they construct prompt-dependent transformations whose parameters are generated during inference. This form of computation is named…
Transforming Rank: How Architecture Navigates the Spectral Pathologies of Depth
The research investigates how the design of Transformer feedforward block architecture affects the preservation of 'rank' (a mathematical concept related to the expressiveness of a network) across depth during…
Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks
The authors present a theoretical framework explaining the development of inductive reasoning abilities in Transformer language models. This framework applies to a wide range of inductive tasks and provides insights…
The State-Prediction Separation Hypothesis
Researchers propose a new model for language modeling called 'state-prediction separation' which aims to improve efficiency by separating the prediction of the next token and the storage of useful state for future…
