Search Results
DenseFormer: Enhancing Information Flow in Transformers via Depth Weighted Averaging
zhaoolee/garss
•
•
The transformer architecture by Vaswani et al. (2017) is now ubiquitous across application domains, from natural language processing to speech processing and image understanding.
The Elements of Differentiable Programming
zhaoolee/garss
•
•
Artificial intelligence has recently experienced remarkable advances, fueled by large models, vast datasets, accelerated hardware, and, last but not least, the transformative power of differentiable programming.
FABLES: Evaluating faithfulness and content selection in book-length summarization
zhaoolee/garss
•
•
While LLM-based auto-raters have proven reliable for factuality and coherence in other settings, we implement several LLM raters of faithfulness and find that none correlates strongly with human annotations, especially with regard to detecting unfaithful claims.
Rendering string diagrams recursively
zhaoolee/garss
•
•
The algebraic representation can be seen as a term of a free monoidal category or a proof tree for a small fragment of linear logic.
Category Theory Computational Geometry
Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
zhaoolee/garss
•
•
Our method enforces a total compute budget by capping the number of tokens ($k$) that can participate in the self-attention and MLP computations at a given layer.
