Search Results for author:
Found 5 papers, 5 papers with code
Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference
Our approach is inspired by two intriguing phenomena we have observed: (1) the attention sink phenomenon that is prevalent in LLMs also persists in MLLMs, suggesting that initial tokens and nearest tokens receive the majority of attention, while middle vision tokens garner minimal attention in deep layers; (2) the presence of information migration, which implies that visual information is transferred to subsequent text tokens within the first few layers of MLLMs.
CutDiffusion: A Simple, Fast, Cheap, and Strong Diffusion Extrapolation Method
Transforming large pre-trained low-resolution diffusion models to cater to higher-resolution demands, i. e., diffusion extrapolation, significantly improves diffusion adaptability.
ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models
Based on these properties, we develop ReMax, a new algorithm tailored for RLHF.
SMMix: Self-Motivated Image Mixing for Vision Transformers
Due to the subtle designs of the self-motivated paradigm, our SMMix is significant in its smaller training overhead and better performance than other CutMix variants.
Learning Best Combination for Efficient N:M Sparsity
In this paper, we show that the N:M learning can be naturally characterized as a combinatorial problem which searches for the best combination candidate within a finite collection.
