Attention Sink

The Attention Sink is a phenomenon observed in Transformer-based Large Language Models (LLMs). It describes how the first token of a sequence (often a 'Beginning of Sequence' or BOS token) receives disproportionately high attention from subsequent tokens during the attention computation. This occurs even though the BOS token is often semantically irrelevant to the sequence's content. This phenomenon can redirect subsequent tokens' attention to less informative parts of the input, potentially impairing model performance by disrupting the efficient utilization of contextual information.

For companies utilizing precise and context-sensitive AI models, particularly for text analysis and generation, understanding the Attention Sink is crucial. This phenomenon can impair the model's ability to extract relevant information from long texts or generate coherent responses. Awareness allows for adaptation of model architectures and training strategies to optimize the efficiency and accuracy of AI applications.

A company using an LLM for automated summarization of lengthy customer feedback reports might encounter issues with summary conciseness due to the Attention Sink. The model might over-focus on the first token instead of extracting the most critical information from the entire text. Adapted tokenization or architectural changes can counteract this.

Xiao et al. (MIT, 2023) discovered attention sinks and developed StreamingLLM. The insight: only 4 sink tokens + window suffice for stable inference over millions of tokens.

• StreamingLLM PaperPaper
• StreamingLLM GitHubDocumentation