Do transformers need three projections? Systematic study of QKV variants

This paper explores the necessity of three projections in the query, key, and value (QKV) attention formulation of transformers, proposing variants with shared projections and evaluating their performance on various tasks.