It’s actually super inspiring to see how different linear attention models compare as you’ll form a clearer picture of how authors make their design choices.