📝 Vision Transformers Need Registers 🔭

"Adds additional tokens to the input sequence of the vision transformer that fills the role of low informative background areas, thus making the feature maps and attention maps more smooth." [gal30b+] 🤖 #CV

⚙️ https://github.com/facebookresearch/deit
🔗 https://arxiv.org/abs/2309.16588v1 #arxiv

https://creative.ai/system/media_attachments/files/111/159/487/324/163/321/original/71b7e5ea3a9fbb6c.jpg

https://creative.ai/system/media_attachments/files/111/159/487/415/279/611/original/2882cbe22a00ec33.jpg

https://creative.ai/system/media_attachments/files/111/159/487/477/678/088/original/7ce633fe7ac9acec.jpg

https://creative.ai/system/media_attachments/files/111/159/487/537/859/843/original/842774180aaab9fa.jpg