📝 DropPos: Pre-Training Vision Transformers by Reconstructing Dropped Positions 🔭

"Learns to classify the actual position for each non-overlapping patch among all possible positions solely based on their visual appearance, by minimizing the negative log-likelihood." [gal30b+] 🤖 #CV

⚙️ https://github.com/Haochen-Wang409/DropPos
🔗 https://arxiv.org/abs/2309.03576v1 #arxiv

https://creative.ai/system/media_attachments/files/111/032/522/565/913/890/original/11c6a1cb9ef661b0.jpg

https://creative.ai/system/media_attachments/files/111/032/522/624/635/968/original/04b1a8a529d17e50.jpg

https://creative.ai/system/media_attachments/files/111/032/522/679/132/611/original/d9c5124764e444f6.jpg