Same. In this case, it’s the idea of neural networks taking advantage of the sparsity of the embeddings to encode more features than just the dimensionality of the vector space (the set of orthogonal vectors). I probably can’t do it justice in a nostr note after a few whiskeys. 

 Any comment on this will make me sound stupid, so I'll refrain 😅