Another way to see it is that we need randomness.
We are guided by the information we process, and the only limiting factor in someone’s ability to control our thoughts is the interjection of randomness. Privacy means increasing entropy, and thus increasing randomness.
The issue is that humans are really bad randomness generators. If our minds are isolated from sources of randomness, they become pretty deterministic. Mentalists temporarily reduce your experienced randomness and manage to get pretty close to deterministic behaviour.
Luckily we more or less constantly source randomness from our environment. This is only true in so far as a process is random both to an individual and to the actor trying to control it.
Now extrapolate that to virtual realities. True randomness is nullified, every variable can be controlled and thus every thought can be made deterministic.
We should advocate for the injection of true randomness into any virtual reality we create to avoid giving anyone deterministic control over those that experience the virtual reality.