And this is usually why some form of government, preferably very limited in scope is required.
Every one has a right to move freely.
Every also has a right to property.
The US has often ignored the right to movement. Property owners can prohibit others from their property entirely. And driving is treated a "privilege" that can be revoked.
Other nations try to strike a balance. Where one can traverse a property but not damage it or cause a permanent change.
Rights WILL conflict with another. Some rights take priority over other rights. Some rights only exist as a consequence of other rights (for example, life is the ultimate right. Without it no other right can exist).
The proper role of government is to protect one's rights. Part of this is solving the conflicts that occur when rights conflict with one another.