In the US, the First Amendment phrase "freedom of speech" is prefaced by "Congress shall make no law ... abridging the" meaning it was not seen as a positive right that others owe you but rather a negative right against government interference.