That's something English speakers have a hard time relating with because they associate sex and gender. But in most languages, gender and sex a different things. For example, in my native Portuguese, we can say sentences as:

- A baleia é macho (the whale is male)

In this sentence, "A" is female, but "macho" means "male", thus i am using a female noun (baleia, meaning whale) and specifying  it is actually a male animal. The gramatical gender of the word bears no relation to the sex of the animal.

For us, the use of 4th gender for vegetables is a plausible thing. Because gender is a language construct, not related to sex. 

 the more you know