Exactly. 

Don't think about the IP as an info that is sent because of nostr. Think about the IP as a house address.

You have a house address, and the nostr server (relay) has a house address. The relay needs to know your house address to deliver to you the data/notes you want to see.

So first you know the relays house address, you send them that "hey relay, I want to see all notes containing #funnycatvideos", then relay searches, and send to your house address, the notes containing #funnycatvideos. But in this process, each time you ask something from the relay, the relay gets your house address, so it can respond to you. So they can log it. It does not mean they do it, but they CAN.