Sure they can't hold your funds, or spend them, but they do get to control some portion of your ability to send/receive them. I don't know that it's fair to say that what defines a custodial service who's holding your private key. 

A cloud VPS service is custodial because it handles hardware provisioning and networking for me. 
A DNS provider is custodial because technically it's possibly to host your own name servers
My ISP is custodial because it also does networking and routing for me as a service.