Not necessarily and for a few reasons.

I'm more pro-AI than anti. AI running locally is better than running in the cloud. I would hope more local controls could be had over time.

There's inevitably a worry about any chip on the phone. But this is no different on any other phone.

And, to that end, within five years, all devices will likely have an AI chip. One could refuse to upgrade but they they face the security risk of using outdated tech with no update support.