Google had accepted a version of Conversations with address book integration that explicitly asks the user for consent to process the contact list locally on their device before promoting for contacts permission. Our privacy policy included explicit wording w.r.t. local processing of the contact list.

However it is unthinkable for Google that someone would request contacts permission and then *not* upload them. A few days later they changed their mind and threatened to remove the app again. 

 I tried arguing with Google for a while however their 7 day deadline to admit publicly that I'm uploading the contact list is running out tomorrow so to not risk the app being removed again I was forced to publish a version without address book integration.

It seems like you can only have an app that requests contacts permission on the PlayStore if you boldly declare that you upload them.