It's because they have your number as an iMessage contact.
👀 I'm switching from iphone soon, will there be this problem somwhow to be expected?
Turn off iMessage before you switch. That will probably help.