If they come back, they are usually following me and start replying to my notes, then I follow them again. 

 Yeah that seems like a reasonable way to do it