Try it with less than 10 relays. I also turned off “use this relay for looking up posts” in the relay settings. It’s pretty fast. 
I do agree though, Damus seems a bit faster. But not by a lot 

 There’s a relay bug when you have more than 10 relays or more than 1000 follows. Maybe why you had some sluggishness 

 @note1htu2eecyyevzf6gsulcuemplvzazxhuc00lp26ted2zeqht5z6qsd4ggvr