I had to do that too recently, and I wish I had a way to filter for anyone that I haven't interacted with and just remove those. I mainly do that when I realize Im spending too much time scrolling. I removed just ones that I don't ever remember seeing or responding to.
I also did the following: unfollow all big accounts (like Derek etc) even tho they seem like cool people. Mainly if they say something important that I should know, one the smaller accounts that I interact with will probably mention it. So it's like an information hierarchy or kinda like a smaller working group session.
I don't look at my follower list anymore. I realized that's a dangerous pitfall bc if I notice someone drops off, I then go into this spiral of "what did I say that offended them".
My mantra now is it's their business whom they follow. It's my business whom I follow.