But does it? I can switch pools in seconds. Does it really matter how many pools there are after a certain number? Even with only 2 or 3 pools if one gets too big it's not like it's a huge task to switch pools if one becomes too big.  

 You of course know that though. You are a pleb mining inspiration to me 🥰 

 The mining protocol is called Stratum.

Today, pools decide which transactions go inside the new block, aka the block template. After this, the block template is send  to miners which will try to mine it. All miners inside a pool use the same block template.

So, the block template decision is centralised on a dozen pools. One evil entity just need to apply pressure on a dozen individuals to censor some type of transaction.

With the next version of the protocol, Stratum v2, miners are able to decide which block template they want to work on, without loosing the benefits of being in a pool.

So, the block template decision will be decentralised  into thousands of individual miners, which makes it a lot harder for the evil entity to apply pressure.

https://ocean.xyz will use Stratum v2 in a near future
https://dmnd.work is using Stratum v2 today (I think) 

 I don't understand or know about how pools can integrate stratum v2 but if it is possible I wonder why did ocean not launch with stratum v2 right away?

There are plenty of small pools that sign blocks/create block templates other than the big ones. The big pools are just the ones finding blocks. If they collude, they still don't control the hash and miners can effortlessly point to a small pool like kano pool. Kano pool for example went over a year before it found a block and now it's been several months since the last block they found. Kano is an example of where hash can go if the big pools start colluding. Hash is concentrated to a few large pools but not under centralized control. 

 There are 2 questions here:

1. Why not stratum v2 right away?

I'm not an expert, but maybe miners need to upgrade their firmware? Well, some sort of software upgrade must happen, maybe they are not ready?

2. Why not Kano pool?

Miners seem to prefer FPPS to other payment schemes, which makes sense, due to the steady stream of revenue. Since Kano only finds one block per year, I don't see how it could offer FPPS to its "customers", so in that regard Kano pool will always be behind bigger pools. So, in this particular question, size matters 😉

Again, not an expert, just thinking out loud.

https://originstamp.com/blog/pps-vs-fpps-vs-pplns-vs-pps-mining-pool-payouts/