I did this with Arch, and ever since, I haven't been able to let go of Arch (many years) - It's, essentially, my Arch. 
What you get is a super lean, lightweight but flexible systems with only the stuff you need to get it to work, and it's amazing. 
Sure, it's grown a bit as time goes, but at least it's easier to keep track of what's there and once in a while I'll just go in and take out anything that doesn't belong anymore. 

Running Gnome though, as I try to not multi-task so much.  

 Yeah, so far I'm very happy with the results. No apps that I don't want installed are on there, which is pretty great. 

 its gets even more interesting when you compile a kernel for your machine specifically, select your userspace tools yourself and chose if you use a service manager or not, and which one.