I like to brush them off as I am harvesting. If a specimen is really dirty, best to just leave it. If you are trying to seed another forest you can cut off the real dirty part of the base and toss them whey you have more hardwood trees. Sounds like you found a good spot. You can now start checking that spot any time between June to September.