What I do is add all crates I know I'll want upfront to Cargo.toml by editing it, and then add the rest as I go with cargo add.
That makes sense. There's definitely a few I seem to always include. But pretty soon I've added a dozen more. I'm like "how do I add one month to a date" and boom, five more crates added (ok I'm exaggerating slightly).