Yeah, that's the only caveat. With gold, keeping it in a bank was safer than in your home vault. Not to mention you couldn't use it for transactions at the same time. With #Bitcoin, you can take self-custody of it and use it to transact with it.
#Bitcoin lives only on the blockchain, which is replicated to thousands of computers worldwide. People take custody of a private key, really.