Oddbean new post about | logout
 i use badgerdb to store the actual events as values in keys in my database, but they are rarely over 500k... bitcoin blocks are 2-4mb typically, honestly, they should not be stored in the DB, the differential is the age old stacking problem of slivers and chunks that used to be a big problem with networks until about 10 years ago

if i was going to write a database driver for btcd i'd use badger for the indexes, and store the blocks in a flat filesystem named by the block hash... they are too big

or, it might work with badger, because badger actually stacks the values in one file and the keys in another...

anyway, my point here is that the entire filesystem of bitcoin's leveldb mutates so much you can't really snapshot it properly, they randomly change half the dataset for whatever reason and when you use `cp` with `-rfvpu` which retains the perms and only copies files that have been changed... it still copies all of the files because all of them have been changed so, yeah

it's dumb because bitcoin values are mostly just the blocks, and the indexes are mostly just keys, so having them separated would actually make backup time-effective instead of a collossal pain in the ass 
 lol, the events are stored in values, the indexes in keys

and yes, this would instantly solve the problem of bitcoin

if i had any time spare i'd make a PR to create a btcd database driver with badger because it would probably fix most of its slow IBD