Fulcrum DB got corrupted, AGAIN. fml 💀🤬🪦
It's a pain to sync a fulcrum db
I have a secret: I always sync on my desktop and then copy it to my node. Not only is it much faster, I also always have a backup on my desktop up to the last time I did this, so I never have to do it from scratch.
Nice. I'll start doing it next time I need
Do you know the cause? I got a UPS for my dojo but still haven't gone back to Fulcrum.
For me it was power loss and one time the SSD was overheating
I honestly don't know. This happened by just rebooting... My theory is that when you reboot, it kills all processes "nicely" with SIGTERM but if they don't stop after some time, it kills them forcefully with SIGKILL. I think Fulcrum can take a while to shutdown if it gets killed while writing to the DB, so when it gets SIGKILLed later, DB gets corrupted.
Do you have timeoutstopseconds set? I guess I should ask if you're using a systemd unit first 😁 It should wait for it on a reboot if you have the settings set and it's running in the system.slice
Use the latest version. The problem is not Fulcrum, but the underlying rocksdb, which was very shaky <v8. Now it’s really stable, I love Fulcrum. It’s great software. However, I regularly make rsync backups of the database directory, just in case. But I do that with the Bitcoin blockchain as well because I don’t want to wait weeks for it to resync/reindex.