what are you using for storage? 

 was using a hdd, switched to a 4tb ssd recently. I suspect it was due to a low dbcache (32) value but not sure yet 

 did not see until after that you swapped storage. if you do in fact have an intermittent hardware problem the best way in my experience is to swap parts where you can. since you have the storage out of the way, if you have multiple memory modules I would run one at a time and see how that turns out 

 You may want to keep an eye on uptime to be sure something isn’t hard power cycling the host, and dmesg to be sure the bitcoind process isn’t getting OOM killed. 

I’ve had issues with the later on an RPI4. JSON-RPC calls from Fulcrum or Electrs syncs also seem to be able to use an unbounded amount of memory…

I think ungraceful stops of the bitcoind process at the right time can cause the chainstate DB to come unsynced from the blockchain DB in ways that require reindexing. The really bad throughput of the RPi4’s USB3 ports for block storage may exasperate this 

 This machine is on a UPS and doesn’t ever randomly restart. It’s never unstable. I don’t run it hot. If it can get corrupted from ungraceful stops then thats a bug. 

 Due to interleaving and similar you already mostly achieve that 

 ycruncher? 

 I remember prime95 but maybe that was like 20 years ago 

 damn im old 

 whatever type of error you are having must be a more widespread issue than a few pages to be triggered so often

the ideal program would have a small pool it nonstop allocates to and deallocates from and a pool it very slowly checks and rotates allocations in/out

write random data and CRC it as I said

you then want to stop the process and get a debugger if there’s a mismatch and see the physical location along with identifying the RAM module 

 Prime95 will definitely get a CPU toasty, but you would need to ensure it's running with a mix of large and small FFT and for a while (e.g. hours/days). Small FFT maximizes CPU heat, but isn't the best for finding instabilities working with RAM. Large FFT helps there. In general, I find the overclocking community sometimes works on "vibes" rather than completely proven test methodologies, so take this with a grain of salt. 

 not that you're overclocking, but that group tends to accumulate tribal knowledge of CPU/RAM stability tests. Hardware can be "fun". A ton of variables. Even things like bios versions can cause instabilities.