there is also the possibility that you load the key from a remote point and the main question is just about how strong security is... there's memory access detection fences that can be used that can trigger a wipe-down if any memory near the secrets are tampered with, and that it never copies this data anywhere, always using it from in situ (aside from putting it into a register

yeah, there is exploits for some of the registers too... AVX registers famously with their "marked empty" heartbleed... but most kernels have mitigations now that frequently clear those registers between processes and whatnot