Oh weird. My font of choice, Berkeley Mono, is causing really wild lag in emacs, especially when using vterm. Hmmm
I have indeed narrowed it down to specifically being with Berkeley Mono. That's... disappointing. And a bit baffling. Guess I'll swap to something else for a little while despite having paid for this font.