Setting touch delegates wasn't working great because the app sometimes uses though as well.

But then I had a better idea: after ACTION_DOWN, find the touch target and set a detach listener on it. If it gets detached before ACTION_UP, crash immediately.