I've been shocked that a theory-driven method yields practical results this good, especially on attention approximation. I proposed my best new optimizer design originally as a dumb baseline; the fact that you can get these efficiency gains with a principled approach makes me a lil insecure.