as i write the third footnote in the www.nber.org/papers/w33729 revision addressing it, would like to apologize to @albertjmenkveld.com and @flxhlzmstr.bsky.social for suggesting the "standard deviation across researchers divided by mean standard error of each analysis" metric

which doesn't really work as intended in terms of comparing across different contexts or data cleaning choices, since it's sensitive to things like the sample size of the relevant data set or noise in the true model, even for constant levels of researcher variation

probably better to try to think of what meaningful effect size variation is in a given context which is obviously not as satisfyingly objective :-/