František Bartoš
PhD Candidate | Psychological Methods | UvA Amsterdam | interested in statistics, meta-analysis, and publication bias | once flipped a coin too many times
- Come to Amsterdam or join online for the full week of JASP workshops (24th-28th of August)! If you can't do the full week or you are only interested in meta-analysis, I will be giving the Meta-Analysis workshop on 25th of August. jasp-stats.org/2026/02/05/h...
- Reposted by František BartošDoes it make sense to preregister simulation studies? This question has sparked a lot of debate. ▶️We* work through the why, when, and how ▶️We discuss different phases of methodological research to clarify where preregistration might (or might not) add value 📝 Preprint: doi.org/10.31234/osf...
- We just posted a preprint with a comprehensive meta-meta-analysis of the effects of AI/LLMs on learning. TLDR: - 1,840 effect sizes - extreme between-study heterogeneity - extreme publication bias - small average effects (three times lower than usually reported) (osf.io/preprints/ps...)
- Reposted by František Bartoš
[Not loaded yet] - Reposted by František Bartoš[Not loaded yet]
- "we did not find any evidence for publication bias (p=0.077)"
- The Robust Bayesian Meta-Analysis package got updated with additional vignettes explaining how to perform Bayesian model-averaged publication bias-adjusted - multilevel meta-analysis (cran.r-project.org/web/packages...) - multilevel meta-regression (cran.r-project.org/web/packages...)
- As such, it provides an easy-to-apply state-of-the-art Bayesian meta-analytic methodology for most meta-analytic settings! See an overview of the current functionality with a brief description of all vignettes fbartos.github.io/RoBMA/articl...
- Reposted by František Bartoš[Not loaded yet]
- Reposted by František Bartoš[Not loaded yet]
- Reposted by František Bartoš[Not loaded yet]
- We just preprinted a huge meta-meta-analysis examining the effects of exercise on cognition, memory, and executive function In short - 2239 effect sizes - extreme between-study heterogeneity - extensive publication bias - some subgroup/exercise-specific effects More below (doi.org/10.31234/osf...)
- Previous meta-meta-analysis (doi.org/10.1136/bjsp...) indicated consistent benefits of exercise for cognitive benefits across all domains and populations. However, it synthesized meta-analytic estimates and, as such, it could not adjust for publication nor evaluate heterogeneity.
- Reposted by František Bartoš[Not loaded yet]
- Reposted by František Bartoš[Not loaded yet]
- Reposted by František Bartoš[Not loaded yet]
- Reposted by František Bartoš[Not loaded yet]
- Reposted by František BartošWe built the openESM database: ▶️60 openly available experience sampling datasets (16K+ participants, 740K+ obs.) in one place ▶️Harmonized (meta-)data, fully open-source software ▶️Filter & search all data, simply download via R/Python Find out more: 🌐 openesmdata.org 📝 doi.org/10.31234/osf...
- Simulation studies have a conflict of interest problem. The same team: - develops a new method - designs a simulation study to evaluate it However, the new method has to show good performance to get published. We propose living synthetic benchmarks to address the issue (doi.org/10.48550/arX...).
- We want to separate those two steps. New simulations should be published without new methods. Instead, they should evaluate all existing methods. New methods should be published without new simulations. Instead, they should be assessed on all existing simulations.
- Reposted by František Bartoš[Not loaded yet]
- Reposted by František Bartoš[Not loaded yet]
- Reposted by František BartošSimonsohn has now posted a blog response to our recent paper about the poor statistical properties of the P curve. @clintin.bsky.social and I are finishing up a less-technical paper that will serve as a response. But I wanted to address a meta-issue *around* this that may clarify some things. 1/x
- Would p-curve work if you dropped a piano on it? datacolada.org/129
- Reposted by František Bartoš[Not loaded yet]
- Can anyone point me to the simulation studies showing that p-curve performs well under realistic conditions? And any done by someone else than pcurve authors? As far as I know, p-curve fails horrendously as long as any heterogeneity is involved... doi.org/10.1177/1745... doi.org/10.1002/jrsm...
- We released two preprints describing the JASP Meta-Analytic functionality in detail. Meta-Analysis with JASP, Part I: Classical Approaches (arxiv.org/abs/2509.09845) Meta-Analysis with JASP, Part II: Bayesian Approaches (arxiv.org/abs/2509.09850)
- Z-curve plot is a new visual model fit diagnostic for #metaanalysis with an emphasis on #publicationbias. In contrast to funnel plots, z-curve plots - visualize the distribution of z-statistics (where bias usually occurs) - compare the fit of multiple models simultaneously
