David Holzmüller
Postdoc in machine learning with Francis Bach &
@GaelVaroquaux: neural networks, tabular data, uncertainty, active learning, atomistic ML, learning theory.
https://dholzmueller.github.io
- Reposted by David Holzmüller[Not loaded yet]
- Reposted by David Holzmüller[Not loaded yet]
- Reposted by David Holzmüller[Not loaded yet]
- Reposted by David HolzmüllerStill using temperature scaling? With @dholzmueller.bsky.social, Michael I. Jordan and @bachfrancis.bsky.social we argue that with well designed regularization, more expressive models like matrix scaling can outperform simpler ones across calibration set sizes, data dimensions, and applications.
- Reposted by David Holzmüller[Not loaded yet]
- Reposted by David Holzmüller[Not loaded yet]
- I got 3rd out of 691 in a tabular kaggle competition – with only neural networks! 🥉 My solution is short (48 LOC) and relatively general-purpose – I used skrub to preprocess string and date columns, and pytabkit to create an ensemble of RealMLP and TabM models. Link below👇
- Excited to have co-contributed the SquashingScaler, which implements the robust numerical preprocessing from RealMLP!
- Reposted by David Holzmüller[Not loaded yet]
- Reposted by David Holzmüller[Not loaded yet]
- Reposted by David HolzmüllerMissed the school? We have uploaded recordings of most talks to our YouTube Channel www.youtube.com/@AutoML_org 🙌
- Reposted by David Holzmüller[Not loaded yet]
- 🚨ICLR poster in 1.5 hours, presented by @danielmusekamp.bsky.social : Can active learning help to generate better datasets for neural PDE solvers? We introduce a new benchmark to find out! Featuring 6 PDEs, 6 AL methods, 3 architectures and many ablations - transferability, speed, etc.!
- Reposted by David HolzmüllerThe Skrub TableReport is a lightweight tool that allows to get a rich overview of a table quickly and easily. ✅ Filter columns 🔎 Look at each column's distribution 📊 Get a high level view of the distributions through stats and plots, including correlated columns 🌐 Export the report as html
- Reposted by David Holzmüller[Not loaded yet]
- Reposted by David Holzmüller[Not loaded yet]
- Reposted by David Holzmüller[Not loaded yet]
- Reposted by David HolzmüllerTrying something new: A 🧵 on a topic I find many students struggle with: "why do their 📊 look more professional than my 📊?" It's *lots* of tiny decisions that aren't the defaults in many libraries, so let's break down 1 simple graph by @jburnmurdoch.bsky.social 🔗 www.ft.com/content/73a1...
- Reposted by David Holzmüller[Not loaded yet]
- Practitioners are often sceptical of academic tabular benchmarks, so I am elated to see that our RealMLP model outperformed boosted trees in two 2nd place Kaggle solutions, for a $10,000 forecasting challenge and a research competition on survival analysis.
- A new tabular classification benchmark provides another independent evaluation of our RealMLP. RealMLP is the best classical DL model, although some other recent baselines are missing. TabPFN is better on small datasets and boosted trees on larger datasets, though.
- Reposted by David HolzmüllerLearning rate schedules seem mysterious? Why is the loss going down so fast during cooldown? Turns out that this behaviour can be described with a bound from *convex, nonsmooth* optimization. A short thread on our latest paper 🚞 arxiv.org/abs/2501.18965
- Reposted by David HolzmüllerEarly stopping on validation loss? This leads to suboptimal calibration and refinement errors—but you can do better! With @dholzmueller.bsky.social, Michael I. Jordan, and @bachfrancis.bsky.social, we propose a method that integrates with any model and boosts classification performance across tasks.
- The first independent evaluation of our RealMLP is here! On a recent 300-dataset benchmark with many baselines, RealMLP takes a shared first place overall. 🔥 Importantly, RealMLP is also relatively CPU-friendly, unlike other SOTA DL models (including TabPFNv2 and TabM). 🧵 1/
- Reposted by David Holzmüller[Not loaded yet]
- Reposted by David HolzmüllerMy book is (at last) out, just in time for Christmas! A blog post to celebrate and present it: francisbach.com/my-book-is-o...
- I'll present our paper in the afternoon poster session at 4:30pm - 7:30 pm in East Exhibit Hall A-C, poster 3304!
- We wrote a benchmark paper with many practical insights on (the benefits of) active learning for training neural PDE solvers. 🚀 I was happy to be a co-advisor on this project - most of the credit goes to Daniel and Marimuthu.
- Reposted by David Holzmüller[Not loaded yet]
- I'll be at #NeurIPS2024 next week to present this paper (Thu afternoon) as well as a workshop paper on active learning for neural PDE solvers. Let me know if you'd like to chat about tabular data, uncertainty, active learning, etc.!
- Reposted by David Holzmüller[Not loaded yet]
- If you have train+validation data, should you refit on the whole data with the stopping epoch found on the train-validation split? In the quoted paper, we did an experiment including 5-fold ensembles on a 5-fold cross-validation splits (bagging) and with refitting. (short 🧵)
- Reposted by David HolzmüllerI recently shared some of my reflections on how to use probabilistic classifiers for optimal decision-making under uncertainty at @pydataparis.bsky.social 2024. Here is the recording of the presentation: www.youtube.com/watch?v=-gYn...
- One thing I learned from this project is that accuracy is a quite noisy metric. With small validation sets (~1K samples), hyperparameter opt. using AUROC instead of accuracy can yield better accuracy on the test set. We also did some experiments on metrics for early stopping. 🧵
- PyTabKit 1.1 is out! - Includes TabM and provides a scikit-learn interface - some baseline NN parameter names are renamed (removed double-underscores) - other small changes, see the readme. github.com/dholzmueller...
- Reposted by David Holzmüller[Not loaded yet]
- Reposted by David HolzmüllerFor those who missed this post on the-network-that-is-not-to-be-named, I made public my "secrets" for writing a good CVPR paper (or any scientific paper). I've compiled these tips of many years. It's long but hopefully it helps people write better papers. perceiving-systems.blog/en/post/writ...
