Home
Blog
Light
Dark
Automatic
Posts
Swoosh: Rethinking Activation Functions
Introducing the new Swoosh activation function. Perfect test set generalization guaranteed.
Apr 1, 2023
4 min read
Einsum is easy and useful
einsum
is one of the most useful functions in Numpy/Pytorch/Tensorflow and yet many people don’t use it. It seems to have a reputation as being difficult to understand and use, which is completely backwards in my view: the reason
einsum
is great is precisely because it is
easier
to use and reason about than the alternatives. So this post tries to set the record straight and show how simple
einsum
really is.
Nov 5, 2022
8 min read
Distributions Part II: What can we do with distributions?
As promised in part I, we can do a lot of the same things with Schwartz distributions as with classical functions. To see how, we’ll cover derivatives, convolutions, and Fourier transforms of distributions.
Mar 6, 2022
7 min read
Visualizing random vs grid search
Random search usually works better than grid search for hyperparameter optimization. This brief post suggests a way to visualize the reason for this geometrically.
Dec 22, 2021
2 min read
Extensions of Karger's algorithm
If you prefer videos, check out our ICCV presentation, which covers similar content as this blog post. For more details, see our paper. Karger’s contraction algorithm is a fast and very famous method for finding global minimum graph cuts.
Sep 10, 2021
8 min read
Distributions Part I: the Delta distribution
Did you always want to know kind of object this weird Dirac delta “function” actually is? Well, it’s a Schwartz distribution. If that doesn’t help much, then keep reading.
Jul 6, 2021
9 min read
Scripting for personal productivity
If you can program, you can use that to support your habits and automate some routines. This post gives a few examples.
Apr 14, 2021
5 min read
Troubles with the Bias-Variance tradeoff
The bias-variance tradeoff is a key idea in machine learning. But I’ll argue that we know surprisingly little about it: when does it hold? How does it relate to the Double Descent phenomenon? And what do we even formally mean when we talk about it?
Apr 7, 2021
13 min read
Collection of quick computer tips
Many of us spend a lot of time working with our computer, so it’s worth spending some time to make that experience as pleasent and productive as possible. This is a collection of tips that are relatively quick to implement and still very valuable in the long run in my opinion. Mainly geared towards developers and others who work with the shell a lot.
Mar 31, 2021
7 min read
State formally, reason informally
There’s a style of teaching mathematics that I really like: stating definitions and theorems as formally as in any textbook, but focusing on informal arguments for why they should be true.
Mar 24, 2021
4 min read
»
Cite
×