Posts

Introducing the new Swoosh activation function. Perfect test set generalization guaranteed.

einsum is one of the most useful functions in Numpy/Pytorch/Tensorflow and yet many people don’t use it. It seems to have a reputation as being difficult to understand and use, which is completely backwards in my view: the reason einsum is great is precisely because it is easier to use and reason about than the alternatives. So this post tries to set the record straight and show how simple einsum really is.

As promised in part I, we can do a lot of the same things with Schwartz distributions as with classical functions. To see how, we’ll cover derivatives, convolutions, and Fourier transforms of distributions.

Random search usually works better than grid search for hyperparameter optimization. This brief post suggests a way to visualize the reason for this geometrically.

If you prefer videos, check out our ICCV presentation, which covers similar content as this blog post. For more details, see our paper. Karger’s contraction algorithm is a fast and very famous method for finding global minimum graph cuts.

Did you always want to know kind of object this weird Dirac delta “function” actually is? Well, it’s a Schwartz distribution. If that doesn’t help much, then keep reading.

If you can program, you can use that to support your habits and automate some routines. This post gives a few examples.

The bias-variance tradeoff is a key idea in machine learning. But I’ll argue that we know surprisingly little about it: when does it hold? How does it relate to the Double Descent phenomenon? And what do we even formally mean when we talk about it?

Many of us spend a lot of time working with our computer, so it’s worth spending some time to make that experience as pleasent and productive as possible. This is a collection of tips that are relatively quick to implement and still very valuable in the long run in my opinion. Mainly geared towards developers and others who work with the shell a lot.

There’s a style of teaching mathematics that I really like: stating definitions and theorems as formally as in any textbook, but focusing on informal arguments for why they should be true.