Theano

» see deeplearning.net as it lived, on the wayback machine

~ the eulogy ~

The first framework I installed for this kind of work was not Theano. I came into computer vision on the tooling of the late 2010s, when the answer to every question was TensorFlow and then very quickly PyTorch. Theano is the generation before that one, and most of how the later libraries think was worked out in it first.

It was written from 2007 onward at the LISA lab at the Université de Montréal, Yoshua Bengio's group, and named after a Greek mathematician. The pitch was narrow and it was enormous: you write an expression in Python over arrays, Theano turns it into a graph, differentiates the graph for you, and compiles it into C or CUDA. In 2007 nobody else handed you automatic differentiation and a GPU in the same import statement.

For about eight years that was where academic deep learning lived. If you read a paper from that window and went looking for the code, the code was Theano.

~ the dossier ~

~ the graph ~

The design choice that mattered was that you did not run your math, you described it. Theano built a symbolic graph of the whole computation, optimized it, and only then compiled something that could execute. That is why it could differentiate anything you wrote without you doing the calculus, and why it could move the work onto a graphics card without you writing a line of CUDA.

The cost was that debugging happened at one remove. Your error came out of a compiled graph rather than the line you typed, and the compile step itself could take longer than the training run on a small model. Everyone who used it complained about this, and everyone kept using it, because the alternative was writing the gradients by hand.

~ what got built on it ~

Keras began life as a friendly front end for Theano, before it grew a TensorFlow backend and then became part of TensorFlow itself. Lasagne and Blocks were the same instinct with different taste. PyMC, the Bayesian modeling library, used Theano as its engine for years and had no interest at all in neural networks, which says something about how general the thing underneath was.

Then Google released TensorFlow in November 2015 with an engineering organization attached to it, and PyTorch arrived in early 2017 with a nicer way to write the same programs. A university lab cannot answer that. The Montreal message said so plainly: supporting Theano was no longer the best way to help new research ideas exist.

~ the fork that kept it alive ~

Version 1.0.0 shipped on 15 November 2017, seven weeks after the announcement, followed by a year of minimal maintenance. The repository stayed up and the mailing list went quiet. What is unusual is that the code did not stop there. The PyMC maintainers, who needed the engine for reasons that had nothing to do with the deep learning race, took over the work, forked it as Aesara in January 2021, then forked again as PyTensor in November 2022.

So the tensor library that lost the framework war is still compiling graphs today under a third name, for statisticians. The rest of the inheritance is less visible and much larger. Every time a modern framework traces your function into a graph and compiles it for a device, that is the Theano idea, running under people who never installed it.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »