Chainer

» see chainer.org as it lived, on the wayback machine

~ the goodbye ~

By the time I was training networks of my own, the argument Chainer started had already been settled, and the thing that settled it was called PyTorch. That is the ordinary way a good idea arrives. Somebody else's name is on it by the time it gets to you.

Chainer was released on 9 June 2015 by Preferred Networks, a Tokyo company whose actual business was selling machine learning to Japanese industry, robot arms and factory inspection and the like. The framework was pure Python sitting on NumPy, MIT licensed from the start, written by a small team that was not trying to win anything. It had one architectural decision in it and everything else followed from that decision.

In 2015 the frameworks you could actually get work done in were Theano and Caffe, with TensorFlow arriving that November. All of them asked you to describe your network first and hand the description over to be compiled, then push data through whatever came out. Chainer refused to do that, and for about eighteen months it was the only one that refused.

~ the chronicle ~

~ define-by-run ~

The name for the old way is define-and-run. You write out the shape of the computation, the library turns it into a graph, and only after that does any data touch it. The graph is fixed before the first example arrives. Chainer's way was define-by-run: the graph gets built as a side effect of your Python actually executing, one operation at a time, and it is thrown away and rebuilt on the next forward pass.

That sounds like a small implementation detail and it is not. It means a network can change its own structure based on the input it just saw, which is the normal case for anything that reads a sentence of unknown length or walks a tree. It also means your debugger works. When something breaks in a define-by-run framework, the stack trace points at the line you typed, not at a compiled object several layers below it. Anyone who spent 2014 trying to find a bug inside a compiled Theano graph understood the appeal immediately.

~ the fifteen minutes ~

Preferred Networks was not only writing a library, it was running one of the larger private GPU clusters outside the US. In November 2017 the ChainerMN team trained ResNet-50 on ImageNet for 90 epochs in 15 minutes using 1024 Tesla P100s, which cut Facebook's published record by a factor of four. For a moment a Japanese startup held the number that everyone in the field quoted.

CuPy came out of the same work and is the part that survived intact. It is a NumPy-compatible array library that puts everything on the GPU, it was split out into its own project, and it is still maintained and still in wide use by people who have never heard of the framework it was written for.

~ how it ended ~

On 5 December 2019 Preferred Networks shipped Chainer v7 and said in the same breath that v7 would be the last major release and that the company was moving its own research platform onto PyTorch, where it would work as a contributor rather than a competitor. The final maintenance release, 7.8.1, came on 5 January 2022 and nothing has happened since.

The reason given was not that Chainer was worse. It was that PyTorch had taken the define-by-run model, put a large American engineering organization behind it, and become the thing every paper's code was written in. A framework is only as useful as the ecosystem that already speaks it, and by 2019 nobody was publishing Chainer code. Preferred Networks made the correct call about its own engineers' time and killed the better-known half of its reputation to do it.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »