Learning Activation Functions to Improve Deep Neural Networks
By: Researcher
Disclaimer: The provided code links for this paper are external links. Science Nest has no responsibility for the accuracy, legality or content of these links. Also, by downloading this code(s), you agree to comply with the terms of use as set out by the author(s) of the code(s).
Please contact us in case of a broken link from here
| Authors | Matthew Hoffman, Pierre Baldi, Forest Agostinelli, Peter Sadowski |
| Journal/Conference Name | 3rd International Conference on Learning Representations, ICLR 2015 - Workshop Track Proceedings |
| Paper Category | Artificial Intelligence |
| Paper Abstract | Artificial neural networks typically have a fixed, non-linear activation function at each neuron. We have designed a novel form of piecewise linear activation function that is learned independently for each neuron using gradient descent. With this adaptive activation function, we are able to improve upon deep neural network architectures composed of static rectified linear units, achieving state-of-the-art performance on CIFAR-10 (7.51%), CIFAR-100 (30.83%), and a benchmark from high-energy physics involving Higgs boson decay modes. |
| Date of publication | 2014 |
| Code Programming Language | Multiple |
| Comment |
Copyright Researcher 2022