Classification datasets results


Discover the current state of the art in objects classification.

Result Method Venue Details
0.21% Regularization of Neural Networks using DropConnect ICML 2013
0.23% Multi-column Deep Neural Networks for Image Classification CVPR 2012
0.23% APAC: Augmented PAttern Classification with Neural Networks arXiv 2015
0.24% Batch-normalized Maxout Network in Network arXiv 2015 Details

(k=5 maxout pieces in each maxout unit).

0.29% Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree AISTATS 2016 Details

Single model without data augmentation

0.31% Recurrent Convolutional Neural Network for Object Recognition CVPR 2015
0.31% On the Importance of Normalisation Layers in Deep Learning with Piecewise Linear Activation Units arXiv 2015
0.32% Fractional Max-Pooling arXiv 2015 Details

Uses 12 passes at test time. Reaches 0.5% when using a single pass at test time.

0.33% Competitive Multi-scale Convolution arXiv 2015
0.35% Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition Neural Computation 2010 Details

6-layer NN 784-2500-2000-1500-1000-500-10 (on GPU), uses elastic distortions

0.35% C-SVDDNet: An Effective Single-Layer Network for Unsupervised Feature Learning arXiv 2014
0.37% Enhanced Image Classification With a Fast-Learning Shallow Convolutional Neural Network arXiv 2015 Details
0.39% Efficient Learning of Sparse Representations with an Energy-Based Model NIPS 2006 Details

Large conv. net, unsup pretraining, uses elastic distortions

0.39% Convolutional Kernel Networks arXiv 2014 Details
0.39% Deeply-Supervised Nets arXiv 2014
0.4% Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis Document Analysis and Recognition 2003
0.40% Hybrid Orthogonal Projection and Estimation (HOPE): A New Framework to Probe and Learn Neural Networks arXiv 2015
0.42% Multi-Loss Regularized Deep Neural Network CSVT 2015 Details

Based on NiN architecture.

0.45% Maxout Networks ICML 2013 Details

Uses convolution. Does not use dataset augmentation.

0.45% Training Very Deep Networks NIPS 2015 Details

Best result selected on test set. 0.46% average over multiple trained models.

0.45% ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks arXiv 2015
0.46% Deep Convolutional Neural Networks as Generic Feature Extractors IJCNN 2015 Details

feature extraction part of convnet is trained on imagenet (external training data), classification part is trained on cifar-10

0.47% Network in Network ICLR 2014 Details

NIN + Dropout

The code for NIN available at https://github.com/mavenlin/cuda-convnet

0.52 % Trainable COSFIRE filters for keypoint detection and pattern recognition PAMI 2013 Details
0.53% What is the Best Multi-Stage Architecture for Object Recognition? ICCV 2009 Details

Large conv. net, unsup pretraining, no distortions

0.54% Deformation Models for Image Recognition PAMI 2007 Details

K-NN with non-linear deformation (IDM) (Preprocessing: shiftable edges)

0.54% A trainable feature extractor for handwritten digit recognition Journal Pattern Recognition 2007 Details

Trainable feature extractor + SVMs, uses affine distortions

0.56% Training Invariant Support Vector Machines Machine Learning 2002 Details

Virtual SVM, deg-9 poly, 2-pixel jittered (Preprocessing: deskewing)

0.59% Simple Methods for High-Performance Digit Recognition Based on Sparse Coding TNN 2008 Details

Unsupervised sparse features + SVM, no distortions

0.62% Unsupervised learning of invariant feature hierarchies with applications to object recognition CVPR 2007 Details

Large conv. net, unsup features, no distortions

0.62% PCANet: A Simple Deep Learning Baseline for Image Classification? arXiv 2014 Details
0.63% Shape matching and object recognition using shape contexts PAMI 2002 Details

K-NN, shape context matching (preprocessing: shape context feature extraction)

0.64% Beyond Spatial Pyramids: Receptive Field Learning for Pooled Image Features CVPR 2012
0.68% Handwritten Digit Recognition using Convolutional Neural Networks and Gabor Filters ICCI 2003
0.69% On Optimization Methods for Deep Learning ICML 2011
0.71% Deep Fried Convnets ICCV 2015 Details

Uses about 10x fewer parameters than the reference model, which reaches 0.87%.

0.75% Sparse Activity and Sparse Connectivity in Supervised Learning JMLR 2013
0.78% Explaining and Harnessing Adversarial Examples ICLR 2015 Details

permutation invariant network used

0.82% Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations ICML 2009
0.84% Supervised Translation-Invariant Sparse Coding CVPR 2010 Details

Uses sparse coding + svm.

0.94% Large-Margin kNN Classification using a Deep Encoder Network 2009
0.95% Deep Boltzmann Machines AISTATS 2009
1.01% BinaryConnect: Training Deep Neural Networks with binary weights during propagations NIPS 2015 Details
1.1% StrongNet: mostly unsupervised image recognition with strong neurons technical report on ALGLIB website 2014 Details

StrongNet is a neural design which uses two innovations: (a) “strong neurons” – highly nonlinear neurons with multiple outputs and (b) “mostly unsupervised architecture” – backpropagation-free design with all layers except for the last one being trained in a completely unsupervised setting.

1.12% CS81: Learning words with Deep Belief Networks 2008
1.19% Convolutional Neural Networks 2003 Details

The ConvNN is based on the paper “Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis”.

1.2% Reducing the dimensionality of data with neural networks 2006
1.40% Convolutional Clustering for Unsupervised Learning arXiv 2015 Details
1.5% Deep learning via semi-supervised embedding 2008
14.53% Deep Representation Learning with Target Coding AAAI 2015
Something is off, something is missing ? Feel free to fill in the form.

Units: accuracy %

Classify 32x32 colour images.

Result Method Venue Details
96.53% Fractional Max-Pooling arXiv 2015 Details

Uses 100 passes at test time. Reaches 95.5% when using a single pass at test time, and 96.33% when using 12 passes.. Uses data augmentation during training.

95.59% Striving for Simplicity: The All Convolutional Net ICLR 2015 Details

  1. 92% without data augmentation, 92.75% with small data augmentation, 95.59% when using agressive data augmentation and larger network.

94.16% All you need is a good init ICLR 2016 Details

Only mirroring and random shifts, no extreme data augmentation. Uses thin deep residual net with maxout activations.

94% Lessons learned from manually classifying CIFAR-10 unpublished 2011 Details

Rough estimate from a single individual, over 400 training images (~1% of training data).

93.95% Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree AISTATS 2016 Details

Single model with data augmentation, 92.38% without.

93.72% Spatially-sparse convolutional neural networks arXiv 2014
93.63% Scalable Bayesian Optimization Using Deep Neural Networks ICML 2015
93.57% Deep Residual Learning for Image Recognition arXiv 2015 Details

Best performance reached with 110 layers. Using 1202 layers leads to 92.07%, 56 layers lead to 93.03%.

93.45% Fast and Accurate Deep Network Learning by Exponential Linear Units arXiv 2015 Details

Without data augmentation.

93.34% Universum Prescription: Regularization using Unlabeled Data arXiv 2015
93.25% Batch-normalized Maxout Network in Network arXiv 2015 Details

(k=5 maxout pieces in each maxout unit). Reaches 92.15% without data augmentation.

93.13% Competitive Multi-scale Convolution arXiv 2015
92.91% Recurrent Convolutional Neural Network for Object Recognition CVPR 2015 Details

Reaches 91.31% without data augmentation.

92.49% Learning Activation Functions to Improve Deep Neural Networks ICLR 2015 Details

Uses an adaptive piecewise linear activation function. 92.49% accuracy with data augmentation and 90.41% accuracy without data augmentation.

92.45% cifar.torch unpublished 2015 Details

Code available at https://github.com/szagoruyko/cifar.torch

92.40% Training Very Deep Networks NIPS 2015 Details

Best result selected on test set. 92.31% average over multiple trained models.

92.23% Stacked What-Where Auto-encoders arXiv 2015
91.88% Multi-Loss Regularized Deep Neural Network CSVT 2015 Details

With data augmentation, 90.45% without. Based on NiN architecture.

91.78% Deeply-Supervised Nets arXiv 2014 Details

Single model, with data augmentation: 91.78%. Without data augmentation: 90.22%.

91.73% BinaryConnect: Training Deep Neural Networks with binary weights during propagations NIPS 2015 Details

These results were obtained without using any data-augmentation.

91.48% On the Importance of Normalisation Layers in Deep Learning with Piecewise Linear Activation Units arXiv 2015
91.40% Spectral Representations for Convolutional Neural Networks NIPS 2015
91.2% Network In Network ICLR 2014 Details

The code for NIN available at https://github.com/mavenlin/cuda-convnet

NIN + Dropout 89.6% NIN + Dropout + Data Augmentation 91.2%

91.19% Speeding up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves IJCAI 2015 Details

Based on the “all convolutional” architecture. which reaches 90.92% by itself.

90.78% Deep Networks with Internal Selective Attention through Feedback Connections NIPS 2014 Details
90.68% Regularization of Neural Networks using DropConnect ICML 2013
90.65% Maxout Networks ICML 2013 Details

This result was obtained using both convolution and synthetic translations / horizontal reflections of the training data.

Reaches 88.32% when using convolution, but without any synthetic transformations of the training data.

90.61% Improving Deep Neural Networks with Probabilistic Maxout Units ICLR 2014 Details

  1. 65% without data augmentation.
  2. 61% when using data augmentation.

90.5% Practical Bayesian Optimization of Machine Learning Algorithms NIPS 2012 Details

Reaches 85.02% without data augmentation.

With data augmented with horizontal reflections and translations, 90.5% accuracy on test set is achieved.

89.67% APAC: Augmented PAttern Classification with Neural Networks arXiv 2015
89.14% Deep Convolutional Neural Networks as Generic Feature Extractors IJCNN 2015 Details

feature extraction part of convnet is trained on imagenet (external training data), classification part is trained on cifar-10

89% ImageNet Classification with Deep Convolutional Neural Networks NIPS 2012 Details

87% error on the unaugmented data.

88.80% Empirical Evaluation of Rectified Activations in Convolution Network ICML workshop 2015 Details

Using Randomized Leaky ReLU

88.79% Multi-Column Deep Neural Networks for Image Classification CVPR 2012 Details
87.65% ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks arXiv 2015
86.70 % An Analysis of Unsupervised Pre-training in Light of Recent Advances ICLR 2015 Details

Unsupervised pre-training, with supervised fine-tuning. Uses dropout and data-augmentation.

84.87% Stochastic Pooling for Regularization of Deep Convolutional Neural Networks arXiv 2013
84.4% Improving neural networks by preventing co-adaptation of feature detectors arXiv 2012 Details

So called “dropout” method.

83.96% Discriminative Learning of Sum-Product Networks NIPS 2012
82.9% Stable and Efficient Representation Learning with Nonnegativity Constraints ICML 2014 Details

Full data, 3-layers + multi-dict.

  1. 4 with 3-layers only.
  2. 0 with 1-layers only.
82.2% Learning Invariant Representations with Local Transformations ICML 2012 Details
82.18% Convolutional Kernel Networks arXiv 2014 Details
82% Discriminative Unsupervised Feature Learning with Convolutional Neural Networks NIPS 2014 Details

Unsupervised feature learning + linear SVM

80.02% Learning Smooth Pooling Regions for Visual Recognition BMVC 2013
80% Object Recognition with Hierarchical Kernel Descriptors CVPR 2011
79.7% Learning with Recursive Perceptual Representations NIPS 2012 Details
79.6 % An Analysis of Single-Layer Networks in Unsupervised Feature Learning AISTATS 2011 Details

  1. 6% obtained using K-means over whitened patches, with triangle encoding and 4000 features (clusters).

78.67% PCANet: A Simple Deep Learning Baseline for Image Classification? arXiv 2014 Details

No data augmentation. Multiple feature scales combined. 77.14% when using only a single scale.

75.86% Enhanced Image Classification With a Fast-Learning Shallow Convolutional Neural Network arXiv 2015 Details
Something is off, something is missing ? Feel free to fill in the form.

Units: error %

The Street View House Numbers (SVHN) Dataset.

SVHN is a real-world image dataset for developing machine learning and object recognition algorithms with minimal requirement on data preprocessing and formatting. It can be seen as similar in flavor to MNIST(e.g., the images are of small cropped digits), but incorporates an order of magnitude more labeled data (over 600,000 digit images) and comes from a significantly harder, unsolved, real world problem (recognizing digits and numbers in natural scene images). SVHN is obtained from house numbers in Google Street View images.

Something is off, something is missing ? Feel free to fill in the form.

Last updated on 2016-02-22.

© 2013-2016 Rodrigo Benenson.

Built using middleman and bootstrap.