A faster learning neural network classifier using selective backpropagation

Craven, Michael P.

Abstract

The problem of saturation in neural network classification problems is discussed. The listprop algorithm is presented which reduces saturation and dramatically increases the rate of convergence.

The technique uses selective application of the backpropagation algorithm, such that training is only carried out for patterns which have not yet been learnt to a desired output activation tolerance. Furthermore, in the output layer, training is only carried out for weights connected to those output neurons in the output vector which are still in error, which further reduces neuron saturation and learning time.

Results are presented for a 196-100-46 Multi-Layer Perceptron (MLP) neural network used for text-to-speech conversion, which show that convergence is achieved for up to 99.7% of the training set compared to at best 94.8% for standard backpropagation. Convergence is achieved in 38% of the time taken by the standard algorithm.

Citation

Craven, M. P. (1997). A faster learning neural network classifier using selective backpropagation.

• Conference Name: Fourth IEEE International Conference on Electronics, Circuits and Systems
• End Date: Dec 18, 1997
• Publication Date: Jan 1, 1997
• Deposit Date: Feb 20, 2013
• Publicly Available Date: Feb 20, 2013
• Peer Reviewed: Peer Reviewed
• Keywords: neural networks, ANN, satuation, convergence, backpropagation, backprop, text-to-speech, classification, classifier
• Public URL: https://nottingham-repository.worktribe.com/output/1024324
• Additional Information: © 1997 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.