The Generalisation Ability of Neural Networks

Neural Networks (NN) can be trained to perform tasks such as image and handwriting recognition, credit card application approval and the prediction of stock market trends. During the learning process, the outputs of a supervised NN come to approximate the target values given the inputs in the training set. This ability may be good in itself, but often the more important purpose for a NN is to generalise i.e. to have the outputs of the NN approximate target values given inputs that are not in the training set. This project examines the impact a selection of key features has on the generalisation ability of NNs. This is achieved through a critical analysis of the following aspects; inputs to the network, selection of training data, size of training data, prior knowledge and the smoothness of the function. Techniques devised to measure the effects these factors have on generalisation are implemented. The results of testing are discussed in detail and are used to form the basis of further work, directed at continuing to refine the processes involved during the training and testing of NNs.