Global Versus Local Constructive Function Approximation For On-Line Reinforcement Learning
Vamplew, P and Ollington, R (2005) Global Versus Local Constructive Function Approximation For On-Line Reinforcement Learning. Technical Report. Unpublished.
|PDF - Requires a PDF viewer|
In order to scale to problems with large or continuous state-spaces, reinforcement learning algorithms need to be combined with function approximation techniques. The majority of work on function approximation for reinforcement learning has so far focused either on global function approximation with a static structure (such as multi-layer perceptrons), or on constructive architectures using locally responsive units. The former, whilst achieving some notable successes, has also been shown to fail on some relatively simple tasks. The locally constructive approach has been shown to be more stable, but may scale poorly to higher-dimensional inputs, as it will require a dramatic increase in resources. This paper explores the use of two constructive algorithms using non-locally responsive neurons based on the popular Cascade-Correlation supervised-learning algorithm. The algorithms are applied within the sarsa reinforcement learning algorithm, and their performance compared against both a multi-layer perceptron and a locally constructive algorithm (the Resource Allocating Network) across three reinforcement learning tasks. It is shown that the globally constructive algorithms are less stable, but that on some tasks they can achieve similar performance to the locally constructive approach, whilst generating much more compact solutions.
|Item Type:||Report (Technical Report)|
|Additional Information:||This report is an extended version of a paper of the same title presented at AI'05: The 18th Australian Joint Conference on Artificial Intelligence, Sydney, Australia, 5-9 Dec 2005. The proceedings of that conference were published as a volume of the Springer Lecture Notes in Computer Science series, and are available from http://www.springeronline.com/lncs|
|Keywords:||Reinforcement learning, constructive neural networks, function approximation, sarsa|
|Deposited By:||utas eprints|
|Deposited On:||15 Sep 2005|
|Last Modified:||18 Jul 2008 19:39|
|ePrint Statistics:||View statistics for this ePrint|
Repository Staff Only: item control page