Provided by Cognitive Sciences ePrint Archive
Evolving robust and specialized car racing skills
Julian Togelius
Department of Computer Science
University of Essex, UK
[email protected]
Abstract— Neural network-based controllers are evolved for
racing simulated R/C cars around several tracks of varying
difficulty. The transferability of driving skills acquired when
evolving for a single track is evaluated, and different ways of
evolving controllers able to perform well on many different
tracks are investigated. It is further shown that such generally
proficient controllers can reliably be developed into specialized
controllers for individual tracks. Evolution of sensor parameters
together with network weights is shown to lead to higher final
fitness, but only if turned on after a general controller is
developed, otherwise it hinders evolution. It is argued that
simulated car racing is a scalable and relevant testbed for
evolutionary robotics research, and that the results of this
research can be useful for commercial computer games.
Keywords: Evolutionary robotics, games, car racing,
driving, incremental evolution
I. Introduction
Car racing is a remarkably popular preoccupation - both to
watch and to participate in - be it in a computer simulation
or in the “real world”. But it is not only popular, it is
also challenging: racing well requires fast and accurate
reactions, knowledge of the car’s behaviour in different
environments, and various forms of real-time planning, such
as path planning and deciding when to overtake a competitor.
In other words, it requires many of the core components
of intelligence being researched within computational in-
telligence and robotics. The success of the recent DARPA
Grand Challenge[1], where completely autonomous real cars
raced in a demanding desert environment, may be taken as
a measure of the interest in car racing within these research
communities.
This paper deals with using evolutionary algorithms to
create neural network controllers for simulated car racing.
Specifically, we evolve controllers that have robust perfor-
mance over different tracks, and can be specialized to work
better on particular tracks.
Evolutionary robotics (the use of evolutionary algorithms
for embodied control problems) and simulated car racing
are in many ways ideal companions. The benefit for the
development of racing games and simulations is clear: evo-
lutionary robotics offers a way to automatically develop
controllers, possibly specialized for specific tracks or types
of tracks, driving styles, skill levels, competitors etc. One
could envision a racing simulator where the user is allowed to
construct his own tracks and cars, and the game automatically
develops a set of controllers to drive these tracks. The game
could also automatically adapt to the user’s driving style,
Simon M. Lucas
Department of Computer Science
University of Essex, UK
[email protected]
or learn from other drivers (humans or machines) on the
Internet.
The benefits for evolutionary robotics might require some
explanation. While evolutionary robotics has successfully
been used for various interdisciplinary investigations (e.g. of
memory mechanisms, neural architectures and evolutionary
dynamics), and for parameter tuning of some more complex
controllers, its approximately 15 years of development have
not seen much scaling up[2]. That is, we have yet to see the
evolution of robot controllers (as opposed to just parameters
of such) for any really complex problem - problems where
artificial evolution becomes a superior alternative to manual
design of controllers.
We believe that some of the reason for this lack of progress
is the limited environments, sensor data, embodiments, and
tasks in most evolutionary robotics experiments. A typical
such experiment uses a semi-holonomic robot operating in
an impoverished environment (in many ways resembling
a “Skinner box”, the simplistic boxes pioneered by B. F.
Skinner for studying operant conditioning[3]), using simple,
low-bandwidth sensor input, doing a task that is hard to incre-
mentally scale up. The car racing task uses a more complex
and interesting robot morphology, as a car is more complex
to control than a semi-holonomic robot, but at the same
time it has more capabilites. While a simple racing track
might be as impoverished an environment as ever a Skinner
box, it can be scaled up. A controller might be evolved
to race a simple track, which can then be progressively
complexified (by adding competitors, gears, crossroads, blind
alleys, bridges, jumps etc.) up to and above the level of the
DARPA Grand Challenge, without ever changing the nature
of the fitness function, thus ensuring smooth scaling up.
This solution to the problems of the environment and task
scalability does come at cost: the car will probably need ever
more sophisticated sensors, including high-bandwitdh visual
input, to navigate more complex tracks. But such input can be
supplied, if we use one of today’s graphically sophisticated
racing games as experimental environment. This shifts the
problem to one of controller encodings that can handle such
complex input.
A. Prior research
1) Evolutionary car racing: A few investigations into
evolutionary car racing can be found in the recent litera-
ture. Togelius and Lucas[4] investigated various controller
architectures and sensor input representations for simulated
car racing. It was concluded that the only combination out