Evolving robust and specialized car racing skills

TABLE III

Evolving controllers for individual tracks from s cratch
with sensor mutation turned on; format as in table I.

race around one of these track without getting stuck or taking
excessively long time, the evolutionary process in itself is
not reliable. In fact, most of the evolutionary runs are false
starts. For tracks 3, 4, 5 and 6, the situation is different
as at least half of all evolutionary runs produce proficient
controllers. The best evolved controllers for these tracks get
around the track fairly fast without colliding with walls. For
tracks 7 and 8, however, we have not been able to evolve
proficient controllers from scratch at all. The “best” (least
bad) controllers evolved for track 7 might get halfway around
the track before getting stuck on a wall, or losing orientation
and starting to move back along the track.

2) Generality of evolved controllers: Next, we examined
the generality of these controllers by testing their perfor-
mance of the best controller for each track on each of the
ten tracks. The results are presented in figure II, and clearly
show that the generality is very low. No controller performed
very well on any track it had not been evolved on, with the
interesting exception of the controller evolved for track 1,
that actually performed better on track 3 than on the track
for which it had been evolved, and on which it had a rather
mediocre performance. It should be noted that both track
1 and track 3 (like all odd-numbered tracks) run counter-
clockwise, and there indeed seems to be a slight bias for the
other controllers to get higher fitness on tracks running in
the same direction as the track for which they were evolved.
We have not analysed this further.

B. Evolved sensor parameters

1) Evolving from scratch: Evolving controllers from
scratch with sensor parameter mutations turned on resulted in
somewhat lower average fitnesses and numbers of proficient
controllers, as can be seen in table III. The controllers that
reached proficiency seemed to be roughly equally fit as those
evolved with fixed sensors, but more evolutionary runs got
stuck in some local optimum and never produced proficient
controllers when sensor parameters were evolvable. It is not
known whether this is simply because of the increase in
search space dimensionality caused by the addition of sensor
parameters, or if they complicate the evolutionary process in
some other way.

2) Generality of evolved controllers: Controllers evolved
with evolvable sensor parameters turn out to generalize really
badly, almost as badly as the controllers evolved with fixed
sensors. However, there are some interesting differences, and
the controllers evolved for track 1, 2 and 6 (but not the others)
actually perform better on tracks for which they were not
evolved. It is quite hard to see any kind of logic in which
controllers will do well on which tracks, except those they
were evolved for, and more data would definitely be needed
to resolve this.

V. Evolving robust driving skills

The next suite of experiments were on evolving robust
controllers, i.e. controllers that can drive proficiently on a
large set of tracks.

A. Simultaneous evolution

Our first attempt consisted in evolving controllers on all
tracks at once. For this purpose, we ran several evolutionary
runs where each controller was tested on all the first six
tracks, each for three trials, and the fitness was averaged over
all these trials. We ran several evolutionary runs with this
setup, and with both evolvable and fixed sensor parameters,
for long periods of time, but found very little progress - no
controller reached an average fitness above 1.

B. Incremental evolution

Abandoning this method, we tried incremental evolution.
The idea here was to evolve a controller on one track, and
when it reached proficiency (mean fitness above 1.5) add
another track to the training set - so that controllers are
now evaluated on both tracks and fitness averaged - and
continue evolving. This procedure is then repeated, with a
new track added to the fitness function each time the best
controller of the population has an average fitness of 1.5 or
over, until we have a controller that races all of the first
six tracks proficiently. The order of the tracks was 5, 6,
3, 4, 1 and finally 2, the rationale being that the balance
between clockwise and counterclockwise should be as equal
as possible in order to prevent lopsided controllers, and that
easier tracks should be added to the mix before harder ones.

This approach turned out to work much better than si-
multaneous evolution. Several runs were performed, and
while some of them failed to produce generally proficient
controllers, some others fared better. A successful run usually
takes a long time, on the order of several hundred gener-
ations, but it seems that once a run has come up with a
controller that is proficient on the first three or four tracks,
it almost always proceeds to produce a generally proficient
controller. One of the successful runs is depicted in figure 3,
and the mean fitness of the best controller of that run when
tested on all eight tracks separately is shown in IV. As can
be seen from this table, the controller does a good job on the
six tracks for which it was evolved, bar that it occasionally
gets stuck on a wall in track 2. It never makes its way around
track 7 or 8.

The successful runs were all made with sensor mutation
turned off. Some runs of incremental evolution were made
with sensor mutation allowed; however, they failed to pro-
duce any proficient controllers. We speculate that this is

<<   <   2   3   4   5   6   7   8   >   >>