biosim4

What is this?

This pile of code was used to simulate biological creatures that evolve through natural selection.
The results of the experiments are summarized in this YouTube video:

      “I programmed some creatures. They evolved.”

      https://www.youtube.com/watch?v=N3tRFayqVtk

This code lacks a friendly interface, so compiling and executing the program may
require attention to details. If you ask questions in the Issues,
I’ll try to help if I can.

Document Contents

Code walkthrough

Main data structures

The code in the src directory compiles to a single console program named biosim4. When it is
invoked, it will read parameters from a config file named biosim4.ini by default. A different
config file can be specified on the command line.

The simulator will then configure a 2D arena where the creatures live. Class Grid (see grid.h and grid.cpp)
contains a 2D array of 16-bit indexes, where each nonzero index refers to a specific individual in class Peeps (see below).
Zero values in Grid indicate empty locations. Class Grid does not know anything else about the world; it only
stores indexes to represent who lives where.

The population of creatures is stored in class Peeps (see peeps.h and peeps.cpp). Class Peeps contains
all the individuals in the simulation, stored as instances of struct Indiv in a std::vector container.
The indexes in class Grid are indexes into the vector of individuals in class Peeps. Class Peeps keeps a
container of struct Indiv, but otherwise does not know anything about the internal workings of individuals.

Each individual is represented by an instance of struct Indiv (see indiv.h and indiv.cpp). Struct Indiv
contains an individual’s genome, its corresponding neural net brain, and a redundant copy of the individual’s
X,Y location in the 2D grid. It also contains a few other parameters for the individual, such as its
“responsiveness” level, oscillator period, age, and other personal parameters. Struct Indiv knows how
to convert an individual’s genome into its neural net brain at the beginning of the simulation.
It also knows how to print the genome and neural net brain in text format to stdout during a simulation.
It also has a function Indiv::getSensor() that is called to compute the individual’s input neurons for
each simulator step.

All the simulator code lives in the BS namespace (short for “biosim”.)

Config file

The config file, named biosim4.ini by default, contains all the tunable parameters for a
simulation run. The biosim4 executable reads the config file at startup, then monitors it for
changes during the simulation. Although it’s not foolproof, many parameters can be modified during
the simulation run. Class ParamManager (see params.h and params.cpp) manages the configuration
parameters and makes them available to the simulator through a read-only pointer provided by
ParamManager::getParamRef().

See the provided biosim4.ini for documentation for each parameter. Most of the parameters
in the config file correspond to members in struct Params (see params.h). A few additional
parameters may be stored in struct Params. See the documentation in params.h for how to
support new parameters.

Program output

Depending on the parameters in the config file, the following data can be produced:

  • The simulator will append one line to logs/epoch.txt after the completion of
    each generation. Each line records the generation number, number of individuals
    who survived the selection criterion, an estimate of the population’s genetic
    diversity, average genome length, and number of deaths due to the “kill” gene.
    This file can be fed to tools/graphlog.gp to produce a graphic plot.

  • The simulator will display a small number of sample genomes at regular
    intervals to stdout. Parameters in the config file specify the number and interval.
    The genomes are displayed in hex format and also in a mnemonic format that can
    be fed to tools/graph-nnet.py to produce a graphic network diagram.

  • Movies of selected generations will be created in the images/ directory. Parameters
    in the config file specify the interval at which to make movies. Each movie records
    a single generation.

  • At intervals, a summary is printed to stdout showing the total number of neural
    connections throughout the population from each possible sensory input neuron and to each
    possible action output neuron.

Main program loop

The simulator starts with a call to simulator() in simulator.cpp. After initializing the
world, the simulator executes three nested loops: the outer loop for each generation,
an inner loop for each simulator step within the generation, and an innermost loop for
each individual in the population. The innermost loop is thread-safe so that it can
be parallelized by OpenMP.

At the end of each simulator step, a call is made to endOfSimStep() in single-thread
mode (see endOfSimStep.cpp) to create a video frame representing the locations of all
the individuals at the end of the simulator step. The video frame is pushed on to a
stack to be converted to a movie later. Also some housekeeping may be done for certain
selection scenarios. See the comments in endOfSimStep.cpp for more information.

At the end of each generation, a call is made to endOfGeneration() in single-thread
mode (see endOfGeneration.cpp) to create a video from the saved video frames.
Also a new graph might be generated showing the progress of the simulation. See
endOfGeneraton.cpp for more information.

Sensory inputs and action outputs

See the YouTube video (link above) for a description of the sensory inputs and action
outputs. Each sensory input and each action output is a neuron in the individual’s
neural net brain.

The header file sensors-actions.h contains enum Sensor which enumerates all the possible sensory
inputs and enum Action which enumerates all the possible action outputs.
In enum Sensor, all the sensory inputs before the enumerant NUM_SENSES will
be compiled into the executable, and all action outputs before NUM_ACTIONS
will be compiled. By rearranging the enumerants in those enums, you can select
a subset of all possible sensory and action neurons to be compiled into the
simulator.

Basic value types

There are a few basic value types:

  • enum Compass represents eight-way directions with enumerants N=0, NE, E, SW, S, SW, W, NW, CENTER.

  • struct Dir is an abstract representation of the values of enum Compass.

  • struct Coord is a signed 16-bit integer X,Y coordinate pair. It is used to represent a location
    in the 2D world, or can represent the difference between two locations.

  • struct Polar holds a signed 32-bit integer magnitude and a direction of type Dir.

Various conversions and math are possible between these basic types. See unitTestBasicTypes.cpp
for examples. Also see basicTypes.h for more information.

Pheromones

A simple system is used to simulate pheromones emitted by the individuals. Pheromones
are called “signals” in simulator-speak (see signals.h and signals.cpp). Struct Signals
holds a single layer that overlays the 2D world in class Grid. Each location can contain
a level of pheromone (there’s only a single kind of pheromone supported at present). The
pheromone level at any grid location is stored as an unsigned 8-bit integer, where zero means no
pheromone, and 255 is the maximum. Each time an individual emits a pheromone, it increases
the pheromone values in a small neighborhood around the individual up to the maximum
value of 255. Pheromone levels decay over time if they are not replenished
by the individuals in the area.

Useful utility functions

The utility function visitNeighborhood() in grid.cpp can be used to execute a
user-defined lambda or function over each location
within a circular neighborhood defined by a center point and floating point radius. The function
calls the user-defined function once for each location, passing it a Coord value. Only locations
within the bounds of the grid are visited. The center location is included among the visited
locations. For example, a radius of 1.0 includes only the center location plus four neighboring locations.
A radius of 1.5 includes the center plus all the eight-way neighbors. The radius can be arbitrarily large
but large radii require lots of CPU cycles.

Installing the code


Copy the directory structure to a location of your choice.

Building the executable


System requirements

This code is known to run in the following environment:

  • Ubuntu 21.04
  • cimg-dev 2.8.4 or later
  • libopencv-dev 4.2 or later
  • gcc 9.3 or 10.3
  • python-igraph 0.8.3 (used only by tools/graph-nnet.py)
  • gnuplot 5.2.8 (used only by tools/graphlog.gp)

The code also runs in distributions based on Ubuntu 20.04, but only if the default version of
cimg-dev is replaced with version 2.8.4 or later.

Compiling

Two (and a half) ways to compile:

  • The file named “biosim4.cbp” is a configuration file for the Code::Blocks IDE version 20.03.

  • A Makefile is provided which
    was created from biosim4.cbp with cbp2make, but is not tested. A default “make” will generate a debug and a
    release version.

  • A Dockerfile is provided which leverages the aforementioned Makefile.

To build a Docker environment in which you can compile the program:

docker build -t biosim4 .

You can then compile the program with an ephemeral container:

docker run --rm -ti -v `pwd`:/app --name biosim biosim4 make

When you exit the container, the files compiled in your container files will persist in ./bin.

Bugs


If you try to compile the simulator under a distribution based on Ubuntu 20.04, you will encounter this
bug in the version of CImg.h (package cimg-dev) provided by the package maintainer:

      https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=951965

In biosim4, CImg.h is used only as a convenient interface to OpenCV
to generate movies of the simulated creatures in their 2D world. You have several
choices if you want to proceed with Ubuntu 20.04:

  • You can strip out the code that generates the movies and just run the simulator without the movies. Most of
    that graphics code is in imageWriter.cpp and imageWriter.h.

  • You can upgrade your CImg.h to version 2.8.4 or later by getting it from the appropriate Debian repository.
    Sorry I don’t have the instructions at hand to do this.

  • You could convert the CImg.h function calls to use OpenCV directly. Sorry I don’t have a guide for how
    to do that.

Execution


Edit the config file (default “biosim4.ini”) for the parameters you want for the simulation run, then execute the Debug
or Release executable in the bin directory. Optionally specify the name of the config file as the first
command line argument, e.g.:

./bin/Release/biosim4 [biosim4.ini]

Note: If using docker,

docker run --rm -ti -v `pwd`:/app --name biosim biosim4 bash

will put you into
an environment where you can run the above and have all your files persist when you exit (using Ctrl-D).

Tools directory


tools/graphlog.gp takes the generated log file logs/epoch-log.txt
and generates a graphic plot of the simulation run in images/log.png. You may need to adjust
the directory paths in graphlog.gp for your environment. graphlog.gp can be invoked manually,
or if the option “updateGraphLog” is set to true
in the simulation config file, the simulator will try to invoke tools/graphlog.gp automatically
during the simulation run. Also see the parameter named updateGraphLogStride in the config file.

tools/graph-nnet.py takes a text file (hardcoded name “net.txt”) and generates a neural net
connection diagram using igraph. The file net.txt contains an encoded form of one genome, and
must be the same format as the files
generated by displaySampleGenomes() in src/analysis.cpp which is called by simulator() in
src/simulator.cpp. The genome output is printed to stdout automatically
if the parameter named “displaySampleGenomes” is set to nonzero in the config file.
An individual genome can be copied from that output stream and renamed “net.txt” in order to run
graph-nnet.py.

Note: If using the docker run ... bash command, the presumed directory structure would necessitate the
following syntax:

gnuplot tools/graphlog.gp
cd tools && python3 graph-nnet.py

Build log


In case it helps for debugging the build process, here is a build log from Code::Blocks running under Ubuntu 21.04:

-------------- Clean: Release in biosim4 (compiler: GNU GCC Compiler)---------------

Cleaned "biosim4 - Release"

-------------- Build: Release in biosim4 (compiler: GNU GCC Compiler)---------------

g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/analysis.cpp -o obj/Release/src/analysis.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/basicTypes.cpp -o obj/Release/src/basicTypes.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/createBarrier.cpp -o obj/Release/src/createBarrier.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/endOfGeneration.cpp -o obj/Release/src/endOfGeneration.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/endOfSimStep.cpp -o obj/Release/src/endOfSimStep.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/executeActions.cpp -o obj/Release/src/executeActions.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/feedForward.cpp -o obj/Release/src/feedForward.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/genome-compare.cpp -o obj/Release/src/genome-compare.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/genome.cpp -o obj/Release/src/genome.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/getSensor.cpp -o obj/Release/src/getSensor.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/grid.cpp -o obj/Release/src/grid.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/imageWriter.cpp -o obj/Release/src/imageWriter.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/indiv.cpp -o obj/Release/src/indiv.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/main.cpp -o obj/Release/src/main.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/params.cpp -o obj/Release/src/params.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/peeps.cpp -o obj/Release/src/peeps.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/random.cpp -o obj/Release/src/random.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/signals.cpp -o obj/Release/src/signals.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/simulator.cpp -o obj/Release/src/simulator.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/spawnNewGeneration.cpp -o obj/Release/src/spawnNewGeneration.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/survival-criteria.cpp -o obj/Release/src/survival-criteria.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/unitTestBasicTypes.cpp -o obj/Release/src/unitTestBasicTypes.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/unitTestConnectNeuralNetWiringFromGenome.cpp -o obj/Release/src/unitTestConnectNeuralNetWiringFromGenome.o
g++ -Wall -fexceptions -fopenmp -O3 -I/usr/include/opencv4 -c /home/dm/sw/biosim4-git/src/unitTestGridVisitNeighborhood.cpp -o obj/Release/src/unitTestGridVisitNeighborhood.o
g++  -o bin/Release/biosim4 obj/Release/src/analysis.o obj/Release/src/basicTypes.o obj/Release/src/createBarrier.o obj/Release/src/endOfGeneration.o obj/Release/src/endOfSimStep.o obj/Release/src/executeActions.o obj/Release/src/feedForward.o obj/Release/src/genome-compare.o obj/Release/src/genome.o obj/Release/src/getSensor.o obj/Release/src/grid.o obj/Release/src/imageWriter.o obj/Release/src/indiv.o obj/Release/src/main.o obj/Release/src/params.o obj/Release/src/peeps.o obj/Release/src/random.o obj/Release/src/signals.o obj/Release/src/simulator.o obj/Release/src/spawnNewGeneration.o obj/Release/src/survival-criteria.o obj/Release/src/unitTestBasicTypes.o obj/Release/src/unitTestConnectNeuralNetWiringFromGenome.o obj/Release/src/unitTestGridVisitNeighborhood.o  -lX11 -lgomp -pthread -O3 -s  /usr/lib/x86_64-linux-gnu/libopencv_core.so /usr/lib/x86_64-linux-gnu/libopencv_video.so /usr/lib/x86_64-linux-gnu/libopencv_videoio.so
Output file is bin/Release/biosim4 with size 778.42 KB
Process terminated with status 0 (0 minute(s), 11 second(s))
0 error(s), 0 warning(s) (0 minute(s), 11 second(s))

GitHub

View Github