Genetic Algorithm with Neural Network

Recently I implemented genetic algorithm with three layer neural network to balance inverted pendulum (Open AI gym CartPole). Results were pretty amazing.

Genetic Algorithm is so powerful that model learns very fast(less than 5 iterations). Most of the cases it performs perfectly during testing may be due to search space is not that large compared to other environments. If we do not end environment it will hold the pole for forever. I run environment up to 5000-time steps :D. Here is complete video of model test:-

We have four observations and one action so neural network consists four input one out. Initially, weights and biases are randomly selected but tuning is required to get perfect action for each time step. Here genetic algorithm plays an important role for the optimal solution of weights and biases. This algorithm works on the survival of the fittest principle so generation(one set of weights and biases) with thebest performance will survive. Above diagram simplified below showing different weight-bias sets associated with different nodes.

Initially, this weights and biases randomly selected then for each selection the cartpole environment run and scores stored until game over. Now sets with top scores selected and arranged in a particular sequence similar to DNA for crossover (in our case swapping two sequences random) to generate next sequence. The image below shows that how DNAs with red and gray dots exchange some porting with each other to generate two new DNAs.

Exchange point can be anywhere as it is chosen randomly. After crossover mutation is done which is also a random update of few of the dots as a brown dot and a red dot become blue and black respectively after mutation. These new generated along with parent DNAs(weights-biases) put inside the neural network and again see the scores from each and then loop-over the evolution process of genetic algorithm again until perfect score achieved.

Please share your thoughts about this write-up. What do you think about this algorithm?

References:-

Related posts:

Short selling in the stock market is a practice where the trader sells a share that they do not own when he or she sells them. The investor sells the stock expecting that the share price will drop…

A group of international researchers has collaboratively demonstrated how a nanotechnology-based compound can deliver an oral vaccine against hepatitis B to the immune system. When particles…

Storichain is a DApp and storytellers settlement protocol that create stories, distribute and control story IPs. It is also a protocol that gives a contract for story advent contracts. Users can…