`pygad.gacnn`

Module¶

This section of the PyGAD’s library documentation discusses the
**pygad.gacnn** module.

The `pygad.gacnn`

module trains convolutional neural networks using
the genetic algorithm. It makes use of the 2 modules `pygad`

and
`pygad.cnn`

.

`pygad.gacnn.GACNN`

Class¶

The `pygad.gacnn`

module has a class named `pygad.gacnn.GACNN`

for
training convolutional neural networks (CNNs) using the genetic
algorithm. The constructor, methods, function, and attributes within the
class are discussed in this section.

`__init__()`

¶

In order to train a CNN using the genetic algorithm, the first thing to
do is to create an instance of the `pygad.gacnn.GACNN`

class.

The `pygad.gacnn.GACNN`

class constructor accepts the following
parameters:

`model`

: model: An instance of the pygad.cnn.Model class representing the architecture of all solutions in the population.`num_solutions`

: Number of CNNs (i.e. solutions) in the population. Based on the value passed to this parameter, a number of identical CNNs are created where their parameters are optimized using the genetic algorithm.

## Instance Attributes¶

All the parameters in the `pygad.gacnn.GACNN`

class constructor are
used as instance attributes. Besides such attributes, there is an extra
attribute added to the instances from the `pygad.gacnn.GACNN`

class
which is:

`population_networks`

: A list holding references to all the solutions (i.e. CNNs) used in the population.

## Methods in the GACNN Class¶

This section discusses the methods available for instances of the
`pygad.gacnn.GACNN`

class.

`create_population()`

¶

The `create_population()`

method creates the initial population of the
genetic algorithm as a list of CNNs (i.e. solutions). All the networks
are copied from the CNN model passed to constructor of the GACNN class.

The list of networks is assigned to the `population_networks`

attribute of the instance.

`update_population_trained_weights()`

¶

The `update_population_trained_weights()`

method updates the
`trained_weights`

attribute of the layers of each network (check the
documentation of the `pygad.cnn`

module) for more information)
according to the weights passed in the `population_trained_weights`

parameter.

Accepts the following parameters:

`population_trained_weights`

: A list holding the trained weights of all networks as matrices. Such matrices are to be assigned to the`trained_weights`

attribute of all layers of all networks.

# Functions in the `pygad.gacnn`

Module¶

This section discusses the functions in the `pygad.gacnn`

module.

`pygad.gacnn.population_as_vectors()`

¶

Accepts the population as a list of references to the
`pygad.cnn.Model`

class and returns a list holding all weights of the
layers of each solution (i.e. network) in the population as a vector.

For example, if the population has 6 solutions (i.e. networks), this function accepts references to such networks and returns a list with 6 vectors, one for each network (i.e. solution). Each vector holds the weights for all layers for a single network.

Accepts the following parameters:

`population_networks`

: A list holding references to the`pygad.cnn.Model`

class of the networks used in the population.

Returns a list holding the weights vectors for all solutions (i.e. networks).

`pygad.gacnn.population_as_matrices()`

¶

Accepts the population as both networks and weights vectors and returns the weights of all layers of each solution (i.e. network) in the population as a matrix.

For example, if the population has 6 solutions (i.e. networks), this function returns a list with 6 matrices, one for each network holding its weights for all layers.

Accepts the following parameters:

`population_networks`

: A list holding references to the`pygad.cnn.Model`

class of the networks used in the population.`population_vectors`

: A list holding the weights of all networks as vectors. Such vectors are to be converted into matrices.

Returns a list holding the weights matrices for all solutions (i.e. networks).

# Steps to Build and Train CNN using Genetic Algorithm¶

The steps to use this project for building and training a neural network using the genetic algorithm are as follows:

- Prepare the training data.
- Create an instance of the
`pygad.gacnn.GACNN`

class. - Fetch the population weights as vectors.
- Prepare the fitness function.
- Prepare the generation callback function.
- Create an instance of the
`pygad.GA`

class. - Run the created instance of the
`pygad.GA`

class. - Plot the Fitness Values
- Information about the best solution.
- Making predictions using the trained weights.
- Calculating some statistics.

Let’s start covering all of these steps.

## Prepare the Training Data¶

Before building and training neural networks, the training data (input and output) is to be prepared. The inputs and the outputs of the training data are NumPy arrays.

The data used in this example is available as 2 files:

- dataset_inputs.npy: Data inputs. https://github.com/ahmedfgad/NumPyCNN/blob/master/dataset_inputs.npy
- dataset_outputs.npy: Class labels. https://github.com/ahmedfgad/NumPyCNN/blob/master/dataset_outputs.npy

The data consists of 4 classes of images. The image shape is
`(100, 100, 3)`

and there are 20 images per class. For more
information about the dataset, check the **Reading the Data** section of
the `pygad.cnn`

module.

Simply download these 2 files and read them according to the next code.

```
import numpy
train_inputs = numpy.load("dataset_inputs.npy")
train_outputs = numpy.load("dataset_outputs.npy")
```

For the output array, each element must be a single number representing
the class label of the sample. The class labels must start at `0`

. So,
if there are 80 samples, then the shape of the output array is `(80)`

.
If there are 5 classes in the data, then the values of all the 200
elements in the output array must range from 0 to 4 inclusive.
Generally, the class labels start from `0`

to `N-1`

where `N`

is
the number of classes.

Note that the project only supports that each sample is assigned to only one class.

## Building the Network Architecture¶

Here is an example for a CNN architecture.

```
import pygad.cnn
input_layer = pygad.cnn.Input2D(input_shape=(80, 80, 3))
conv_layer = pygad.cnn.Conv2D(num_filters=2,
kernel_size=3,
previous_layer=input_layer,
activation_function="relu")
average_pooling_layer = pygad.cnn.AveragePooling2D(pool_size=5,
previous_layer=conv_layer,
stride=3)
flatten_layer = pygad.cnn.Flatten(previous_layer=average_pooling_layer)
dense_layer = pygad.cnn.Dense(num_neurons=4,
previous_layer=flatten_layer,
activation_function="softmax")
```

After the network architecture is prepared, the next step is to create a CNN model.

## Building Model¶

The CNN model is created as an instance of the `pygad.cnn.Model`

class. Here is an example.

```
model = pygad.cnn.Model(last_layer=dense_layer,
epochs=5,
learning_rate=0.01)
```

After the model is created, a summary of the model architecture can be printed.

## Model Summary¶

The `summary()`

method in the `pygad.cnn.Model`

class prints a
summary of the CNN model.

```
model.summary()
```

```
----------Network Architecture----------
<class 'cnn.Conv2D'>
<class 'cnn.AveragePooling2D'>
<class 'cnn.Flatten'>
<class 'cnn.Dense'>
----------------------------------------
```

The next step is to create an instance of the `pygad.gacnn.GACNN`

class.

## Create an Instance of the `pygad.gacnn.GACNN`

Class¶

After preparing the input data and building the CNN model, an instance
of the `pygad.gacnn.GACNN`

class is created by passing the appropriate
parameters.

Here is an example where the `num_solutions`

parameter is set to 4
which means the genetic algorithm population will have 6 solutions (i.e.
networks). All of these 6 CNNs will have the same architectures as
specified by the `model`

parameter.

```
import pygad.gacnn
GACNN_instance = pygad.gacnn.GACNN(model=model,
num_solutions=4)
```

After creating the instance of the `pygad.gacnn.GACNN`

class, next is
to fetch the weights of the population as a list of vectors.

## Fetch the Population Weights as Vectors¶

For the genetic algorithm, the parameters (i.e. genes) of each solution are represented as a single vector.

For this task, the weights of each CNN must be available as a single vector. In other words, the weights of all layers of a CNN must be grouped into a vector.

To create a list holding the population weights as vectors, one for each
network, the `pygad.gacnn.population_as_vectors()`

function is used.

```
population_vectors = gacnn.population_as_vectors(population_networks=GACNN_instance.population_networks)
```

Such population of vectors is used as the initial population.

```
initial_population = population_vectors.copy()
```

After preparing the population weights as a set of vectors, next is to prepare 2 functions which are:

- Fitness function.
- Callback function after each generation.

## Prepare the Fitness Function¶

The PyGAD library works by allowing the users to customize the genetic algorithm for their own problems. Because the problems differ in how the fitness values are calculated, then PyGAD allows the user to use a custom function as a maximization fitness function. This function must accept 2 positional parameters representing the following:

- The solution.
- The solution index in the population.

The fitness function must return a single number representing the fitness. The higher the fitness value, the better the solution.

Here is the implementation of the fitness function for training a CNN.

It uses the `pygad.cnn.predict()`

function to predict the class labels
based on the current solution’s weights. The `pygad.cnn.predict()`

function uses the trained weights available in the `trained_weights`

attribute of each layer of the network for making predictions.

Based on such predictions, the classification accuracy is calculated. This accuracy is used as the fitness value of the solution. Finally, the fitness value is returned.

```
def fitness_func(solution, sol_idx):
global GACNN_instance, data_inputs, data_outputs
predictions = GACNN_instance.population_networks[sol_idx].predict(data_inputs=data_inputs)
correct_predictions = numpy.where(predictions == data_outputs)[0].size
solution_fitness = (correct_predictions/data_outputs.size)*100
return solution_fitness
```

## Prepare the Generation Callback Function¶

After each generation of the genetic algorithm, the fitness function
will be called to calculate the fitness value of each solution. Within
the fitness function, the `pygad.cnn.predict()`

function is used for
predicting the outputs based on the current solution’s
`trained_weights`

attribute. Thus, it is required that such an
attribute is updated by weights evolved by the genetic algorithm after
each generation.

PyGAD has a parameter accepted by the `pygad.GA`

class constructor
named `on_generation`

. It could be assigned to a function that is
called after each generation. The function must accept a single
parameter representing the instance of the `pygad.GA`

class.

This callback function can be used to update the `trained_weights`

attribute of layers of each network in the population.

Here is the implementation for a function that updates the
`trained_weights`

attribute of the layers of the population networks.

It works by converting the current population from the vector form to
the matric form using the `pygad.gacnn.population_as_matrices()`

function. It accepts the population as vectors and returns it as
matrices.

The population matrices are then passed to the
`update_population_trained_weights()`

method in the `pygad.gacnn`

module to update the `trained_weights`

attribute of all layers for all
solutions within the population.

```
def callback_generation(ga_instance):
global GACNN_instance, last_fitness
population_matrices = gacnn.population_as_matrices(population_networks=GACNN_instance.population_networks, population_vectors=ga_instance.population)
GACNN_instance.update_population_trained_weights(population_trained_weights=population_matrices)
print("Generation = {generation}".format(generation=ga_instance.generations_completed))
```

After preparing the fitness and callback function, next is to create an
instance of the `pygad.GA`

class.

## Create an Instance of the `pygad.GA`

Class¶

Once the parameters of the genetic algorithm are prepared, an instance
of the `pygad.GA`

class can be created. Here is an example where the
number of generations is 10.

```
import pygad
num_parents_mating = 4
num_generations = 10
mutation_percent_genes = 5
ga_instance = pygad.GA(num_generations=num_generations,
num_parents_mating=num_parents_mating,
initial_population=initial_population,
fitness_func=fitness_func,
mutation_percent_genes=mutation_percent_genes,
on_generation=callback_generation)
```

The last step for training the neural networks using the genetic
algorithm is calling the `run()`

method.

## Run the Created Instance of the `pygad.GA`

Class¶

By calling the `run()`

method from the `pygad.GA`

instance, the
genetic algorithm will iterate through the number of generations
specified in its `num_generations`

parameter.

```
ga_instance.run()
```

## Plot the Fitness Values¶

After the `run()`

method completes, the `plot_fitness()`

method can
be called to show how the fitness values evolve by generation.

```
ga_instance.plot_fitness()
```

## Information about the Best Solution¶

The following information about the best solution in the last population
is returned using the `best_solution()`

method in the `pygad.GA`

class.

- Solution
- Fitness value of the solution
- Index of the solution within the population

Here is how such information is returned.

```
solution, solution_fitness, solution_idx = ga_instance.best_solution()
print("Parameters of the best solution : {solution}".format(solution=solution))
print("Fitness value of the best solution = {solution_fitness}".format(solution_fitness=solution_fitness))
print("Index of the best solution : {solution_idx}".format(solution_idx=solution_idx))
```

```
...
Fitness value of the best solution = 83.75
Index of the best solution : 0
Best fitness value reached after 4 generations.
```

## Making Predictions using the Trained Weights¶

The `pygad.cnn.predict()`

function can be used to make predictions
using the trained network. As printed, the network is able to predict
the labels correctly.

```
predictions = pygad.cnn.predict(last_layer=GANN_instance.population_networks[solution_idx], data_inputs=data_inputs)
print("Predictions of the trained network : {predictions}".format(predictions=predictions))
```

## Calculating Some Statistics¶

Based on the predictions the network made, some statistics can be calculated such as the number of correct and wrong predictions in addition to the classification accuracy.

```
num_wrong = numpy.where(predictions != data_outputs)[0]
num_correct = data_outputs.size - num_wrong.size
accuracy = 100 * (num_correct/data_outputs.size)
print("Number of correct classifications : {num_correct}.".format(num_correct=num_correct))
print("Number of wrong classifications : {num_wrong}.".format(num_wrong=num_wrong.size))
print("Classification accuracy : {accuracy}.".format(accuracy=accuracy))
```

```
Number of correct classifications : 67.
Number of wrong classifications : 13.
Classification accuracy : 83.75.
```

# Examples¶

This section gives the complete code of some examples that build and train neural networks using the genetic algorithm. Each subsection builds a different network.

## Image Classification¶

This example is discussed in the **Steps to Build and Train CNN using
Genetic Algorithm** section that builds the an image classifier. Its
complete code is listed below.

```
import numpy
import pygad.cnn
import pygad.gacnn
import pygad
"""
Convolutional neural network implementation using NumPy
A tutorial that helps to get started (Building Convolutional Neural Network using NumPy from Scratch) available in these links:
https://www.linkedin.com/pulse/building-convolutional-neural-network-using-numpy-from-ahmed-gad
https://towardsdatascience.com/building-convolutional-neural-network-using-numpy-from-scratch-b30aac50e50a
https://www.kdnuggets.com/2018/04/building-convolutional-neural-network-numpy-scratch.html
It is also translated into Chinese: http://m.aliyun.com/yunqi/articles/585741
"""
def fitness_func(solution, sol_idx):
global GACNN_instance, data_inputs, data_outputs
predictions = GACNN_instance.population_networks[sol_idx].predict(data_inputs=data_inputs)
correct_predictions = numpy.where(predictions == data_outputs)[0].size
solution_fitness = (correct_predictions/data_outputs.size)*100
return solution_fitness
def callback_generation(ga_instance):
global GACNN_instance, last_fitness
population_matrices = pygad.gacnn.population_as_matrices(population_networks=GACNN_instance.population_networks,
population_vectors=ga_instance.population)
GACNN_instance.update_population_trained_weights(population_trained_weights=population_matrices)
print("Generation = {generation}".format(generation=ga_instance.generations_completed))
print("Fitness = {fitness}".format(fitness=ga_instance.best_solutions_fitness))
data_inputs = numpy.load("dataset_inputs.npy")
data_outputs = numpy.load("dataset_outputs.npy")
sample_shape = data_inputs.shape[1:]
num_classes = 4
data_inputs = data_inputs
data_outputs = data_outputs
input_layer = pygad.cnn.Input2D(input_shape=sample_shape)
conv_layer1 = pygad.cnn.Conv2D(num_filters=2,
kernel_size=3,
previous_layer=input_layer,
activation_function="relu")
average_pooling_layer = pygad.cnn.AveragePooling2D(pool_size=5,
previous_layer=conv_layer1,
stride=3)
flatten_layer = pygad.cnn.Flatten(previous_layer=average_pooling_layer)
dense_layer2 = pygad.cnn.Dense(num_neurons=num_classes,
previous_layer=flatten_layer,
activation_function="softmax")
model = pygad.cnn.Model(last_layer=dense_layer2,
epochs=1,
learning_rate=0.01)
model.summary()
GACNN_instance = pygad.gacnn.GACNN(model=model,
num_solutions=4)
# GACNN_instance.update_population_trained_weights(population_trained_weights=population_matrices)
# population does not hold the numerical weights of the network instead it holds a list of references to each last layer of each network (i.e. solution) in the population. A solution or a network can be used interchangeably.
# If there is a population with 3 solutions (i.e. networks), then the population is a list with 3 elements. Each element is a reference to the last layer of each network. Using such a reference, all details of the network can be accessed.
population_vectors = pygad.gacnn.population_as_vectors(population_networks=GACNN_instance.population_networks)
# To prepare the initial population, there are 2 ways:
# 1) Prepare it yourself and pass it to the initial_population parameter. This way is useful when the user wants to start the genetic algorithm with a custom initial population.
# 2) Assign valid integer values to the sol_per_pop and num_genes parameters. If the initial_population parameter exists, then the sol_per_pop and num_genes parameters are useless.
initial_population = population_vectors.copy()
num_parents_mating = 2 # Number of solutions to be selected as parents in the mating pool.
num_generations = 10 # Number of generations.
mutation_percent_genes = 0.1 # Percentage of genes to mutate. This parameter has no action if the parameter mutation_num_genes exists.
ga_instance = pygad.GA(num_generations=num_generations,
num_parents_mating=num_parents_mating,
initial_population=initial_population,
fitness_func=fitness_func,
mutation_percent_genes=mutation_percent_genes,
on_generation=callback_generation)
ga_instance.run()
# After the generations complete, some plots are showed that summarize how the outputs/fitness values evolve over generations.
ga_instance.plot_fitness()
# Returning the details of the best solution.
solution, solution_fitness, solution_idx = ga_instance.best_solution()
print("Parameters of the best solution : {solution}".format(solution=solution))
print("Fitness value of the best solution = {solution_fitness}".format(solution_fitness=solution_fitness))
print("Index of the best solution : {solution_idx}".format(solution_idx=solution_idx))
if ga_instance.best_solution_generation != -1:
print("Best fitness value reached after {best_solution_generation} generations.".format(best_solution_generation=ga_instance.best_solution_generation))
# Predicting the outputs of the data using the best solution.
predictions = GACNN_instance.population_networks[solution_idx].predict(data_inputs=data_inputs)
print("Predictions of the trained network : {predictions}".format(predictions=predictions))
# Calculating some statistics
num_wrong = numpy.where(predictions != data_outputs)[0]
num_correct = data_outputs.size - num_wrong.size
accuracy = 100 * (num_correct/data_outputs.size)
print("Number of correct classifications : {num_correct}.".format(num_correct=num_correct))
print("Number of wrong classifications : {num_wrong}.".format(num_wrong=num_wrong.size))
print("Classification accuracy : {accuracy}.".format(accuracy=accuracy))
```