Release History¶

PyGAD 1.0.17¶

Release Date: 15 April 2020

The pygad.GA class accepts a new argument named fitness_func which accepts a function to be used for calculating the fitness values for the solutions. This allows the project to be customized to any problem by building the right fitness function.

PyGAD 1.0.20¶

Release Date: 4 May 2020

The pygad.GA attributes are moved from the class scope to the instance scope.
Raising an exception for incorrect values of the passed parameters.
Two new parameters are added to the pygad.GA class constructor (init_range_low and init_range_high) allowing the user to customize the range from which the genes values in the initial population are selected.
The code object __code__ of the passed fitness function is checked to ensure it has the right number of parameters.

PyGAD 2.0.0¶

Release Date: 13 May 2020

The fitness function accepts a new argument named sol_idx representing the index of the solution within the population.
A new parameter to the pygad.GA class constructor named initial_population is supported to allow the user to use a custom initial population to be used by the genetic algorithm. If not None, then the passed population will be used. If None, then the genetic algorithm will create the initial population using the sol_per_pop and num_genes parameters.
The parameters sol_per_pop and num_genes are optional and set to None by default.
A new parameter named callback_generation is introduced in the pygad.GA class constructor. It accepts a function with a single parameter representing the pygad.GA class instance. This function is called after each generation. This helps the user to do post-processing or debugging operations after each generation.

PyGAD 2.1.0¶

Release Date: 14 May 2020

The best_solution() method in the pygad.GA class returns a new output representing the index of the best solution within the population. Now, it returns a total of 3 outputs and their order is: best solution, best solution fitness, and best solution index. Here is an example:

solution, solution_fitness, solution_idx = ga_instance.best_solution()
print("Parameters of the best solution :", solution)
print("Fitness value of the best solution :", solution_fitness, "\n")
print("Index of the best solution :", solution_idx, "\n")

A new attribute named best_solution_generation is added to the instances of the pygad.GA class. it holds the generation number at which the best solution is reached. It is only assigned the generation number after the run() method completes. Otherwise, its value is -1.

Example:

print("Best solution reached after {best_solution_generation} generations.".format(best_solution_generation=ga_instance.best_solution_generation))

The best_solution_fitness attribute is renamed to best_solutions_fitness (plural solution).
Mutation is applied independently for the genes.

PyGAD 2.2.1¶

Release Date: 17 May 2020

Adding 2 extra modules (pygad.nn and pygad.gann) for building and training neural networks with the genetic algorithm.

PyGAD 2.2.2¶

Release Date: 18 May 2020

The initial value of the generations_completed attribute of instances from the pygad.GA class is 0 rather than None.
An optional bool parameter named mutation_by_replacement is added to the constructor of the pygad.GA class. It works only when the selected type of mutation is random (mutation_type="random"). In this case, setting mutation_by_replacement=True means replace the gene by the randomly generated value. If False, then it has no effect and random mutation works by adding the random value to the gene. This parameter should be used when the gene falls within a fixed range and its value must not go out of this range. Here are some examples:

Assume there is a gene with the value 0.5.

If mutation_type="random" and mutation_by_replacement=False, then the generated random value (e.g. 0.1) will be added to the gene value. The new gene value is 0.5+0.1=0.6.

If mutation_type="random" and mutation_by_replacement=True, then the generated random value (e.g. 0.1) will replace the gene value. The new gene value is 0.1.

None value could be assigned to the mutation_type and crossover_type parameters of the pygad.GA class constructor. When None, this means the step is bypassed and has no action.

PyGAD 2.3.0¶

Release date: 1 June 2020

A new module named pygad.cnn is supported for building convolutional neural networks.
A new module named pygad.gacnn is supported for training convolutional neural networks using the genetic algorithm.
The pygad.plot_result() method has 3 optional parameters named title, xlabel, and ylabel to customize the plot title, x-axis label, and y-axis label, respectively.
The pygad.nn module supports the softmax activation function.
The name of the pygad.nn.predict_outputs() function is changed to pygad.nn.predict().
The name of the pygad.nn.train_network() function is changed to pygad.nn.train().

PyGAD 2.4.0¶

Release date: 5 July 2020

A new parameter named delay_after_gen is added which accepts a non-negative number specifying the time in seconds to wait after a generation completes and before going to the next generation. It defaults to 0.0 which means no delay after the generation.
The passed function to the callback_generation parameter of the pygad.GA class constructor can terminate the execution of the genetic algorithm if it returns the string stop. This causes the run() method to stop.

One important use case for that feature is to stop the genetic algorithm when a condition is met before passing though all the generations. The user may assigned a value of 100 to the num_generations parameter of the pygad.GA class constructor. Assuming that at generation 50, for example, a condition is met and the user wants to stop the execution before waiting the remaining 50 generations. To do that, just make the function passed to the callback_generation parameter to return the string stop.

Here is an example of a function to be passed to the callback_generation parameter which stops the execution if the fitness value 70 is reached. The value 70 might be the best possible fitness value. After being reached, then there is no need to pass through more generations because no further improvement is possible.

def func_generation(ga_instance):
 if ga_instance.best_solution()[1] >= 70:
     return "stop"

PyGAD 2.5.0¶

Release date: 19 July 2020

2 new optional parameters added to the constructor of the pygad.GA class which are crossover_probability and mutation_probability.

While applying the crossover operation, each parent has a random value generated between 0.0 and 1.0. If this random value is less than or equal to the value assigned to the crossover_probability parameter, then the parent is selected for the crossover operation.

For the mutation operation, a random value between 0.0 and 1.0 is generated for each gene in the solution. If this value is less than or equal to the value assigned to the mutation_probability, then this gene is selected for mutation.
A new optional parameter named linewidth is added to the plot_result() method to specify the width of the curve in the plot. It defaults to 3.0.
Previously, the indices of the genes selected for mutation was randomly generated once for all solutions within the generation. Currently, the genes’ indices are randomly generated for each solution in the population. If the population has 4 solutions, the indices are randomly generated 4 times inside the single generation, 1 time for each solution.
Previously, the position of the point(s) for the single-point and two-points crossover was(were) randomly selected once for all solutions within the generation. Currently, the position(s) is(are) randomly selected for each solution in the population. If the population has 4 solutions, the position(s) is(are) randomly generated 4 times inside the single generation, 1 time for each solution.
A new optional parameter named gene_space as added to the pygad.GA class constructor. It is used to specify the possible values for each gene in case the user wants to restrict the gene values. It is useful if the gene space is restricted to a certain range or to discrete values. For more information, check the More about the ``gene_space` Parameter <https://pygad.readthedocs.io/en/latest/pygad_more.html#more-about-the-gene-space-parameter>`__ section. Thanks to Prof. Tamer A. Farrag for requesting this useful feature.

PyGAD 2.6.0¶

Release Date: 6 August 2020

A bug fix in assigning the value to the initial_population parameter.
A new parameter named gene_type is added to control the gene type. It can be either int or float. It has an effect only when the parameter gene_space is None.
7 new parameters that accept callback functions: on_start, on_fitness, on_parents, on_crossover, on_mutation, on_generation, and on_stop.

PyGAD 2.7.0¶

Release Date: 11 September 2020

The learning_rate parameter in the pygad.nn.train() function defaults to 0.01.
Added support of building neural networks for regression using the new parameter named problem_type. It is added as a parameter to both pygad.nn.train() and pygad.nn.predict() functions. The value of this parameter can be either classification or regression to define the problem type. It defaults to classification.
The activation function for a layer can be set to the string "None" to refer that there is no activation function at this layer. As a result, the supported values for the activation function are "sigmoid", "relu", "softmax", and "None".

To build a regression network using the pygad.nn module, just do the following:

Set the problem_type parameter in the pygad.nn.train() and pygad.nn.predict() functions to the string "regression".
Set the activation function for the output layer to the string "None". This sets no limits on the range of the outputs as it will be from -infinity to +infinity. If you are sure that all outputs will be nonnegative values, then use the ReLU function.

Check the documentation of the pygad.nn module for an example that builds a neural network for regression. The regression example is also available at this GitHub project: https://github.com/ahmedfgad/NumPyANN

To build and train a regression network using the pygad.gann module, do the following:

Set the problem_type parameter in the pygad.nn.train() and pygad.nn.predict() functions to the string "regression".
Set the output_activation parameter in the constructor of the pygad.gann.GANN class to "None".

Check the documentation of the pygad.gann module for an example that builds and trains a neural network for regression. The regression example is also available at this GitHub project: https://github.com/ahmedfgad/NeuralGenetic

To build a classification network, either ignore the problem_type parameter or set it to "classification" (default value). In this case, the activation function of the last layer can be set to any type (e.g. softmax).

PyGAD 2.7.1¶

Release Date: 11 September 2020

A bug fix when the problem_type argument is set to regression.

PyGAD 2.7.2¶

Release Date: 14 September 2020

Bug fix to support building and training regression neural networks with multiple outputs.

PyGAD 2.8.0¶

Release Date: 20 September 2020

Support of a new module named kerasga so that the Keras models can be trained by the genetic algorithm using PyGAD.

PyGAD 2.8.1¶

Release Date: 3 October 2020

Bug fix in applying the crossover operation when the crossover_probability parameter is used. Thanks to Eng. Hamada Kassem, Research and Teaching Assistant, Construction Engineering and Management, Faculty of Engineering, Alexandria University, Egypt.

PyGAD 2.9.0¶

Release Date: 06 December 2020

The fitness values of the initial population are considered in the best_solutions_fitness attribute.
An optional parameter named save_best_solutions is added. It defaults to False. When it is True, then the best solution after each generation is saved into an attribute named best_solutions. If False, then no solutions are saved and the best_solutions attribute will be empty.
Scattered crossover is supported. To use it, assign the crossover_type parameter the value "scattered".
NumPy arrays are now supported by the gene_space parameter.
The following parameters (gene_type, crossover_probability, mutation_probability, delay_after_gen) can be assigned to a numeric value of any of these data types: int, float, numpy.int, numpy.int8, numpy.int16, numpy.int32, numpy.int64, numpy.float, numpy.float16, numpy.float32, or numpy.float64.

PyGAD 2.10.0¶

Release Date: 03 January 2021

Support of a new module pygad.torchga to train PyTorch models using PyGAD. Check its documentation.
Support of adaptive mutation where the mutation rate is determined by the fitness value of each solution. Read the Adaptive Mutation section for more details. Also, read this paper: Libelli, S. Marsili, and P. Alba. “Adaptive mutation in genetic algorithms.” Soft computing 4.2 (2000): 76-80.
Before the run() method completes or exits, the fitness value of the best solution in the current population is appended to the best_solution_fitness list attribute. Note that the fitness value of the best solution in the initial population is already saved at the beginning of the list. So, the fitness value of the best solution is saved before the genetic algorithm starts and after it ends.
When the parameter parent_selection_type is set to sss (steady-state selection), then a warning message is printed if the value of the keep_parents parameter is set to 0.
More validations to the user input parameters.
The default value of the mutation_percent_genes is set to the string "default" rather than the integer 10. This change helps to know whether the user explicitly passed a value to the mutation_percent_genes parameter or it is left to its default one. The "default" value is later translated into the integer 10.
The mutation_percent_genes parameter is no longer accepting the value 0. It must be >0 and <=100.
The built-in warnings module is used to show warning messages rather than just using the print() function.
A new bool parameter called suppress_warnings is added to the constructor of the pygad.GA class. It allows the user to control whether the warning messages are printed or not. It defaults to False which means the messages are printed.
A helper method called adaptive_mutation_population_fitness() is created to calculate the average fitness value used in adaptive mutation to filter the solutions.
The best_solution() method accepts a new optional parameter called pop_fitness. It accepts a list of the fitness values of the solutions in the population. If None, then the cal_pop_fitness() method is called to calculate the fitness values of the population.

PyGAD 2.10.1¶

Release Date: 10 January 2021

In the gene_space parameter, any None value (regardless of its index or axis), is replaced by a randomly generated number based on the 3 parameters init_range_low, init_range_high, and gene_type. So, the None value in [..., None, ...] or [..., [..., None, ...], ...] are replaced with random values. This gives more freedom in building the space of values for the genes.
All the numbers passed to the gene_space parameter are casted to the type specified in the gene_type parameter.
The numpy.uint data type is supported for the parameters that accept integer values.
In the pygad.kerasga module, the model_weights_as_vector() function uses the trainable attribute of the model’s layers to only return the trainable weights in the network. So, only the trainable layers with their trainable attribute set to True (trainable=True), which is the default value, have their weights evolved. All non-trainable layers with the trainable attribute set to False (trainable=False) will not be evolved. Thanks to Prof. Tamer A. Farrag for pointing about that at GitHub.

PyGAD 2.10.2¶

Release Date: 15 January 2021

A bug fix when save_best_solutions=True. Refer to this issue for more information: https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/25

PyGAD 2.11.0¶

Release Date: 16 February 2021

In the gene_space argument, the user can use a dictionary to specify the lower and upper limits of the gene. This dictionary must have only 2 items with keys low and high to specify the low and high limits of the gene, respectively. This way, PyGAD takes care of not exceeding the value limits of the gene. For a problem with only 2 genes, then using gene_space=[{'low': 1, 'high': 5}, {'low': 0.2, 'high': 0.81}] means the accepted values in the first gene start from 1 (inclusive) to 5 (exclusive) while the second one has values between 0.2 (inclusive) and 0.85 (exclusive). For more information, please check the Limit the Gene Value Range section of the documentation.
The plot_result() method returns the figure so that the user can save it.
Bug fixes in copying elements from the gene space.
For a gene with a set of discrete values (more than 1 value) in the gene_space parameter like [0, 1], it was possible that the gene value may not change after mutation. That is if the current value is 0, then the randomly selected value could also be 0. Now, it is verified that the new value is changed. So, if the current value is 0, then the new value after mutation will not be 0 but 1.

PyGAD 2.12.0¶

Release Date: 20 February 2021

4 new instance attributes are added to hold temporary results after each generation: last_generation_fitness holds the fitness values of the solutions in the last generation, last_generation_parents holds the parents selected from the last generation, last_generation_offspring_crossover holds the offspring generated after applying the crossover in the last generation, and last_generation_offspring_mutation holds the offspring generated after applying the mutation in the last generation. You can access these attributes inside the on_generation() method for example.
A bug fixed when the initial_population parameter is used. The bug occurred due to a mismatch between the data type of the array assigned to initial_population and the gene type in the gene_type attribute. Assuming that the array assigned to the initial_population parameter is ((1, 1), (3, 3), (5, 5), (7, 7)) which has type int. When gene_type is set to float, then the genes will not be float but casted to int because the defined array has int type. The bug is fixed by forcing the array assigned to initial_population to have the data type in the gene_type attribute. Check the issue at GitHub: https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/27

Thanks to Andrei Rozanski [PhD Bioinformatics Specialist, Department of Tissue Dynamics and Regeneration, Max Planck Institute for Biophysical Chemistry, Germany] for opening my eye to the first change.

Thanks to Marios Giouvanakis, a PhD candidate in Electrical & Computer Engineer, Aristotle University of Thessaloniki (Αριστοτέλειο Πανεπιστήμιο Θεσσαλονίκης), Greece, for emailing me about the second issue.

PyGAD 2.13.0¶

Release Date: 12 March 2021

A new bool parameter called allow_duplicate_genes is supported. If True, which is the default, then a solution/chromosome may have duplicate gene values. If False, then each gene will have a unique value in its solution. Check the Prevent Duplicates in Gene Values section for more details.
The last_generation_fitness is updated at the end of each generation not at the beginning. This keeps the fitness values of the most up-to-date population assigned to the last_generation_fitness parameter.

PyGAD 2.14.0¶

PyGAD 2.14.0 has an issue that is solved in PyGAD 2.14.1. Please consider using 2.14.1 not 2.14.0.

Release Date: 19 May 2021

Issue #40 is solved. Now, the None value works with the crossover_type and mutation_type parameters: https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/40
The gene_type parameter supports accepting a list/tuple/numpy.ndarray of numeric data types for the genes. This helps to control the data type of each individual gene. Previously, the gene_type can be assigned only to a single data type that is applied for all genes. For more information, check the More about the ``gene_type` Parameter <https://pygad.readthedocs.io/en/latest/pygad_more.html#more-about-the-gene-type-parameter>`__ section. Thanks to Rainer Engel for asking about this feature in this discussion: https://github.com/ahmedfgad/GeneticAlgorithmPython/discussions/43
A new bool attribute named gene_type_single is added to the pygad.GA class. It is True when there is a single data type assigned to the gene_type parameter. When the gene_type parameter is assigned a list/tuple/numpy.ndarray, then gene_type_single is set to False.
The mutation_by_replacement flag now has no effect if gene_space exists except for the genes with None values. For example, for gene_space=[None, [5, 6]] the mutation_by_replacement flag affects only the first gene which has None for its value space.
When an element has a value of None in the gene_space parameter (e.g. gene_space=[None, [5, 6]]), then its value will be randomly generated for each solution rather than being generate once for all solutions. Previously, the gene with None value in gene_space is the same across all solutions
Some changes in the documentation according to issue #32: https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/32

PyGAD 2.14.2¶

Release Date: 27 May 2021

Some bug fixes when the gene_type parameter is nested. Thanks to Rainer Engel for opening a discussion to report this bug: https://github.com/ahmedfgad/GeneticAlgorithmPython/discussions/43#discussioncomment-763342

Rainer Engel helped a lot in suggesting new features and suggesting enhancements in 2.14.0 to 2.14.2 releases.

PyGAD 2.14.3¶

Release Date: 6 June 2021

Some bug fixes when setting the save_best_solutions parameter to True. Previously, the best solution for generation i was added into the best_solutions attribute at generation i+1. Now, the best_solutions attribute is updated by each best solution at its exact generation.

PyGAD 2.15.0¶

Release Date: 17 June 2021

Control the precision of all genes/individual genes. Thanks to Rainer for asking about this feature: https://github.com/ahmedfgad/GeneticAlgorithmPython/discussions/43#discussioncomment-763452
A new attribute named last_generation_parents_indices holds the indices of the selected parents in the last generation.
In adaptive mutation, no need to recalculate the fitness values of the parents selected in the last generation as these values can be returned based on the last_generation_fitness and last_generation_parents_indices attributes. This speeds-up the adaptive mutation.
When a sublist has a value of None in the gene_space parameter (e.g. gene_space=[[1, 2, 3], [5, 6, None]]), then its value will be randomly generated for each solution rather than being generated once for all solutions. Previously, a value of None in a sublist of the gene_space parameter was identical across all solutions.
The dictionary assigned to the gene_space parameter itself or one of its elements has a new key called "step" to specify the step of moving from the start to the end of the range specified by the 2 existing keys "low" and "high". An example is {"low": 0, "high": 30, "step": 2} to have only even values for the gene(s) starting from 0 to 30. For more information, check the More about the ``gene_space` Parameter <https://pygad.readthedocs.io/en/latest/pygad_more.html#more-about-the-gene-space-parameter>`__ section. https://github.com/ahmedfgad/GeneticAlgorithmPython/discussions/48
A new function called predict() is added in both the pygad.kerasga and pygad.torchga modules to make predictions. This makes it easier than using custom code each time a prediction is to be made.
A new parameter called stop_criteria allows the user to specify one or more stop criteria to stop the evolution based on some conditions. Each criterion is passed as str which has a stop word. The current 2 supported words are reach and saturate. reach stops the run() method if the fitness value is equal to or greater than a given fitness value. An example for reach is "reach_40" which stops the evolution if the fitness is >= 40. saturate means stop the evolution if the fitness saturates for a given number of consecutive generations. An example for saturate is "saturate_7" which means stop the run() method if the fitness does not change for 7 consecutive generations. Thanks to Rainer for asking about this feature: https://github.com/ahmedfgad/GeneticAlgorithmPython/discussions/44
A new bool parameter, defaults to False, named save_solutions is added to the constructor of the pygad.GA class. If True, then all solutions in each generation are appended into an attribute called solutions which is NumPy array.
The plot_result() method is renamed to plot_fitness(). The users should migrate to the new name as the old name will be removed in the future.
Four new optional parameters are added to the plot_fitness() function in the pygad.GA class which are font_size=14, save_dir=None, color="#3870FF", and plot_type="plot". Use font_size to change the font of the plot title and labels. save_dir accepts the directory to which the figure is saved. It defaults to None which means do not save the figure. color changes the color of the plot. plot_type changes the plot type which can be either "plot" (default), "scatter", or "bar". https://github.com/ahmedfgad/GeneticAlgorithmPython/pull/47
The default value of the title parameter in the plot_fitness() method is "PyGAD - Generation vs. Fitness" rather than "PyGAD - Iteration vs. Fitness".
A new method named plot_new_solution_rate() creates, shows, and returns a figure showing the rate of new/unique solutions explored in each generation. It accepts the same parameters as in the plot_fitness() method. This method only works when save_solutions=True in the pygad.GA class’s constructor.
A new method named plot_genes() creates, shows, and returns a figure to show how each gene changes per each generation. It accepts similar parameters like the plot_fitness() method in addition to the graph_type, fill_color, and solutions parameters. The graph_type parameter can be either "plot" (default), "boxplot", or "histogram". fill_color accepts the fill color which works when graph_type is either "boxplot" or "histogram". solutions can be either "all" or "best" to decide whether all solutions or only best solutions are used.
The gene_type parameter now supports controlling the precision of float data types. For a gene, rather than assigning just the data type like float, assign a list/tuple/numpy.ndarray with 2 elements where the first one is the type and the second one is the precision. For example, [float, 2] forces a gene with a value like 0.1234 to be 0.12. For more information, check the More about the ``gene_type` Parameter <https://pygad.readthedocs.io/en/latest/pygad_more.html#more-about-the-gene-type-parameter>`__ section.

PyGAD 2.15.1¶

Release Date: 18 June 2021

Fix a bug when keep_parents is set to a positive integer. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/49

PyGAD 2.15.2¶

Release Date: 18 June 2021

Fix a bug when using the kerasga or torchga modules. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/51

PyGAD 2.16.0¶

Release Date: 19 June 2021

A user-defined function can be passed to the mutation_type, crossover_type, and parent_selection_type parameters in the pygad.GA class to create a custom mutation, crossover, and parent selection operators. Check the User-Defined Crossover, Mutation, and Parent Selection Operators section for more details. https://github.com/ahmedfgad/GeneticAlgorithmPython/discussions/50

PyGAD 2.16.1¶

Release Date: 28 September 2021

The user can use the tqdm library to show a progress bar. https://github.com/ahmedfgad/GeneticAlgorithmPython/discussions/50.

import pygad
import numpy
import tqdm

equation_inputs = [4,-2,3.5]
desired_output = 44

def fitness_func(ga_instance, solution, solution_idx):
    output = numpy.sum(solution * equation_inputs)
    fitness = 1.0 / (numpy.abs(output - desired_output) + 0.000001)
    return fitness

num_generations = 10000
with tqdm.tqdm(total=num_generations) as pbar:
    ga_instance = pygad.GA(num_generations=num_generations,
                           sol_per_pop=5,
                           num_parents_mating=2,
                           num_genes=len(equation_inputs),
                           fitness_func=fitness_func,
                           on_generation=lambda _: pbar.update(1))

    ga_instance.run()

ga_instance.plot_result()

But this work does not work if the ga_instance will be pickled (i.e. the save() method will be called.

ga_instance.save("test")

To solve this issue, define a function and pass it to the on_generation parameter. In the next code, the on_generation_progress() function is defined which updates the progress bar.

import pygad
import numpy
import tqdm

equation_inputs = [4,-2,3.5]
desired_output = 44

def fitness_func(ga_instance, solution, solution_idx):
    output = numpy.sum(solution * equation_inputs)
    fitness = 1.0 / (numpy.abs(output - desired_output) + 0.000001)
    return fitness

def on_generation_progress(ga):
    pbar.update(1)

num_generations = 100
with tqdm.tqdm(total=num_generations) as pbar:
    ga_instance = pygad.GA(num_generations=num_generations,
                           sol_per_pop=5,
                           num_parents_mating=2,
                           num_genes=len(equation_inputs),
                           fitness_func=fitness_func,
                           on_generation=on_generation_progress)

    ga_instance.run()

ga_instance.plot_result()

ga_instance.save("test")

Solved the issue of unequal length between the solutions and solutions_fitness when the save_solutions parameter is set to True. Now, the fitness of the last population is appended to the solutions_fitness array. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/64
There was an issue of getting the length of these 4 variables (solutions, solutions_fitness, best_solutions, and best_solutions_fitness) doubled after each call of the run() method. This is solved by resetting these variables at the beginning of the run() method. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/62
Bug fixes when adaptive mutation is used (mutation_type="adaptive"). https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/65

PyGAD 2.16.2¶

Release Date: 2 February 2022

A new instance attribute called previous_generation_fitness added in the pygad.GA class. It holds the fitness values of one generation before the fitness values saved in the last_generation_fitness.
Issue in the cal_pop_fitness() method in getting the correct indices of the previous parents. This is solved by using the previous generation’s fitness saved in the new attribute previous_generation_fitness to return the parents’ fitness values. Thanks to Tobias Tischhauser (M.Sc. - Mitarbeiter Institut EMS, Departement Technik, OST – Ostschweizer Fachhochschule, Switzerland) for detecting this bug.

PyGAD 2.16.3¶

Release Date: 2 February 2022

Validate the fitness value returned from the fitness function. An exception is raised if something is wrong. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/67

PyGAD 2.17.0¶

Release Date: 8 July 2022

An issue is solved when the gene_space parameter is given a fixed value. e.g. gene_space=[range(5), 4]. The second gene’s value is static (4) which causes an exception.
Fixed the issue where the allow_duplicate_genes parameter did not work when mutation is disabled (i.e. mutation_type=None). This is by checking for duplicates after crossover directly. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/39
Solve an issue in the tournament_selection() method as the indices of the selected parents were incorrect. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/89
Reuse the fitness values of the previously explored solutions rather than recalculating them. This feature only works if save_solutions=True.
Parallel processing is supported. This is by the introduction of a new parameter named parallel_processing in the constructor of the pygad.GA class. Thanks to @windowshopr for opening the issue #78 at GitHub. Check the Parallel Processing in PyGAD section for more information and examples.

PyGAD 2.18.0¶

Release Date: 9 September 2022

Raise an exception if the sum of fitness values is zero while either roulette wheel or stochastic universal parent selection is used. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/129
Initialize the value of the run_completed property to False. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/122
The values of these properties are no longer reset with each call to the run() method self.best_solutions, self.best_solutions_fitness, self.solutions, self.solutions_fitness: https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/123. Now, the user can have the flexibility of calling the run() method more than once while extending the data collected after each generation. Another advantage happens when the instance is loaded and the run() method is called, as the old fitness value are shown on the graph alongside with the new fitness values. Read more in this section: Continue without Losing Progress
Thanks Prof. Fernando Jiménez Barrionuevo (Dept. of Information and Communications Engineering, University of Murcia, Murcia, Spain) for editing this comment in the code. https://github.com/ahmedfgad/GeneticAlgorithmPython/commit/5315bbec02777df96ce1ec665c94dece81c440f4
A bug fixed when crossover_type=None.
Support of elitism selection through a new parameter named keep_elitism. It defaults to 1 which means for each generation keep only the best solution in the next generation. If assigned 0, then it has no effect. Read more in this section: Elitism Selection. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/74
A new instance attribute named last_generation_elitism added to hold the elitism in the last generation.
A new parameter called random_seed added to accept a seed for the random function generators. Credit to this issue https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/70 and Prof. Fernando Jiménez Barrionuevo. Read more in this section: Random Seed.
Editing the pygad.TorchGA module to make sure the tensor data is moved from GPU to CPU. Thanks to Rasmus Johansson for opening this pull request: https://github.com/ahmedfgad/TorchGA/pull/2

PyGAD 2.18.1¶

Release Date: 19 September 2022

A big fix when keep_elitism is used. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/132

PyGAD 2.18.2¶

Release Date: 14 February 2023

Remove numpy.int and numpy.float from the list of supported data types. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/151 https://github.com/ahmedfgad/GeneticAlgorithmPython/pull/152
Call the on_crossover() callback function even if crossover_type is None. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/138
Call the on_mutation() callback function even if mutation_type is None. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/138

PyGAD 2.18.3¶

Release Date: 14 February 2023

Bug fixes.

PyGAD 2.19.0¶

Release Date: 22 February 2023

A new summary() method is supported to return a Keras-like summary of the PyGAD lifecycle.
A new optional parameter called fitness_batch_size is supported to calculate the fitness in batches. If it is assigned the value 1 or None (default), then the normal flow is used where the fitness function is called for each individual solution. If the fitness_batch_size parameter is assigned a value satisfying this condition 1 < fitness_batch_size <= sol_per_pop, then the solutions are grouped into batches of size fitness_batch_size and the fitness function is called once for each batch. In this case, the fitness function must return a list/tuple/numpy.ndarray with a length equal to the number of solutions passed. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/136.
The cloudpickle library (https://github.com/cloudpipe/cloudpickle) is used instead of the pickle library to pickle the pygad.GA objects. This solves the issue of having to redefine the functions (e.g. fitness function). The cloudpickle library is added as a dependency in the requirements.txt file. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/159
Support of assigning methods to these parameters: fitness_func, crossover_type, mutation_type, parent_selection_type, on_start, on_fitness, on_parents, on_crossover, on_mutation, on_generation, and on_stop. https://github.com/ahmedfgad/GeneticAlgorithmPython/pull/92 https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/138
Validating the output of the parent selection, crossover, and mutation functions.
The built-in parent selection operators return the parent’s indices as a NumPy array.
The outputs of the parent selection, crossover, and mutation operators must be NumPy arrays.
Fix an issue when allow_duplicate_genes=True. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/39
Fix an issue creating scatter plots of the solutions’ fitness.
Sampling from a set() is no longer supported in Python 3.11. Instead, sampling happens from a list(). Thanks Marco Brenna for pointing to this issue.
The lifecycle is updated to reflect that the new population’s fitness is calculated at the end of the lifecycle not at the beginning. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/154#issuecomment-1438739483
There was an issue when save_solutions=True that causes the fitness function to be called for solutions already explored and have their fitness pre-calculated. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/160
A new instance attribute named last_generation_elitism_indices added to hold the indices of the selected elitism. This attribute helps to re-use the fitness of the elitism instead of calling the fitness function.
Fewer calls to the best_solution() method which in turns saves some calls to the fitness function.
Some updates in the documentation to give more details about the cal_pop_fitness() method. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/79#issuecomment-1439605442

PyGAD 2.19.1¶

Release Date: 22 February 2023

Add the cloudpickle library as a dependency.

PyGAD 2.19.2¶

Release Date 23 February 2023

Fix an issue when parallel processing was used where the elitism solutions’ fitness values are not re-used. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/160#issuecomment-1441718184

PyGAD 3.0.0¶

Release Date 8 April 2023

The structure of the library is changed and some methods defined in the pygad.py module are moved to the pygad.utils, pygad.helper, and pygad.visualize submodules.
The pygad.utils.parent_selection module has a class named ParentSelection where all the parent selection operators exist. The pygad.GA class extends this class.
The pygad.utils.crossover module has a class named Crossover where all the crossover operators exist. The pygad.GA class extends this class.
The pygad.utils.mutation module has a class named Mutation where all the mutation operators exist. The pygad.GA class extends this class.
The pygad.helper.unique module has a class named Unique some helper methods exist to solve duplicate genes and make sure every gene is unique. The pygad.GA class extends this class.
The pygad.visualize.plot module has a class named Plot where all the methods that create plots exist. The pygad.GA class extends this class.
Support of using the logging module to log the outputs to both the console and text file instead of using the print() function. This is by assigning the logging.Logger to the new logger parameter. Check the Logging Outputs for more information.
A new instance attribute called logger to save the logger.
The function/method passed to the fitness_func parameter accepts a new parameter that refers to the instance of the pygad.GA class. Check this for an example: Use Functions and Methods to Build Fitness Function and Callbacks. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/163
Update the documentation to include an example of using functions and methods to calculate the fitness and build callbacks. Check this for more details: Use Functions and Methods to Build Fitness Function and Callbacks. https://github.com/ahmedfgad/GeneticAlgorithmPython/pull/92#issuecomment-1443635003
Validate the value passed to the initial_population parameter.
Validate the type and length of the pop_fitness parameter of the best_solution() method.
Some edits in the documentation. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/106
Fix an issue when building the initial population as (some) genes have their value taken from the mutation range (defined by the parameters random_mutation_min_val and random_mutation_max_val) instead of using the parameters init_range_low and init_range_high.
The summary() method returns the summary as a single-line string. Just log/print the returned string it to see it properly.
The callback_generation parameter is removed. Use the on_generation parameter instead.
There was an issue when using the parallel_processing parameter with Keras and PyTorch. As Keras/PyTorch are not thread-safe, the predict() method gives incorrect and weird results when more than 1 thread is used. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/145 https://github.com/ahmedfgad/TorchGA/issues/5 https://github.com/ahmedfgad/KerasGA/issues/6. Thanks to this StackOverflow answer.
Replace numpy.float by float in the 2 parent selection operators roulette wheel and stochastic universal. https://github.com/ahmedfgad/GeneticAlgorithmPython/pull/168

PyGAD 3.0.1¶

Release Date 20 April 2023

Fix an issue with passing user-defined function/method for parent selection. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/179

PyGAD 3.1.0¶

Release Date 20 June 2023

Fix a bug when the initial population has duplciate genes if a nested gene space is used.
The gene_space parameter can no longer be assigned a tuple.
Fix a bug when the gene_space parameter has a member of type tuple.
A new instance attribute called gene_space_unpacked which has the unpacked gene_space. It is used to solve duplicates. For infinite ranges in the gene_space, they are unpacked to a limited number of values (e.g. 100).
Bug fixes when creating the initial population using gene_space attribute.
When a dict is used with the gene_space attribute, the new gene value was calculated by summing 2 values: 1) the value sampled from the dict 2) a random value returned from the random mutation range defined by the 2 parameters random_mutation_min_val and random_mutation_max_val. This might cause the gene value to exceed the range limit defined in the gene_space. To respect the gene_space range, this release only returns the value from the dict without summing it to a random value.
Formatting the strings using f-string instead of the format() method. https://github.com/ahmedfgad/GeneticAlgorithmPython/pull/189
In the __init__() of the pygad.GA class, the logged error messages are handled using a try-except block instead of repeating the logger.error() command. https://github.com/ahmedfgad/GeneticAlgorithmPython/pull/189
A new class named CustomLogger is created in the pygad.cnn module to create a default logger using the logging module assigned to the logger attribute. This class is extended in all other classes in the module. The constructors of these classes have a new parameter named logger which defaults to None. If no logger is passed, then the default logger in the CustomLogger class is used.
Except for the pygad.nn module, the print() function in all other modules are replaced by the logging module to log messages.
The callback functions/methods on_fitness(), on_parents(), on_crossover(), and on_mutation() can return values. These returned values override the corresponding properties. The output of on_fitness() overrides the population fitness. The on_parents() function/method must return 2 values representing the parents and their indices. The output of on_crossover() overrides the crossover offspring. The output of on_mutation() overrides the mutation offspring.
Fix a bug when adaptive mutation is used while fitness_batch_size>1. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/195
When allow_duplicate_genes=False and a user-defined gene_space is used, it sometimes happen that there is no room to solve the duplicates between the 2 genes by simply replacing the value of one gene by another gene. This release tries to solve such duplicates by looking for a third gene that will help in solving the duplicates. Check this section for more information.
Use probabilities to select parents using the rank parent selection method. https://github.com/ahmedfgad/GeneticAlgorithmPython/discussions/205
The 2 parameters random_mutation_min_val and random_mutation_max_val can accept iterables (list/tuple/numpy.ndarray) with length equal to the number of genes. This enables customizing the mutation range for each individual gene. https://github.com/ahmedfgad/GeneticAlgorithmPython/discussions/198
The 2 parameters init_range_low and init_range_high can accept iterables (list/tuple/numpy.ndarray) with length equal to the number of genes. This enables customizing the initial range for each individual gene when creating the initial population.
The data parameter in the predict() function of the pygad.kerasga module can be assigned a data generator. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/115 https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/207
The predict() function of the pygad.kerasga module accepts 3 optional parameters: 1) batch_size=None, verbose=0, and steps=None. Check documentation of the Keras Model.predict() method for more information. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/207
The documentation is updated to explain how mutation works when gene_space is used with int or float data types. Check this section. https://github.com/ahmedfgad/GeneticAlgorithmPython/discussions/198

PyGAD 3.2.0¶

Release Date 7 September 2023

A new module pygad.utils.nsga2 is created that has the NSGA2 class that includes the functionalities of NSGA-II. The class has these methods: 1) get_non_dominated_set() 2) non_dominated_sorting() 3) crowding_distance() 4) sort_solutions_nsga2(). Check this section for an example.
Support of multi-objective optimization using Non-Dominated Sorting Genetic Algorithm II (NSGA-II) using the NSGA2 class in the pygad.utils.nsga2 module. Just return a list, tuple, or numpy.ndarray from the fitness function and the library will consider the problem as multi-objective optimization. All the objectives are expected to be maximization. Check this section for an example.
The parent selection methods and adaptive mutation are edited to support multi-objective optimization.
Two new NSGA-II parent selection methods are supported in the pygad.utils.parent_selection module: 1) Tournament selection for NSGA-II 2) NSGA-II selection.
The plot_fitness() method in the pygad.plot module has a new optional parameter named label to accept the label of the plots. This is only used for multi-objective problems. Otherwise, it is ignored. It defaults to None and accepts a list, tuple, or numpy.ndarray. The labels are used in a legend inside the plot.
The default color in the methods of the pygad.plot module is changed to the greenish #64f20c color.
A new instance attribute named pareto_fronts added to the pygad.GA instances that holds the pareto fronts when solving a multi-objective problem.
The gene_type accepts a list, tuple, or numpy.ndarray for integer data types given that the precision is set to None (e.g. gene_type=[float, [int, None]]).
In the cal_pop_fitness() method, the fitness value is re-used if save_best_solutions=True and the solution is found in the best_solutions attribute. These parameters also can help re-using the fitness of a solution instead of calling the fitness function: keep_elitism, keep_parents, and save_solutions.
The value 99999999999 is replaced by float('inf') in the 2 methods wheel_cumulative_probs() and stochastic_universal_selection() inside the pygad.utils.parent_selection.ParentSelection class.
The plot_result() method in the pygad.visualize.plot.Plot class is removed. Instead, please use the plot_fitness() if you did not upgrade yet.

PyGAD 3.3.0¶

Release Date 29 January 2024

Solve bugs when multi-objective optimization is used. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/238
When the stop_ciiteria parameter is used with the reach keyword, then multiple numeric values can be passed when solving a multi-objective problem. For example, if a problem has 3 objective functions, then stop_criteria="reach_10_20_30" means the GA stops if the fitness of the 3 objectives are at least 10, 20, and 30, respectively. The number values must match the number of objective functions. If a single value found (e.g. stop_criteria=reach_5) when solving a multi-objective problem, then it is used across all the objectives. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/238
The delay_after_gen parameter is now deprecated and will be removed in a future release. If it is necessary to have a time delay after each generation, then assign a callback function/method to the on_generation parameter to pause the evolution.
Parallel processing now supports calculating the fitness during adaptive mutation. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/201
The population size can be changed during runtime by changing all the parameters that would affect the size of any thing used by the GA. For more information, check the Change Population Size during Runtime section. https://github.com/ahmedfgad/GeneticAlgorithmPython/discussions/234
When a dictionary exists in the gene_space parameter without a step, then mutation occurs by adding a random value to the gene value. The random vaue is generated based on the 2 parameters random_mutation_min_val and random_mutation_max_val. For more information, check the How Mutation Works with the gene_space Parameter? section. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/229
Add object as a supported data type for int (GA.supported_int_types) and float (GA.supported_float_types). https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/174
Use the raise clause instead of the sys.exit(-1) to terminate the execution. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/213
Fix a bug when multi-objective optimization is used with batch fitness calculation (e.g. fitness_batch_size set to a non-zero number).
Fix a bug in the pygad.py script when finding the index of the best solution. It does not work properly with multi-objective optimization where self.best_solutions_fitness have multiple columns.

self.best_solution_generation = numpy.where(numpy.array(
    self.best_solutions_fitness) == numpy.max(numpy.array(self.best_solutions_fitness)))[0][0]

PyGAD 3.3.1¶

Release Date 17 February 2024

After the last generation and before the run() method completes, update the 2 instance attributes: 1) last_generation_parents 2) last_generation_parents_indices. This is to keep the list of parents up-to-date with the latest population fitness last_generation_fitness. https://github.com/ahmedfgad/GeneticAlgorithmPython/issues/275
4 methods with names starting with run_. Their purpose is to keep the main loop inside the run() method clean. Check the Other Methods section for more information.

PyGAD Projects at GitHub¶

The PyGAD library is available at PyPI at this page https://pypi.org/project/pygad. PyGAD is built out of a number of open-source GitHub projects. A brief note about these projects is given in the next subsections.

GeneticAlgorithmPython ¶

GitHub Link: https://github.com/ahmedfgad/GeneticAlgorithmPython

GeneticAlgorithmPython is the first project which is an open-source Python 3 project for implementing the genetic algorithm based on NumPy.

NumPyANN ¶

GitHub Link: https://github.com/ahmedfgad/NumPyANN

NumPyANN builds artificial neural networks in Python 3 using NumPy from scratch. The purpose of this project is to only implement the forward pass of a neural network without using a training algorithm. Currently, it only supports classification and later regression will be also supported. Moreover, only one class is supported per sample.

NeuralGenetic ¶

GitHub Link: https://github.com/ahmedfgad/NeuralGenetic

NeuralGenetic trains neural networks using the genetic algorithm based on the previous 2 projects GeneticAlgorithmPython and NumPyANN.

NumPyCNN ¶

GitHub Link: https://github.com/ahmedfgad/NumPyCNN

NumPyCNN builds convolutional neural networks using NumPy. The purpose of this project is to only implement the forward pass of a convolutional neural network without using a training algorithm.

CNNGenetic ¶

GitHub Link: https://github.com/ahmedfgad/CNNGenetic

CNNGenetic trains convolutional neural networks using the genetic algorithm. It uses the GeneticAlgorithmPython project for building the genetic algorithm.

KerasGA ¶

GitHub Link: https://github.com/ahmedfgad/KerasGA

KerasGA trains Keras models using the genetic algorithm. It uses the GeneticAlgorithmPython project for building the genetic algorithm.

TorchGA ¶

GitHub Link: https://github.com/ahmedfgad/TorchGA

TorchGA trains PyTorch models using the genetic algorithm. It uses the GeneticAlgorithmPython project for building the genetic algorithm.

pygad.torchga: https://github.com/ahmedfgad/TorchGA

Stackoverflow Questions about PyGAD¶

How do I proceed to load a ga_instance as “.pkl” format in PyGad?¶

Binary Classification NN Model Weights not being Trained in PyGAD ¶

How to solve TSP problem using pyGAD package?¶

How can I save a matplotlib plot that is the output of a function in jupyter?¶

How do I query the best solution of a pyGAD GA instance?¶

Multi-Input Multi-Output in Genetic algorithm (python)¶

https://www.linkedin.com/pulse/validation-short-term-parametric-trading-model-genetic-landolfi

https://itchef.ru/articles/397758

https://audhiaprilliant.medium.com/genetic-algorithm-based-clustering-algorithm-in-searching-robust-initial-centroids-for-k-means-e3b4d892a4be

https://python.plainenglish.io/validation-of-a-short-term-parametric-trading-model-with-genetic-optimization-and-walk-forward-89708b789af6

https://ichi.pro/ko/pygadwa-hamkke-yujeon-algolijeum-eul-sayonghayeo-keras-model-eul-hunlyeonsikineun-bangbeob-173299286377169

https://ichi.pro/tr/pygad-ile-genetik-algoritmayi-kullanarak-keras-modelleri-nasil-egitilir-173299286377169

https://ichi.pro/ru/kak-obucit-modeli-keras-s-pomos-u-geneticeskogo-algoritma-s-pygad-173299286377169

https://blog.csdn.net/sinat_38079265/article/details/108449614

Submitting Issues¶

If there is an issue using PyGAD, then use any of your preferred option to discuss that issue.

One way is submitting an issue into this GitHub project (github.com/ahmedfgad/GeneticAlgorithmPython) in case something is not working properly or to ask for questions.

If this is not a proper option for you, then check the Contact Us section for more contact details.

Ask for Feature¶

PyGAD is actively developed with the goal of building a dynamic library for suporting a wide-range of problems to be optimized using the genetic algorithm.

To ask for a new feature, either submit an issue into this GitHub project (github.com/ahmedfgad/GeneticAlgorithmPython) or send an e-mail to ahmed.f.gad@gmail.com.

Also check the Contact Us section for more contact details.

Projects Built using PyGAD¶

If you created a project that uses PyGAD, then we can support you by mentioning this project here in PyGAD’s documentation.

To do that, please send a message at ahmed.f.gad@gmail.com or check the Contact Us section for more contact details.

Within your message, please send the following details:

Project title
Brief description
Preferably, a link that directs the readers to your project

Tutorials about PyGAD¶

Adaptive Mutation in Genetic Algorithm with Python Examples ¶

In this tutorial, we’ll see why mutation with a fixed number of genes is bad, and how to replace it with adaptive mutation. Using the PyGAD Python 3 library, we’ll discuss a few examples that use both random and adaptive mutation.

Clustering Using the Genetic Algorithm in Python ¶

This tutorial discusses how the genetic algorithm is used to cluster data, starting from random clusters and running until the optimal clusters are found. We’ll start by briefly revising the K-means clustering algorithm to point out its weak points, which are later solved by the genetic algorithm. The code examples in this tutorial are implemented in Python using the PyGAD library.

Working with Different Genetic Algorithm Representations in Python ¶

Depending on the nature of the problem being optimized, the genetic algorithm (GA) supports two different gene representations: binary, and decimal. The binary GA has only two values for its genes, which are 0 and 1. This is easier to manage as its gene values are limited compared to the decimal GA, for which we can use different formats like float or integer, and limited or unlimited ranges.

This tutorial discusses how the PyGAD library supports the two GA representations, binary and decimal.

5 Genetic Algorithm Applications Using PyGAD ¶

This tutorial introduces PyGAD, an open-source Python library for implementing the genetic algorithm and training machine learning algorithms. PyGAD supports 19 parameters for customizing the genetic algorithm for various applications.

Within this tutorial we’ll discuss 5 different applications of the genetic algorithm and build them using PyGAD.

Train Neural Networks Using a Genetic Algorithm in Python with PyGAD ¶

The genetic algorithm (GA) is a biologically-inspired optimization algorithm. It has in recent years gained importance, as it’s simple while also solving complex problems like travel route optimization, training machine learning algorithms, working with single and multi-objective problems, game playing, and more.

Deep neural networks are inspired by the idea of how the biological brain works. It’s a universal function approximator, which is capable of simulating any function, and is now used to solve the most complex problems in machine learning. What’s more, they’re able to work with all types of data (images, audio, video, and text).

Both genetic algorithms (GAs) and neural networks (NNs) are similar, as both are biologically-inspired techniques. This similarity motivates us to create a hybrid of both to see whether a GA can train NNs with high accuracy.

This tutorial uses PyGAD, a Python library that supports building and training NNs using a GA. PyGAD offers both classification and regression NNs.

Building a Game-Playing Agent for CoinTex Using the Genetic Algorithm ¶

In this tutorial we’ll see how to build a game-playing agent using only the genetic algorithm to play a game called CoinTex, which is developed in the Kivy Python framework. The objective of CoinTex is to collect the randomly distributed coins while avoiding collision with fire and monsters (that move randomly). The source code of CoinTex can be found on GitHub.

The genetic algorithm is the only AI used here; there is no other machine/deep learning model used with it. We’ll implement the genetic algorithm using PyGad. This tutorial starts with a quick overview of CoinTex followed by a brief explanation of the genetic algorithm, and how it can be used to create the playing agent. Finally, we’ll see how to implement these ideas in Python.

The source code of the genetic algorithm agent is available here, and you can download the code used in this tutorial from here.

How To Train Keras Models Using the Genetic Algorithm with PyGAD ¶

PyGAD is an open-source Python library for building the genetic algorithm and training machine learning algorithms. It offers a wide range of parameters to customize the genetic algorithm to work with different types of problems.

PyGAD has its own modules that support building and training neural networks (NNs) and convolutional neural networks (CNNs). Despite these modules working well, they are implemented in Python without any additional optimization measures. This leads to comparatively high computational times for even simple problems.

The latest PyGAD version, 2.8.0 (released on 20 September 2020), supports a new module to train Keras models. Even though Keras is built in Python, it’s fast. The reason is that Keras uses TensorFlow as a backend, and TensorFlow is highly optimized.

This tutorial discusses how to train Keras models using PyGAD. The discussion includes building Keras models using either the Sequential Model or the Functional API, building an initial population of Keras model parameters, creating an appropriate fitness function, and more.

Train PyTorch Models Using Genetic Algorithm with PyGAD ¶

PyGAD is a genetic algorithm Python 3 library for solving optimization problems. One of these problems is training machine learning algorithms.

PyGAD has a module called pygad.kerasga. It trains Keras models using the genetic algorithm. On January 3rd, 2021, a new release of PyGAD 2.10.0 brought a new module called pygad.torchga to train PyTorch models. It’s very easy to use, but there are a few tricky steps.

So, in this tutorial, we’ll explore how to use PyGAD to train PyTorch models.

A Guide to Genetic ‘Learning’ Algorithms for Optimization ¶

PyGAD in Other Languages¶

French¶

Cómo los algoritmos genéticos pueden competir con el descenso de gradiente y el backprop

Bien que la manière standard d’entraîner les réseaux de neurones soit la descente de gradient et la rétropropagation, il y a d’autres joueurs dans le jeu. L’un d’eux est les algorithmes évolutionnaires, tels que les algorithmes génétiques.

Utiliser un algorithme génétique pour former un réseau de neurones simple pour résoudre le OpenAI CartPole Jeu. Dans cet article, nous allons former un simple réseau de neurones pour résoudre le OpenAI CartPole . J’utiliserai PyTorch et PyGAD .

Spanish¶

Cómo los algoritmos genéticos pueden competir con el descenso de gradiente y el backprop

Aunque la forma estandar de entrenar redes neuronales es el descenso de gradiente y la retropropagacion, hay otros jugadores en el juego, uno de ellos son los algoritmos evolutivos, como los algoritmos geneticos.

Usa un algoritmo genetico para entrenar una red neuronal simple para resolver el Juego OpenAI CartPole. En este articulo, entrenaremos una red neuronal simple para resolver el OpenAI CartPole . Usare PyTorch y PyGAD .

Korean¶

[PyGAD] Python 에서 Genetic Algorithm 을 사용해보기 ¶

파이썬에서 genetic algorithm을 사용하는 패키지들을 다 사용해보진 않았지만, 확장성이 있어보이고, 시도할 일이 있어서 살펴봤다.

이 패키지에서 가장 인상 깊었던 것은 neural network에서 hyper parameter 탐색을 gradient descent 방식이 아닌 GA로도 할 수 있다는 것이다.

개인적으로 이 부분이 어느정도 초기치를 잘 잡아줄 수 있는 역할로도 쓸 수 있고, Loss가 gradient descent 하기 어려운 구조에서 대안으로 쓸 수 있을 것으로도 생각된다.

일단 큰 흐름은 다음과 같이 된다.

사실 완전히 흐름이나 각 parameter에 대한 이해는 부족한 상황

Turkish¶

PyGAD ile Genetik Algoritmayı Kullanarak Keras Modelleri Nasıl Eğitilir ¶

This is a translation of an original English tutorial published at Paperspace: How To Train Keras Models Using the Genetic Algorithm with PyGAD

PyGAD, genetik algoritma oluşturmak ve makine öğrenimi algoritmalarını eğitmek için kullanılan açık kaynaklı bir Python kitaplığıdır. Genetik algoritmayı farklı problem türleri ile çalışacak şekilde özelleştirmek için çok çeşitli parametreler sunar.

PyGAD, sinir ağları (NN’ler) ve evrişimli sinir ağları (CNN’ler) oluşturmayı ve eğitmeyi destekleyen kendi modüllerine sahiptir. Bu modüllerin iyi çalışmasına rağmen, herhangi bir ek optimizasyon önlemi olmaksızın Python’da uygulanırlar. Bu, basit problemler için bile nispeten yüksek hesaplama sürelerine yol açar.

En son PyGAD sürümü 2.8.0 (20 Eylül 2020’de piyasaya sürüldü), Keras modellerini eğitmek için yeni bir modülü destekliyor. Keras Python’da oluşturulmuş olsa da hızlıdır. Bunun nedeni, Keras’ın arka uç olarak TensorFlow kullanması ve TensorFlow’un oldukça optimize edilmiş olmasıdır.

Bu öğreticide, PyGAD kullanılarak Keras modellerinin nasıl eğitileceği anlatılmaktadır. Tartışma, Sıralı Modeli veya İşlevsel API’yi kullanarak Keras modellerini oluşturmayı, Keras model parametrelerinin ilk popülasyonunu oluşturmayı, uygun bir uygunluk işlevi oluşturmayı ve daha fazlasını içerir.

Hungarian¶

Tensorflow alapozó 10. Neurális hálózatok tenyésztése genetikus algoritmussal PyGAD és OpenAI Gym használatával ¶

Hogy kontextusba helyezzem a genetikus algoritmusokat, ismételjük kicsit át, hogy hogyan működik a gradient descent és a backpropagation, ami a neurális hálók tanításának általános módszere. Az erről írt cikkemet itt tudjátok elolvasni.

A hálózatok tenyésztéséhez a PyGAD nevű programkönyvtárat használjuk, így mindenek előtt ezt kell telepítenünk, valamint a Tensorflow-t és a Gym-et, amit Colabban már eleve telepítve kapunk.

Maga a PyGAD egy teljesen általános genetikus algoritmusok futtatására képes rendszer. Ennek a kiterjesztése a KerasGA, ami az általános motor Tensorflow (Keras) neurális hálókon történő futtatását segíti. A 47. sorban létrehozott KerasGA objektum ennek a kiterjesztésnek a része és arra szolgál, hogy a paraméterként átadott modellből a második paraméterben megadott számosságú populációt hozzon létre. Mivel a hálózatunk 386 állítható paraméterrel rendelkezik, ezért a DNS-ünk itt 386 elemből fog állni. A populáció mérete 10 egyed, így a kezdő populációnk egy 10x386 elemű mátrix lesz. Ezt adjuk át az 51. sorban az initial_population paraméterben.

Russian¶

PyGAD: библиотека для имплементации генетического алгоритма ¶

PyGAD — это библиотека для имплементации генетического алгоритма. Кроме того, библиотека предоставляет доступ к оптимизированным реализациям алгоритмов машинного обучения. PyGAD разрабатывали на Python 3.

Библиотека PyGAD поддерживает разные типы скрещивания, мутации и селекции родителя. PyGAD позволяет оптимизировать проблемы с помощью генетического алгоритма через кастомизацию целевой функции.

Кроме генетического алгоритма, библиотека содержит оптимизированные имплементации алгоритмов машинного обучения. На текущий момент PyGAD поддерживает создание и обучение нейросетей для задач классификации.

Библиотека находится в стадии активной разработки. Создатели планируют добавление функционала для решения бинарных задач и имплементации новых алгоритмов.

PyGAD разрабатывали на Python 3.7.3. Зависимости включают в себя NumPy для создания и манипуляции массивами и Matplotlib для визуализации. Один из изкейсов использования инструмента — оптимизация весов, которые удовлетворяют заданной функции.

Research Papers using PyGAD¶

A number of research papers used PyGAD and here are some of them:

Alberto Meola, Manuel Winkler, Sören Weinrich, Metaheuristic optimization of data preparation and machine learning hyperparameters for prediction of dynamic methane production, Bioresource Technology, Volume 372, 2023, 128604, ISSN 0960-8524.
Jaros, Marta, and Jiri Jaros. “Performance-Cost Optimization of Moldable Scientific Workflows.”
Thorat, Divya. “Enhanced genetic algorithm to reduce makespan of multiple jobs in map-reduce application on serverless platform”. Diss. Dublin, National College of Ireland, 2020.
Koch, Chris, and Edgar Dobriban. “AttenGen: Generating Live Attenuated Vaccine Candidates using Machine Learning.” (2021).
Bhardwaj, Bhavya, et al. “Windfarm optimization using Nelder-Mead and Particle Swarm optimization.” 2021 7th International Conference on Electrical Energy Systems (ICEES). IEEE, 2021.
Bernardo, Reginald Christian S. and J. Said. “Towards a model-independent reconstruction approach for late-time Hubble data.” (2021).
Duong, Tri Dung, Qian Li, and Guandong Xu. “Prototype-based Counterfactual Explanation for Causal Classification.” arXiv preprint arXiv:2105.00703 (2021).
Farrag, Tamer Ahmed, and Ehab E. Elattar. “Optimized Deep Stacked Long Short-Term Memory Network for Long-Term Load Forecasting.” IEEE Access 9 (2021): 68511-68522.
Antunes, E. D. O., Caetano, M. F., Marotta, M. A., Araujo, A., Bondan, L., Meneguette, R. I., & Rocha Filho, G. P. (2021, August). Soluções Otimizadas para o Problema de Localização de Máxima Cobertura em Redes Militarizadas 4G/LTE. In Anais do XXVI Workshop de Gerência e Operação de Redes e Serviços (pp. 152-165). SBC.
M. Yani, F. Ardilla, A. A. Saputra and N. Kubota, “Gradient-Free Deep Q-Networks Reinforcement learning: Benchmark and Evaluation,” 2021 IEEE Symposium Series on Computational Intelligence (SSCI), 2021, pp. 1-5, doi: 10.1109/SSCI50451.2021.9659941.
Yani, Mohamad, and Naoyuki Kubota. “Deep Convolutional Networks with Genetic Algorithm for Reinforcement Learning Problem.”
Mahendra, Muhammad Ihza, and Isman Kurniawan. “Optimizing Convolutional Neural Network by Using Genetic Algorithm for COVID-19 Detection in Chest X-Ray Image.” 2021 International Conference on Data Science and Its Applications (ICoDSA). IEEE, 2021.
Glibota, Vjeko. Umjeravanje mikroskopskog prometnog modela primjenom genetskog algoritma. Diss. University of Zagreb. Faculty of Transport and Traffic Sciences. Division of Intelligent Transport Systems and Logistics. Department of Intelligent Transport Systems, 2021.
Zhu, Mingda. Genetic Algorithm-based Parameter Identification for Ship Manoeuvring Model under Wind Disturbance. MS thesis. NTNU, 2021.
Abdalrahman, Ahmed, and Weihua Zhuang. “Dynamic pricing for differentiated pev charging services using deep reinforcement learning.” IEEE Transactions on Intelligent Transportation Systems (2020).

For More Information¶

There are different resources that can be used to get started with the genetic algorithm and building it in Python.

Tutorial: Implementing Genetic Algorithm in Python¶

To start with coding the genetic algorithm, you can check the tutorial titled Genetic Algorithm Implementation in Python available at these links:

This tutorial is prepared based on a previous version of the project but it still a good resource to start with coding the genetic algorithm.

Tutorial: Introduction to Genetic Algorithm¶

Get started with the genetic algorithm by reading the tutorial titled Introduction to Optimization with Genetic Algorithm which is available at these links:

Tutorial: Build Neural Networks in Python¶

Read about building neural networks in Python through the tutorial titled Artificial Neural Network Implementation using NumPy and Classification of the Fruits360 Image Dataset available at these links:

Tutorial: Optimize Neural Networks with Genetic Algorithm¶

Read about training neural networks using the genetic algorithm through the tutorial titled Artificial Neural Networks Optimization using Genetic Algorithm with Python available at these links:

Tutorial: Building CNN in Python¶

To start with coding the genetic algorithm, you can check the tutorial titled Building Convolutional Neural Network using NumPy from Scratch available at these links:

This tutorial) is prepared based on a previous version of the project but it still a good resource to start with coding CNNs.

Tutorial: Derivation of CNN from FCNN¶

Get started with the genetic algorithm by reading the tutorial titled Derivation of Convolutional Neural Network from Fully Connected Network Step-By-Step which is available at these links:

Book: Practical Computer Vision Applications Using Deep Learning with CNNs¶

You can also check my book cited as Ahmed Fawzy Gad ‘Practical Computer Vision Applications Using Deep Learning with CNNs’. Dec. 2018, Apress, 978-1-4842-4167-7 which discusses neural networks, convolutional neural networks, deep learning, genetic algorithm, and more.

Find the book at these links:

Contact Us¶

Thank you for using PyGAD :)