Same code in Python

[Sandy4321]

Great code thanks
But do you know similar approach coded in python

[ojdo]

Well, scipy's optimize.minimize gives you over 80% of this code in one line. All that you need to implement around is the probabilistic restart functionality and "select the best solution" scaffold around the giant for iRestart=1:options.maxRestarts Nelder-Mead block.

[Sandy4321]

May you clarify what scaffold is

[ojdo]

This I meant by scaffold (mainly the if current < best part):

from math import inf
from scipy import optimize

best_solution = [0, 0, 0]  # actually n-dimensional
best_objective = inf

for r in range(num_restarts):
    initial_simplex = probabilistic_restart(...)
    current_solution, current_objective = optimize.minimize(initial_simplex)

    if current_objective < best_objective:
        best_solution = current_solution
        best_objective = current_objective

[Sandy4321]

I see thanks
but this key line
initial_simplex = probabilistic_restart(...)
need to sample some grid in space

grid choosing is the art...

[ojdo]

The GBNM way of "sampling the grid" is this (as implemented here):

• Create N (nPoints) uniformly distributed random points in the search space (box between xmin, xmax)
• For each of those random points, determine their "newness" by evaulating gauss(...) (less probable == better).

What gauss does: erect a gaussian probability distribution around the previously visited points (named usedPoints) here. This is a list of all previous initial and final simplex points of the previous restarts. Each such visited point leads to a single gaussian spike in my function. That way, points farer from already visited regions are favoured in a restart, slightly biasing the algorithm towards exploring uncharted regions of the search space.

But yes, this algorithm is very parameterisable (e.g. sigmainv for the width of a single gaussian spike, that means how far an already visited point's location pushes into the search space).