Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Best solution (f-value and params) printed as best by CMASolutions object differs from best seen solution #199

Closed
kostayScr opened this issue Feb 9, 2019 · 2 comments

Comments

@kostayScr
Copy link

kostayScr commented Feb 9, 2019

When doing (or using cmasols.print()):
std::stringstream s; s << cmasols << std::endl;
Example output is:
best solution => f-value=-7.15984 / fevals=1404 / sigma=39.6261 / iter=234 / elaps=516809ms / x=4.77428 9.80868
That is not correct though.
Actual best for this example:
Correct f-vlaue: -8.03673
The output is pretty random, because it outputs Current best solution, not the best seen candidate.
The bug is in cmasolutions.cc, line 265:
out << "best solution => f-value=" << best_candidate().get_fvalue() << " / fevals=" << _nevals << " / sigma=" << _sigma << " / iter=" << _niter << " / elaps=" << _elapsed_time << "ms" << " / x=" << gp.pheno(best_candidate().get_x_dvec()).transpose();
The used best_candidate() member function is wrong. Must be get_best_seen_candidate(). The correct results in the example were obtained using it.
The current output is very random, can be heavily different from real best value i.e. you m ay think that it doesn't converge at all with this lambda, while it converged nicely.

@beniz beniz changed the title Critical bug: Wrong solution (f-value and params) printed as best when printing CMASolutions Final solution (f-value and params) printed as best by CMASolutions object differs from best seen solution Feb 9, 2019
@beniz beniz changed the title Final solution (f-value and params) printed as best by CMASolutions object differs from best seen solution Best solution (f-value and params) printed as best by CMASolutions object differs from best seen solution Feb 9, 2019
@beniz
Copy link
Collaborator

beniz commented Feb 9, 2019

Hi, this is not a bug, the best (or final) value is that of the current (or final) iteration, to get best value over the full run, the call to get_best_seen_candidate() is the right way to do it, as you are mentioning.

This comment might help you as well, #195 (comment)

@kostayScr
Copy link
Author

kostayScr commented Feb 11, 2019

Ok but the description is heavily misleading then.
It says "best solution" but it's not the best solution that has been found..
These solutions can often heavily differentiate e.g. -3 - which was good enough and +6, as well as an entirely different parameters values region.
I'm not the only one confused by it , as you saw in the #195.
Btw would be nice to catch some dumb errors like this:
GenoPheno<pwqBoundStrategy, linScalingStrategy> gp( mins.data(), maxes.data(), dim ); CMAParameters<decltype(gp)> cmaparams( x0, sigma, lambda, 0, gp ); CMASolutions cmasols = cmaes<>( optimFn, cmaparams );// <-- no decltype(gp) given, but compiles without a hitch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants