Optimization failed. Call limit was reached in pyhf.infer.mle.fit

[ssaha1234]

Dear pyhf developers,
I am running an mle fit using pyhf.infer.mle.fit on my model. I have attached my script here - https://gist.github.com/ssaha1234/ee7ef8a6a5d6355b2bd24a3d1cd12c2f

The error I am getting is similar to #1577, as below -

File "/afs/cern.ch/user/s/ssaha/.local/lib/python3.8/site-packages/pyhf/optimize/mixins.py", line 61, in _internal_minimize
    assert result.success
AssertionError
Traceback (most recent call last):
  File "/afs/cern.ch/user/s/ssaha/.local/lib/python3.8/site-packages/pyhf/optimize/mixins.py", line 61, in _internal_minimize
    assert result.success
AssertionError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):

  File "asimovfit.py", line 20, in <module>
    result, result_obj = pyhf.infer.mle.fit(
  File "/afs/cern.ch/user/s/ssaha/.local/lib/python3.8/site-packages/pyhf/infer/mle.py", line 131, in fit
    return opt.minimize(
  File "/afs/cern.ch/user/s/ssaha/.local/lib/python3.8/site-packages/pyhf/optimize/mixins.py", line 184, in minimize
    result = self._internal_minimize(
  File "/afs/cern.ch/user/s/ssaha/.local/lib/python3.8/site-packages/pyhf/optimize/mixins.py", line 64, in _internal_minimize
    raise exceptions.FailedMinimization(result)
pyhf.exceptions.FailedMinimization: Optimization failed. Call limit was reached.

I have followed the comments on #1577, about histosys, which are absolute values and not relative to the nominal values. Following is the output for python -m pip freeze: https://gist.github.com/ssaha1234/1674dbb5eb60670d1688cf39b9ab76cb

I am going to try with setting the maximum iteration now to see if I get successful fit.
If you would have any comments on this error, please let me know.
Thanks !
Best,
Shreya

[alexander-held]

Hi Shreya, increasing the number of iterations via maxiter should resolve this issue (as in #1577 (comment)). I see that you are using MINUIT, you can also try out different fit strategies (strategy) or increase your tolerance (tolerance) to potentially trade precision for less steps to convergence.

[ssaha1234]

Hi @alexander-held ,
Thanks a lot for the suggestions. I would implement the same and let you know how it goes ! By changing the strategy do you mean using something other that MINUIT ?
Best,
Shreya

[alexander-held]

No, I mean strategy as a parameter of MINUIT (https://iminuit.readthedocs.io/en/stable/reference.html#iminuit.Minuit.strategy). The defaults in pyhf are 0 if you're providing gradients and 1 otherwise. You are not using automatic gradients in your script, so 0 is likely a bit faster but less precise than the default of 1.

[ssaha1234]

I see, thanks for the information and the link. I will try it out as well.
Best,
Shreya

[ssaha1234]

Hi @alexander-held ,
Just to let you know, the fit worked by adding maxiter=1000000.
Thanks a lot !
-Shreya