SLSQP search stopped too early on tutorial constraints #514
Conversation
SLSQP was giving the wrong answer on the tutorial in 't_python.py': ``` optimum at [0.50777849 4.76252907] minimum value = 2.182321944309753 ``` instead of: ``` optimum at [0.33333333 0.29629629] minimum value = 0.5443310523133087 ``` Slightly tweaking the function (e.g. to use repeated multiplication instead of pow), or increasing xtol_rel to 1e-9 made it find the answer. But the bogus optimum and real optimum are different by more than xtol_rel. Printing the function evaluations show that it reaches xtol_rel at an infeasible point, and then returns the last feasible point: ``` f(0.337085195789626, 0.287031459045266)=0.535753169888211 c(0.337085195789626, 0.287031459045266)=0.019382838080164 c(0.337085195789626, 0.287031459045266)=0.004290454106768 f(0.337085195789626, 0.287031459045266)=0.535753169888211 c(0.337085195789626, 0.287031459045266)=0.019382838080164 c(0.337085195789626, 0.287031459045266)=0.004290454106768 optimum at [0.50777849 4.76252907] minimum value = 2.182321944309753 result code = 4 nevals = 22 ``` Fix this by checking xtol_rel on mode==-1 only if the current point is feasible, not when any previous point was feasible (which matches how the original SLSQP's stopping condition was, and how the other xtol_rel check is done in this same function). Now the search continues and finds the correct answer: ``` f(0.337085195789626, 0.287031459045266)=0.535753169888211 c(0.337085195789626, 0.287031459045266)=0.019382838080164 c(0.337085195789626, 0.287031459045266)=0.004290454106768 f(0.298264335607084, 0.000001000000000)=0.001000000000000 c(0.298264335607084, 0.000001000000000)=0.212271613303731 c(0.298264335607084, 0.000001000000000)=0.345556758201188 [...] f(0.333333334001826, 0.296296294513450)=0.544331052314168 c(0.333333334001826, 0.296296294513450)=0.000000003565495 c(0.333333334001826, 0.296296294513450)=0.000000000891522 optimum at [0.33333333 0.29629629] minimum value = 0.5443310523133087 result code = 4 nevals = 48 ``` Fixes: 42c43f3 ("Only exit SLSQP successfully if solution is feasible (stevengj#465)") Signed-off-by: Edwin Török <[email protected]>
Only disable tests on those that require subalgorithms. Signed-off-by: Edwin Török <[email protected]>
Signed-off-by: Edwin Török <[email protected]>
Signed-off-by: Edwin Török <[email protected]>
|
LGTM. |
|
some tests fail |
|
Interesting it fails only on windows but not Linux. I'll take a look, but perhaps for now I'll mark those tests as skipped and fix the problems in another PR (those algorithms were not being tested at all previously I think). |
|
I can reproduce one of the failures on Linux: the starting point is randomly generated and some starting points cause the algorithm to fail with I'll remove algorithms 30 and 31 from the tests so far, it looks like their algorithms aren't quite as numerically stable as the other ones. |
Note that before the following commit these were disabled: a6ceab9 Sometimes these algorithms fail depending on the (random) starting point with NLOPT_ROUNDOFF_LIMITED. Signed-off-by: Edwin Török <[email protected]>
You aren't passing any stopping tolerance to That being said, the tests should probably also pass |
|
Still failing on Windows 😢 |
|
Sad that this is still stuck after more than a year. SLSQP is unusable with any good stopping criteria (i.e., not just maximum number of iterations or time) due to this bug. The fix is failing tests, but the code in 2.8.0 is clearly incorrect and only passes the tests by accident. (In fact, the claimed SLSQP fix in 2.8.0 basically does not work due to the typo fixed by this PR.) |
|
SLSQP patched with commit c168a7b works great for me. SLSQP without the patch just plain does not work if you use any |
|
Looks like disabling algo_index 32 in addition to 30 and 31 should make the tests pass. |
|
Checking the enum, algorithms 30 to 33 are the AUGLAG algorithms: so the test failure is not even in SLSQP (the wrapped algorithm is not SLSQP either, but AugLag is known to have issues, see, e.g., #538, but that should be addressed in that issue and not blocking an SLSQP fix. Commit d4794f4 disables It is sad that 2.8.0 shipped without this SLSQP fix, which was submitted more than a year before the 2.8.0 release, all the more because 2.8.0 claims to fix SLSQP, but the fix is ineffective without this followup fix. But we cannot change the past. What we need now is a quick 2.8.1 release with this fix. Since the original submitter has not done it, do you want me to submit a cleaned-up pull request? a) One with only the SLSQP fix? Or b) one with the additional enabled tests, but all 4 AUGLAG tests (not just 2 of them) disabled until further notice? |
|
Or c) with the SLSQP fix (as in a) + enabling only SLSQP (should be algo_index 40 if I counted correctly) in the testsuite, not any other unrelated algorithms. Please let me know what your preferences are so I can submit a PR. |
|
I integrated your slsqp fix + the python test update but not the c tests part which is still failling, It could be good to publish a new release with this fix /cc @stevengj |
|
I also integrated the c++ test part, closing |
I noticed that the python and C++ tutorials gave different answers when used with SLSQP:
SLSQP was giving the wrong answer on the tutorial in 't_python.py':
optimum at [0.50777849 4.76252907] minimum value = 2.182321944309753instead of:
optimum at [0.33333333 0.29629629] minimum value = 0.5443310523133087They are implemented slightly differently, using float multiplication vs
** 2and** 3, which result in slightly different floating point values, however that shouldn't cause SLSQP to fail this badly.The problem isn't with the Python wrapper, because if I reimplement the same function for 'SciPy' it gives the correct answer.
After printing the individual 'f' evaluations I noticed that it stopped the search due to hitting 'xtol_rel', but the function was infeasible at that point (so it returned the last feasible one, which is way off the optimal value).
If I instead let the search continue until it finds a feasible point again then it finds the correct answer.
The problem was hinted to at #368 (comment), which mentions that perhaps
feasible_curshould be used, and I think as this example shows that would be the correct approach(the original SLSQP fortran function also had a check for 'h3', which looks similar to the feasibility check done here).
While fixing the bug I also tweaked the tutorial function to accept the algorithm as parameter and run this during
make testto check a few algorithms (those that accept inequality constraints and infinite bounds).