Some tests fail

[yurivict]

Some tests fail:

Available solvers: ['GLPK_CMD', 'PULP_CBC_CMD', 'COIN_CMD', 'SCIP_CMD', 'HiGHS_CMD']
Unavailable solvers: {'PYGLPK', 'COINMP_DLL', 'HiGHS', 'MOSEK', 'FSCIP_CMD', 'SAS94', 'SASCAS', 'COPT_CMD', 'MIPCL_CMD', 'GUROBI', 'CPLEX_CMD', 'SCIP_PY', 'GUROBI_CMD', 'CPLEX_PY', 'CHOCO_CMD', 'XPRESS_PY', 'COPT', 'XPRESS', 'COPT_DLL'}
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss.......................................................Welcome to the CBC MILP Solver 
Version: 2.10.12 
Build Date: Feb 15 2025 

command line - cbc /tmp/adc2a1dc73914582a27938250031fc5f-pulp.mps -timeMode elapsed -branch -printingOptions all -solution /tmp/adc2a1dc73914582a27938250031fc5f-pulp.sol (default strategy 1)
At line 2 NAME          MODEL
At line 3 ROWS
At line 9 COLUMNS
At line 20 RHS
At line 25 BOUNDS
At line 29 ENDATA
Problem MODEL has 4 rows, 4 columns and 7 elements
Coin0008I MODEL read with 0 errors
Option for timeMode changed from cpu to elapsed
Presolve 1 (-3) rows, 2 (-2) columns and 2 (-5) elements
0  Obj 51.9 Primal inf 2.099999 (1)
1  Obj 54
Optimal - objective value 54
After Postsolve, objective 54, infeasibilities - dual 0 (0), primal 0 (0)
Optimal objective 54 - 1 iterations time 0.002, Presolve 0.00
Option for printingOptions changed from normal to all
Total time (CPU seconds):       0.00   (Wallclock seconds):       0.00

....................ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss.......................................................GLPSOL--GLPK LP/MIP Solver 5.0
Parameter(s) specified in the command line:
 --cpxlp /tmp/bfd70f878655411f87d715854fe9d5ce-pulp.lp -o /tmp/bfd70f878655411f87d715854fe9d5ce-pulp.sol
Reading problem data from '/tmp/bfd70f878655411f87d715854fe9d5ce-pulp.lp'...
4 rows, 4 columns, 7 non-zeros
12 lines were read
GLPK Simplex Optimizer 5.0
4 rows, 4 columns, 7 non-zeros
Preprocessing...
1 row, 2 columns, 2 non-zeros
Scaling...
 A: min|aij| =  1.000e+00  max|aij| =  1.000e+00  ratio =  1.000e+00
Problem data seem to be well scaled
Constructing initial basis...
Size of triangular part is 1
      0: obj =   5.000000000e+01 inf =   4.000e+00 (1)
      1: obj =   5.400000000e+01 inf =   0.000e+00 (0)
*     2: obj =   5.400000000e+01 inf =   0.000e+00 (0)
OPTIMAL LP SOLUTION FOUND
Time used:   0.0 secs
Memory used: 0.0 Mb (40400 bytes)
Writing basic solution to '/tmp/bfd70f878655411f87d715854fe9d5ce-pulp.sol'...
....................ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss.......................................................Running HiGHS 1.9.0 (git hash: n/a): Copyright (c) 2024 HiGHS under MIT licence terms
LP   552bddddbb2e4073b86687d716f46c94-pulp has 4 rows; 4 cols; 7 nonzeros
Coefficient ranges:
  Matrix [1e+00, 1e+00]
  Cost   [1e+00, 9e+00]
  Bound  [1e+00, 4e+00]
  RHS    [5e+00, 1e+01]
Presolving model
1 rows, 2 cols, 2 nonzeros  0s
0 rows, 0 cols, 0 nonzeros  0s
Presolve : Reductions: rows 0(-4); columns 0(-4); elements 0(-7) - Reduced to empty
Solving the original LP from the solution after postsolve
Model name          : 552bddddbb2e4073b86687d716f46c94-pulp
Model status        : Optimal
Objective value     :  5.4000000000e+01
Relative P-D gap    :  0.0000000000e+00
HiGHS run time      :          0.00
Writing the solution to /tmp/552bddddbb2e4073b86687d716f46c94-pulp.sol
.............E......sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss.........................................................Welcome to the CBC MILP Solver 
Version: 2.10.12 
Build Date: Feb 15 2025 

command line - /usr/ports/math/py-PuLP/work-py311/pulp-3.0.2/pulp/solverdir/cbc/linux/64/cbc /tmp/5e7bba407237410cb71301263e094874-pulp.mps -timeMode elapsed -branch -printingOptions all -solution /tmp/5e7bba407237410cb71301263e094874-pulp.sol (default strategy 1)
At line 2 NAME          MODEL
At line 3 ROWS
At line 9 COLUMNS
At line 20 RHS
At line 25 BOUNDS
At line 29 ENDATA
Problem MODEL has 4 rows, 4 columns and 7 elements
Coin0008I MODEL read with 0 errors
Option for timeMode changed from cpu to elapsed
Presolve 1 (-3) rows, 2 (-2) columns and 2 (-5) elements
0  Obj 51.9 Primal inf 2.099999 (1)
1  Obj 54
Optimal - objective value 54
After Postsolve, objective 54, infeasibilities - dual 0 (0), primal 0 (0)
Optimal objective 54 - 1 iterations time 0.002, Presolve 0.00
Option for printingOptions changed from normal to all
Total time (CPU seconds):       0.00   (Wallclock seconds):       0.00

......................sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss...................E..........................E.....F..SCIP version 9.2.0 [precision: 8 byte] [memory: block] [mode: optimized] [LP solver: Soplex 7.1.1] [GitHash: 74cea9222e]
Copyright (c) 2002-2024 Zuse Institute Berlin (ZIB)

External libraries: 
  Readline 8.2         GNU library for command line editing (gnu.org/s/readline)
  Soplex 7.1.1         Linear Programming Solver developed at Zuse Institute Berlin (soplex.zib.de) [GitHash: 9a11f87c]
  CppAD 20180000.0     Algorithmic Differentiation of C++ algorithms developed by B. Bell (github.com/coin-or/CppAD)
  ZLIB 1.3.1           General purpose compression library by J. Gailly and M. Adler (zlib.net)
  GMP 6.3.0            GNU Multiple Precision Arithmetic Library developed by T. Granlund (gmplib.org)
  AMPL/MP 690e9e7      AMPL .nl file reader library (github.com/ampl/mp)
  Nauty 2.8.8          Computing Graph Automorphism Groups by Brendan D. McKay (users.cecs.anu.edu.au/~bdm/nauty)
  sassy 1.1            Symmetry preprocessor by Markus Anders (github.com/markusa4/sassy)
  Ipopt 3.14.17        Interior Point Optimizer developed by A. Waechter et.al. (github.com/coin-or/Ipopt)

reading user parameter file </tmp/f3cbef66a2d04c5a8868432c85a8acee-pulp.set>


read problem </tmp/f3cbef66a2d04c5a8868432c85a8acee-pulp.lp>
============

original problem has 4 variables (0 bin, 0 int, 0 impl, 4 cont) and 4 constraints

presolving:
(round 1, fast)       1 del vars, 3 del conss, 0 add conss, 2 chg bounds, 0 chg sides, 0 chg coeffs, 0 upgd conss, 0 impls, 0 clqs
(round 2, fast)       3 del vars, 4 del conss, 0 add conss, 3 chg bounds, 0 chg sides, 0 chg coeffs, 0 upgd conss, 0 impls, 0 clqs
presolving (3 rounds: 3 fast, 1 medium, 1 exhaustive):
 4 deleted vars, 4 deleted constraints, 0 added constraints, 3 tightened bounds, 0 added holes, 0 changed sides, 0 changed coefficients
 0 implications, 0 cliques
transformed 1/1 original solutions to the transformed problem space
Presolving Time: 0.00

SCIP Status        : problem is solved [optimal solution found]
Solving Time (sec) : 0.00
Solving Nodes      : 0
Primal Bound       : +5.40000000000000e+01 (1 solutions)
Dual Bound         : +5.40000000000000e+01
Gap                : 0.00 %


written solution information to file </tmp/f3cbef66a2d04c5a8868432c85a8acee-pulp.sol>


......../usr/ports/math/py-PuLP/work-py311/pulp-3.0.2/pulp/apis/scip_api.py:138: UserWarning: SCIP_CMD does not allow a problem to be relaxed
  warnings.warn(f"{self.name} does not allow a problem to be relaxed")
............ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss..ssss
======================================================================
ERROR: test_time_limit_no_solution (pulp.tests.test_pulp.HiGHS_CMDTest.test_time_limit_no_solution)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/math/py-PuLP/work-py311/pulp-3.0.2/pulp/tests/test_pulp.py", line 69, in skip_wrapper
    return test_item(test_obj, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/ports/math/py-PuLP/work-py311/pulp-3.0.2/pulp/tests/test_pulp.py", line 1321, in test_time_limit_no_solution
    status = prob.solve(self.solver)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/ports/math/py-PuLP/work-py311/pulp-3.0.2/pulp/pulp.py", line 1996, in solve
    status = solver.actualSolve(self, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/ports/math/py-PuLP/work-py311/pulp-3.0.2/pulp/apis/highs_api.py", line 176, in actualSolve
    model_line = [line for line in lines if "Status" in line][0]
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

======================================================================
ERROR: test_elastic_constraints_penalty_unbounded (pulp.tests.test_pulp.SCIP_CMDTest.test_elastic_constraints_penalty_unbounded)
Test the ability to use Elastic constraints (penalty unbounded)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/math/py-PuLP/work-py311/pulp-3.0.2/pulp/tests/test_pulp.py", line 839, in test_elastic_constraints_penalty_unbounded
    pulpTestCheck(prob, self.solver, [const.LpStatusNotSolved])
  File "/usr/ports/math/py-PuLP/work-py311/pulp-3.0.2/pulp/tests/test_pulp.py", line 2078, in pulpTestCheck
    raise PulpError(
pulp.constants.PulpError: Tests failed for solver <pulp.apis.scip_api.SCIP_CMD object at 0xa2455195b10>:
status == -2 not in [0]
status == Unbounded not in ['Not Solved']

======================================================================
ERROR: test_invalid_var_names (pulp.tests.test_pulp.SCIP_CMDTest.test_invalid_var_names)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/math/py-PuLP/work-py311/pulp-3.0.2/pulp/tests/test_pulp.py", line 1340, in test_invalid_var_names
    pulpTestCheck(
  File "/usr/ports/math/py-PuLP/work-py311/pulp-3.0.2/pulp/tests/test_pulp.py", line 2091, in pulpTestCheck
    raise PulpError(
pulp.constants.PulpError: Tests failed for solver <pulp.apis.scip_api.SCIP_CMD object at 0xa24547147d0>:
var a == 0.0 != 4

======================================================================
FAIL: test_measuring_solving_time (pulp.tests.test_pulp.SCIP_CMDTest.test_measuring_solving_time)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/math/py-PuLP/work-py311/pulp-3.0.2/pulp/tests/test_pulp.py", line 69, in skip_wrapper
    return test_item(test_obj, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/ports/math/py-PuLP/work-py311/pulp-3.0.2/pulp/tests/test_pulp.py", line 1307, in test_measuring_solving_time
    self.assertEqual(status, const.LpStatusOptimal)
AssertionError: 0 != 1

----------------------------------------------------------------------
Ran 1737 tests in 249.950s

FAILED (failures=1, errors=3, skipped=1356)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/ports/math/py-PuLP/work-py311/pulp-3.0.2/pulp/tests/run_tests.py", line 17, in pulpTestAll
    raise pulp.PulpError("Tests Failed")
pulp.constants.PulpError: Tests Failed
*** Error code 1

No final test report is produced.

Version: 3.0.2
Python-3.11
FreeBSD 14.2

[MBradbury]

I thought that this might be due to me. However, the following test fails on c06fe08:

• SCIP_PYTest.test_elastic_constraints_penalty_unbounded

Might this one just be a test error? Should the tests be checking for LpStatusUnbounded instead of LpStatusNotSolved?

The following tests pass on both 3.0.2 and c06fe08:

• SCIP_PYTest.test_measuring_solving_time
• SCIP_PYTest.test_invalid_var_names
• HiGHS_PYTest.test_time_limit_no_solution

Any instructions on running the _CMD variants would be appreciated.

I spent a bit of time trying to understand FixedElasticSubProblem, so sent a PR with those changes #800.

[pchtsp]

@yurivict could you give us a set of instructions to download and try SCIP_CMD with pulp?
I remember the SCIP team required registering with an account and fill an online form before downloading the solver. I hope that's changed.
I'd really like to have it installed as part of our CI/CD workflow.

[MBradbury]

Added #801 to address SCIP_PYTest.test_elastic_constraints_penalty_unbounded.

[yurivict]

could you give us a set of instructions to download and try SCIP_CMD with pulp?

Here is the URL: https://www.scipopt.org/download/release/scip-9.2.0.tgz

SCIP license was changed to Apache 2.0 awhile ago.