OpenOpt

the blog is moved to new location

2008-12-15T14:20:00.000-08:00

update your rss feeds if you want to get the posts furthermore.
New location:
http://forum.openopt.org/viewforum.php?id=19

OpenOpt (and my) Future

2008-12-15T13:27:00.000-08:00

My interim in icyb.kiev.ua optimization dept is up to be finished.

Today I have spoken with my chiefs from about my further career.
They have informed me: because of financial crisis the situation is undefined.

Still I have some chances. Moreover, there are some chances I will be permitted to spent some work time for further openopt development.
However, it is provided some conditions to be satisfied.

Some days ago I had been forced to move my code out of scikits framework, my chiefs want me to host it inside Ukraine under our control (mb you know I had been refused to obtain veto rule for openopt code changes within scikits framework; I understand scipy community - they can't take taht something is hosted inside their server and is out of control, but I understand my chiefs position as well).

I have contacted uafoss.org (FOSS Ukraine dept), and they gladly allowed me to host it here.
Full transportation (svn repository, doc, wiki etc) will be finished within several days. But numerical optimization forum already works, you are welcome: http://forum.openopt.org. I had noticed openopt.com and openopt.net are already taken by someone, so it's definitely correct time to go.

Thanks to Michailo Danilenko and Volodimir M. Lisivka from linux.org.ua (LOU) community.
Special thanks to Wadim V. Mashkov and Michael Shigorin from linux.kiev.ua community.

OpenOpt release 0.21

2008-12-15T12:50:00.000-08:00

Hi all,
I'm glad to inform you about new OpenOpt release: v 0.21.

Changes since previous release 0.19 (June 15, 2008):

major changes for NLP / NSP solver ralg
new converter: qp2nlp
new converter: lp2nlp
new converter: nllsp2nlp
new converter: nlsp2nlp
new converter: minimax to NLP
Introducing OpenOpt GUI
PSwarm 1.3 has been connected instead of v 1.1. It has much more parameters to modify and ability to use user-supplied callback function.
Lots of changes related to oofun: add oovar and oolin, now fixed oovars can be correctly handled
enhanced iterfcn connection for scipy fmin_cobyla
some doc updates for result structure
new openopt API func: oosolver
some code cleanup and bugfixes
some other minor changes

Backward incompatibility:

instead of "from scikits.openopt" now you should use "from openopt import ..."
LSP has been renamed to NLLSP
oofun with ordinary variables (x) support had been ceased (it's hard to keep maintaining), use oovars instead.

Until OpenOpt subversion repository will be finally moved to new host, you can download v 0.21 from here

Welcome to http://forum.openopt.org - new forum about numerical optimization and related free and open source software.

For assist with new host and forum
Thanks to Michailo Danilenko and Volodimir M. Lisivka from linux.org.ua (aka LOU) community.
Special thanks to Wadim V. Mashkov and Michael Shigorin from linux.kiev.ua community.

Regards, Dmitrey.

Some changes for oofun

2008-12-15T05:51:00.000-08:00

Some changes for oofun:

oolin now can handle matrices: oolin(C) creates oofun that returns numpy.dot(C, inp_array); oolin(C, d) yields oofun that returns numpy.dot(C, x) + d. See updated oolin example.
add fixed variables handling for oovars. Now you can declare v.fixed = True or just v = oovar(..., fixed = True). So all oofuns that recursively depend on fixed oovars only will be calculated only once, and derivatives will be all-zero (hence no calculations will be required for those parts of code that depend on fixed oofuns only). In future it will be good to have inner fixed coords for oovars, like this: v.fixed = [0, 1, 15] (i.e. positions of fixed coords inside the oovar). See the fixed oovars example.

However, let me remember you that currently oolin(anything) is considered as non-linear code, i.e. it goes to non-linear constraints and objective function only. In future it should be implemented oolin(some_oovars) -> general linear constraints (Ax <= b, Aeq x = beq). As for lb <= x <= ub, they are handled correctly from oovar fields.

some changes

2008-12-06T01:48:00.001-08:00

I have committed some changes to OO Kernel, most important is a fix reducing time for connecting oovars to prob instance (the recursive function throw all oofuncs took too much time previously).

Ironclad v0.7 released (NumPy on IronPython)

2008-11-27T09:59:00.000-08:00

IronClad developers have announced release v 0.7.
I guess it makes possible to use OO and some solvers (like ralg) from IronPython.

CorePy: Assembly Programming in Python

2008-11-25T00:08:00.000-08:00

I've got to know about BSD-licensed v 1.0 release of CorePy - "a Python package for developing assembly-level applications on x86, Cell BE and PowerPC processors".

I guess it would be useful for those objective or non-linear functions that are required to be evaluated sufficiently faster than pure Python-coded.

Of course, using C, C++, Fortran code via Cython, f2py, ctypes, SWIG, Pyrex etc could yield some speedup as well.

new openopt API func: oosolver

2008-11-19T02:29:00.000-08:00

I have committed entry for oosolver (that has been recently implemented) into OO Doc page.

changes for ralg linear constraints

2008-11-18T07:44:00.000-08:00

I have committed some changes for general linear constraints (A x <= b, Aeq x = beq) handling by NLP/NSP ralg solver.

The changes are essential for len(b) >> 1 or len(beq) >> 1 only.

bugfix for nonlinear group + changes for ralg

2008-11-18T03:20:00.000-08:00

I have found and fixed serious bug for non-linear problems group (NLP, NSP etc). Sometimes it has been triggered with some constrained problems and those solvers who can use splitting (ralg, algencan). Still algencan doesn't work essentially better (for those examples I had tried).

Some changes for ralg have been committed (to decrease non-linear inequality constraints evaluation number)

new converter: minimax to NLP

2008-11-17T08:35:00.000-08:00

I have committed the converter along with usage example. Like MATLAB's fminimax and lots of similar MMP solvers, it works via solving NLP

t -> min

subjected to
t >= f0(x)
t >= f1(x)
...
t >= fk(x)

Let me note that the NLP problem obtained is always constrained (in addition to constraints lb, ub, A, Aeq, c, h from original mmp we get new non-linear inequality constraints written above).

some doc updates for result structure

2008-11-16T03:04:00.000-08:00

I have updated the doc page ResultStruct with description of

negative values of r.evals['df'], r.evals['dc'], r.evals['dh'] (it means they have been obtained via finite-difference approximation; in the case we take into account for r.evals['f'] all f calls - both from objFunc and from finite-difference derivatives approximation)
r.iterValues.rt, r.iterValues.ri (type and index of max residual).

enhanced iterfcn connection for scipy fmin_cobyla

2008-11-15T12:10:00.000-08:00

I have connected changes related to handling of scipy_cobyla iterfcn, so now instead of direct line (as before) it looks like this:

Of course, adequate text output is provided as well:


solver: scipy_cobyla   problem: unnamed   goal: minimum
iter    objFunVal    log10(maxResidual)
 0  6.115e+01               2.13
10  2.015e+01              -2.82
20  2.029e+01              -6.46
30  2.030e+01              -7.60
40  2.032e+01              -8.70
50  2.032e+01             -10.30
60  2.033e+01              -9.78
70  2.033e+01             -13.41
80  2.033e+01             -15.58
90  2.033e+01             -12.50
96  2.033e+01             -21.03
istop:  1000
Solver:   Time Elapsed = 0.72     CPU Time Elapsed = 0.69
Plotting: Time Elapsed = 6.72     CPU Time Elapsed = 5.31
objFunValue: 20.329368 (feasible, max constraint =  9.3314e-22)

Also, fEnough, maxTime, maxCPUTime and some other stop criteria work (for scipy_cobyla).

Initially users had to connect iterfcn by themselves (to df, dc, dh etc), then it had been done automatically by default to df, now (with latest changes) it is for f and df only and automatically.

Now NLP instance has parameter f_iter (default is max(nVars,4)), so when number of called objective function exceeds p.f_iter (for scipy_cobyla and other solvers w/o iterfcn connected and gradient using), OO iterfcn function is called (and hence user-supplied callback functions(s) if any are declared).

Also, for those solvers who hasn't native connection to OO iterfcn and use derivatives (algencan, ipopt, scipy_slsqp, some unconstrained and box-bounded ones) there is parameter df_iter, default True (use iterfcn each call for df); if positive integer s(s>1, 1 is same to default True) - use iterfcn each s-th objective function gradient call.

Mb I'll change "f_iter" and "df_iter" to more appropriate field names till next OO release.

Any Toronto openopt users?

2008-11-12T08:07:00.000-08:00

hi all,
if anyone from Toronto is OO user (I had noticed some in webcounters) it would be nice would you mention OO in (imaginary) "Million Dollar Python Project" (link) by PyGTA (Toronto Python User's Group on Tuesday)

OpenOpt in Debian packages

2008-11-09T06:19:00.000-08:00

Yaroslav Halchenko from PyMVPA project (that uses OpenOpt) has organized python-scikits-openopt deb package and put it into Debian Linux repository.

However, I don't know how stable it is (I haven't tried/tested it yet and will hardly do in nearest future, currently I'm busy because of many other urgent things to be done, my dept chiefs require that ones).

some changes & bugfixes

2008-11-09T04:28:00.000-08:00

I have committed

some changes for ralg
some bugfixes for oofun-oovar
some other changes

Major changes for ralg

2008-10-28T13:05:00.000-07:00

I have committed some major changes for NLP/NSP solver ralg.

A couple of ideas have been stolen from SolvOpt (made by our dept worker Alex Kuntsevich during his voyage to Austria, in collaboration with Franz Kappel), that still remains to be most famous Naum Z. Shor r-algorithm implementation with Fortran90, C and MATLAB API.

However, the implementation is more heuristic than original Naum Z. Shor r-algorithm; since convergence of latter hasn't been proved yet even for smooth convex functions, proving convergence for SolvOpt's implementation will hardly be ever possible (especially because of those heuristics like backward line search approach, see SolvOpt's paper for algorithm details).

So my dept chief is not fond of SolvOpt heuristics (he consider they reduce % of problems that can be solved, because SolvOpt can stop further from optimum, then primal Naum Z. Shor r-algorithm), still SolvOpt remains rather popular in our optimization dept.

Currently I have taken 2 ideas from SolvOpt: modifications for initial step size and step multiplier for forward line search. As for backward search, I found SolvOpt's approach very unclear and currently my own way still remains in OO ralg implementation. Handling of constraints is also performed in a different way.

Also, some code cleanup for OO kernel have been committed.

PSwarm 1.3 has been connected

2008-10-21T06:21:00.000-07:00

I have connected new PSwarm version: 1.3.

As PSwarm developer Ismael Vaz promised, now Python API has much more available parameters to modify (see updated GLP page for details) and outputfcn connected (that improves text and graphics output), so now user-provided callback function(s) can be handled by the solver (also, openopt GUI p.manage() function works better, see previous post for description).

Introducing OpenOpt GUI

2008-10-15T10:54:00.000-07:00

I'm glad to introduce initial version of OpenOpt GUI.

I had some months of GUI experience in PyGTK and tcl/tk so I have some experience in GUI development; on the other hand I know very well what's the deuce it is: you have to try all possible combinations of buttons (and the dummy task is NP-Hard of course) to ensure all works ok, to enable/disable buttons in correct order, to story related variables etc.

For OO I decided to use Tkinter, because it doesn't require any additional soft installation (it is included in Python distribution).

I had took a look at AMPL and GAMS GUIs before starting my own one, and of course I have no intention to duplicate Python IDEs.

Running OO GUI is performed via

r = p.manage()

instead of p.solve(), or via

from scikits.openopt import manage
r = manage(p, solver, ...)

Let me also note: manage() can handle named argument start = {False}/True/0/1, that means start w/o waiting for user-pressed "Run".

Currently there are only 3 buttons: "Run/Pause", "Exit" and "Enough".
As you probably know lots of solvers have troubles with stop criteria. Especially it's relevant to NSP solvers, where calculating derivatives only for to check stop criteria isn't a good idea. So pressing "Enough" button yields triggering of stop criterion, like this:
------------
solver: ralg problem: GUI_example goal: minimum
iter objFunVal log10(maxResidual)
...
102 5.542e+01 -6.10
istop: 88 (button Enough has been pressed)
Solver: Time Elapsed = 1.19 CPU Time Elapsed = 1.12
Plotting: Time Elapsed = 6.86 CPU Time Elapsed = 5.97
objFunValue: 55.423444 (feasible, max constraint = 7.98936e-07)

Let me also note that

pressing "Exit" before "Enough" and before solver finish will return None, so there will be no fields r.ff, r.xf etc.
for some IDEs pressing "Exit" doesn't close matplotlib window (if you are using p.plot=1). You should either wait for newer matplotlib version (they intend to fix it) or try to fix it by yourself via choosing correct Agg, see here for details
OO Doc webpage entry related to OO GUI will be committed later
you could play with the GUI example by yourself

bugfix for ooIterPrint.py in Python 2.4

2008-10-11T04:24:00.001-07:00

I have been informed of Python 2.4 issue
File
"/usr/lib/python2.4/site-packages/scikits/openopt/Kernel/ooIterPrint.py",
line 6
'isFeasible': lambda p: ('+' if p.rk < p.contol else
'-')
^
SyntaxError: invalid syntax

I have committed workaround for Python 2.4. Still I guess it's better to use Python 2.5, as it is mentioned in OO install webpage.

Example of 10X speedup for NLP via oofun

2008-10-10T06:57:00.000-07:00

I have committed the example of unconstrained NLP where using oofun yields 10X speedup vs classic style (via reducing of the time required for gradient approximation for a costly function).

Of course,

I could construct example where the profit is 100X or even more, but it's longer to wait for output.
There can be similar examples constructed for constrained NLP or some other classes from non-linear group (NSP, NLSP, LSP).
I used scipy_ncg solver but speedup ~ 4X...10X is observed while using any other non-linear solver as well (provided the solver deals with at least 1st derivatives, so scipy_cobyla, goldenSection, scipy_fminbound, scipy_powell or GLP solvers are inappropriate).

Also, some changes to oofun-, oovar-related and some other OO Kernel files have been committed.

oovar and oolin

2008-10-07T05:03:00.000-07:00

I have committed my implementation of oovar and oolin to OpenOpt svn repository.

So, 2 new entries to oofun doc webpage have been added:

using oovar (OpenOpt Variable)
using oolin (that is linear oofun)

Currently oolin(f) returns (f, x) (that is numpy.dot). In future I intend to add c: oolin(f, c) is (f, x) + c.
Also, mb ooquad(H, f, c) will be implemented in future (that is 0.5 * x^THx + fx + c).

new Python optimization soft: Pyomo

2008-10-03T01:04:00.000-07:00

I've got to know about Pyomo (thanks to Nils Wagner) that is made by Sandia Labs (Trilinos/PyTrilinos developers). In the article they say "However, Coopr Opt is not as mature as the OpenOpt package", I guess that's the reason why my site daily visitors number has jumped from ~ 50-60 to ~ 80-90 this week.

Also, in the paper they inform of willing to collaborate with OO. I had contacted them long time ago but the provided examples of their soft usage were too difficult for me to understand and connect to OO during an adequate time. I guess it will be much easier for them to understand my code instead.

Very important bugfix for nonlinear problems

2008-10-03T00:55:00.000-07:00

I have committed very important bugfix for nonlinear problems.
I don't know how many days the nasty bug was hiding and lurkering (I guess several months), did the one affects some solvers work or no, because it has been somehow (but maybe not everytime) overwritten by other code, and only my latest changes (committed to subversion repository some days ago) have revealed the one.

Changes for converters

2008-09-29T12:33:00.000-07:00

I have committed lots of changes for (all) converters; also, the issue with splitted funcs (for nlps2nlp and lsp2nlp) has been fixed.