Interfacing With Other Languages

Two Interfacing Strategies

There are basically two ways to interface hyperopt with other languages: you can write a Python wrapper around your cost function that is not written in Python, or you can replace the hyperopt-mongo-worker program and communicate with MongoDB directly using JSON.

Wrapping a call to non-Python code

The easiest way to use hyperopt to optimize the arguments to a non-python function, such as for example an external executable, is to write a Python function wrapper around that external executable. Supposing you have an executable foo that takes an integer command-line argument --n and prints out a score, you might wrap it like this:

import subprocess
def foo_wrapper(n):
    # Optional: write out a script for the external executable
    # (we just call foo with the argument proposed by hyperopt)
    proc = subprocess.Popen(['foo', '--n', n], stdout=subprocess.PIPE)
    proc_out, proc_err = proc.communicate()
    # <you might have to do some more elaborate parsing of foo's output here>
    score = float(proc_out)
    return score

Of course, to optimize the n argument to foo you also need to call hyperopt.fmin, and define the search space. I can only imagine that you will want to do this part in Python.

from hyperopt import fmin, hp, random

best_n = fmin(foo_wrapper, hp.quniform('n', 1, 100, 1), algo=random.suggest)

print best_n

When the search space is larger than the simple one here, you might want or need the wrapper function to translate its argument into some kind of configuration file/script for the external executable.

This approach is perfectly compatible with MongoTrials.

Communicating with MongoDB Directly

Hannes Schultz (@temporaer) got hyperopt working with his MDBQ project, which is a standalone mongodb-based task queue:

https://github.com/temporaer/MDBQ/blob/master/src/example/hyperopt_client.cpp

Have a look at that code, as well as the contents of hyperopt/mongoexp.py to understand how worker processes are expected to reserve jobs in the work queue, and store results back to MongoDB.