Usage

Usage#

The sweepexp library provides a straightforward way to run experiments with multiple parameter combinations. This tutorial demonstrate its ease of use, focusing on the basic functionality. I.e. showcase how to define an experiment, run it sequentially, or switch to parallel execution with minimal changes.

Defining Your Experiment#

The first step is to define the custom function that you want to evaluate for different parameter combinations. Here’s an example function:

def my_custom_experiment(x: float, y: float) -> dict:
    """Add and multiply two numbers."""
    return {"addition": x + y, "multiplication": x * y}

This simple function adds and multiplies two input numbers, but it could represent any custom computation, including more complex or expensive tasks, such as training a machine learning model or running a simulation. The input and output values can be customized to fit your specific use case. Important is that the function returns a dictionary with the results.

Setting Up the Sweep#

Next, we create a sweepexp object that will run the experiment for all parameter combinations. We pass the function and the parameters we want to sweep over. Here’s how to set up the sweep for the example function above:

from sweepexp import sweepexp

sweep = sweepexp(
    func = my_custom_experiment,
    parameters = { "x": [1, 2], "y": [3, 4, 5] },
)

The parameters dictionary contains the parameter names as keys and lists of values to test as values. The func parameter is the function to run. In this case, we want to test the “x” parameter with two values (1 and 2) and the “y” parameter with three values (3, 4, and 5). This makes a total of 6 experiments to run. Let’s now run the experiments and print the results:

sweep.run()

print(sweep.data)

The run method executes the experiments and stores the results in the data attribute of the SweepExp object. The data attribute is an xarray Dataset that contains the results of the experiments. The output will look like this:

<xarray.Dataset> Size: 160B
Dimensions:         (x: 2, y: 3)
Coordinates:
  * x               (x) int64 16B 1 2
  * y               (y) int64 24B 3 4 5
Data variables:
    status          (x, y) <U1 24B 'C' 'C' 'C' 'C' 'C' 'C'
    addition        (x, y) float64 48B 4.0 5.0 6.0 5.0 6.0 7.0
    multiplication  (x, y) float64 48B 3.0 4.0 5.0 6.0 8.0 10.0

Parallel Execution using multiprocessing or MPI#

Since the experiments are independent, they can be run in parallel to speed up the process. The sweepexp library provides two methods for parallel execution: - mode=”parallel” for multiprocessing - mode=”mpi” for MPI (high-performance computing) The following example demonstrates how to run the same experiment as above in the three different modes: sequentially, in parallel with multiprocessing, and in parallel with MPI.

Sequentially

from sweepexp import sweepexp


def my_custom_experiment(x: float, y: float) -> dict:
    """Add and multiply two numbers."""
    return {"addition": x + y, "multiplication": x * y}

sweep = sweepexp(
    func = my_custom_experiment,
    parameters = { "x": [1, 2], "y": [3, 4, 5] },
    mode = "sequential",  # Default mode
)

sweep.run()

Parallel (multiprocessing)

from sweepexp import sweepexp


def my_custom_experiment(x: float, y: float) -> dict:
    """Add and multiply two numbers."""
    return {"addition": x + y, "multiplication": x * y}

sweep = sweepexp(
    func = my_custom_experiment,
    parameters = { "x": [1, 2], "y": [3, 4, 5] },
    mode = "parallel",  # Use parallel mode
)

sweep.run()  # Set max_workers=... to control the number of parallel workers

Parallel (MPI)

from sweepexp import sweepexp


def my_custom_experiment(x: float, y: float) -> dict:
    """Add and multiply two numbers."""
    return {"addition": x + y, "multiplication": x * y}

sweep = sweepexp(
    func = my_custom_experiment,
    parameters = { "x": [1, 2], "y": [3, 4, 5] },
    mode = "mpi",  # Use MPI mode
)

sweep.run()

mpiexec -l -n 4 python my_script.py

For more advanced features, such as measuring the duration of experiments, marking experiments to be skipped, or optimizing the execution order, check out the other tutorials: